Scaling Up with R and Apache Arrow

Analyze large datasets directly from R. Scaling Up With R and Arrow provides a guide to working efficiently with larger-than-memory datasets using the arrow R package. As data grows in size and complexity, traditional data analysis methods in R often hit technical limitations. In this book, you'll learn how to overcome these hurdles without needing to set up complex infrastructure. You'll learn about the Apache Arrow project's origins, goals, and its significance in bridging the gap between data science and big data ecosystems. You'll also learn how to leverage the arrow R package to work directly with files in various formats, such as CSV and Parquet, using familiar dplyr syntax. This book explores practical topics like data manipulation, file formats, working with larger datasets, and optimizing workflows for data in cloud storage. Advanced chapters examine user-defined functions, integration with other tools like DuckDB, and extending Arrow's capabilities to work with geospatial data. Written by developers of the Arrow R package, this guide is essential for anyone looking to scale their data processing capabilities in R.