emrpy.data

emrpy.data.load_csv(file_path, engine='polars', lazy=False, sample_n=None, **kwargs)

Load a CSV file using pandas or Polars, with optional row-count sampling.

Parameters:

file_pathstr or Path: Path to the CSV file.
engine{‘pandas’, ‘polars’}, default ‘polars’: Which backend to use: - ‘pandas’: calls pd.read_csv - ‘polars’: calls pl.read_csv (eager) or pl.scan_csv (lazy)
lazybool, default False: If True and engine=’polars’, returns a pl.LazyFrame via pl.scan_csv. Ignored when engine=’pandas’.
sample_nint, optional: Number of rows to load. For pandas, passed as nrows; for Polars, as n_rows. If None, loads the entire file.
**kwargs: Passed to the underlying reader.

: pandas.DataFrame or polars.DataFrame or polars.LazyFrame

Loaded (and optionally sampled) table.

>>> # Eager pandas
>>> df = load_csv("data.csv", engine="pandas")
>>> type(df)
<class 'pandas.core.frame.DataFrame'>

>>> # Eager Polars
>>> df = load_csv("data.csv", engine="polars")
>>> type(df)
<class 'polars.internals.frame.DataFrame'>

>>> # Lazy Polars
>>> lf = load_csv("data.csv", engine="polars", lazy=True)
>>> type(lf)
<class 'polars.lazyframe.LazyFrame'>

>>> # Sample first 100 rows with Polars
>>> df = load_csv("data.csv", sample_n=100)
>>> len(df)
100

emrpy.data.load_parquet(file_path, engine='polars', lazy=False, sample_frac=None, sample_n=None, **kwargs)

Load a Parquet file using pandas or Polars, with optional row-count sampling.

file_pathstr or Path: Path to the Parquet file.
engine{‘pandas’, ‘polars’}, default ‘polars’: Which backend to use: - ‘pandas’: calls pd.read_parquet - ‘polars’: calls pl.read_parquet (eager) or pl.scan_parquet (lazy)
lazybool, default False: If True and engine=’polars’, returns a pl.LazyFrame via pl.scan_parquet. Ignored when engine=’pandas’.
sample_nint, optional: Number of rows to load. For pandas, samples after full load; for Polars, as n_rows. If None, loads the entire file.
**kwargs: Passed to the underlying reader.

: pandas.DataFrame or polars.DataFrame or polars.LazyFrame

Loaded (and optionally sampled) table.

>>> # Eager pandas
>>> df = load_parquet("data.parquet", engine="pandas")
>>> type(df)
<class 'pandas.core.frame.DataFrame'>

>>> # Eager Polars
>>> df = load_parquet("data.parquet", engine="polars")
>>> type(df)
<class 'polars.internals.frame.DataFrame'>

>>> # Lazy Polars
>>> lf = load_parquet("data.parquet", engine="polars", lazy=True)
>>> type(lf)
<class 'polars.lazyframe.LazyFrame'>

>>> # Sample first 50 rows with Polars
>>> df = load_parquet("data.parquet", sample_n=50)
>>> len(df)
50

Modules

Data Loading Utilities