bw_processing.io_parquet_helpers
================================

.. py:module:: bw_processing.io_parquet_helpers

.. autoapi-nested-parse::

   This module contains some helpers to serialize/deserialize `numpy.ndarray` objects to/from Apache `parquet` files.
   We convert the `nympy.ndarray` objects to `pyarrow.Table` objects to do so.



Functions
---------

.. autoapisummary::

   bw_processing.io_parquet_helpers.load_ndarray_from_parquet
   bw_processing.io_parquet_helpers.read_parquet_file_to_ndarray
   bw_processing.io_parquet_helpers.save_arr_to_parquet
   bw_processing.io_parquet_helpers.write_ndarray_to_parquet_file


Module Contents
---------------

.. py:function:: load_ndarray_from_parquet(file: io.RawIOBase) -> numpy.ndarray

   Deserialize a `numpy` `ndarray` from a `parquet` `file`.

   Parameters
       file (io.RawIOBase or fsspec file object): File to read from.

   Returns
       The corresponding `numpy` `ndarray`.


.. py:function:: read_parquet_file_to_ndarray(file: io.RawIOBase) -> numpy.ndarray

   Read an `ndarray` from a `parquet` file.

   :param file: File to read from.
   :type file: io.RawIOBase or fsspec file object

   :raises WrongDatatype:

   :returns: The corresponding `numpy` `ndarray`.


.. py:function:: save_arr_to_parquet(file: io.RawIOBase, arr: numpy.ndarray, meta_object: str, meta_type: str) -> None

   Serialize a `numpy` `ndarray` to a `parquet` `file`.

   Parameters
       file (RawIOBase): The file to save to.
       arr (ndarray): The array object to save.
       meta_object (str): "vector" or "matrix".
       meta_type (str): Type of object to serialize (see `io_pyarrow_helpers.py`).


.. py:function:: write_ndarray_to_parquet_file(file: io.BufferedWriter, arr: numpy.ndarray, meta_object: str, meta_type: str)

   Serialize `ndarray` objects to `file`.

   Parameters
       file (io.BufferedWriter): File to save to.
       arr (ndarray): Array to serialize.
       meta_object (str): "vector" or "matrix".
       meta_type (str): Type of object to serialize (see `io_pyarrow_helpers.py`).