bw2io ===== .. py:module:: bw2io Subpackages ----------- .. toctree:: :maxdepth: 1 /content/api/bw2io/export/index /content/api/bw2io/extractors/index /content/api/bw2io/importers/index /content/api/bw2io/strategies/index Submodules ---------- .. toctree:: :maxdepth: 1 /content/api/bw2io/backup/index /content/api/bw2io/compatibility/index /content/api/bw2io/download_utils/index /content/api/bw2io/errors/index /content/api/bw2io/migrations/index /content/api/bw2io/modified_database/index /content/api/bw2io/package/index /content/api/bw2io/remote/index /content/api/bw2io/units/index /content/api/bw2io/unlinked_data/index /content/api/bw2io/utils/index Attributes ---------- .. autoapisummary:: bw2io.activity_hash bw2io.migrations bw2io.normalize_units bw2io.unlinked_data Classes ------- .. autoapisummary:: bw2io.BW2Package bw2io.CSVImporter bw2io.CSVLCIAImporter bw2io.DatabaseSelectionToGEXF bw2io.DatabaseToGEXF bw2io.Ecospold1LCIAImporter bw2io.ExcelImporter bw2io.ExcelLCIAImporter bw2io.Migration bw2io.MultiOutputEcospold1Importer bw2io.SimaProCSVImporter bw2io.SimaProLCIACSVImporter bw2io.SingleOutputEcospold1Importer bw2io.SingleOutputEcospold2Importer bw2io.UnlinkedData Functions --------- .. autoapisummary:: bw2io.backup_data_directory bw2io.backup_project_directory bw2io.bw2setup bw2io.create_core_migrations bw2io.create_default_biosphere3 bw2io.create_default_lcia_methods bw2io.es2_activity_hash bw2io.install_project bw2io.lci_matrices_to_excel bw2io.lci_matrices_to_matlab bw2io.load_json_data_file bw2io.restore_project_directory Package Contents ---------------- .. py:class:: BW2Package Bases: :py:obj:`object` This is a format for saving objects which implement the :ref:`datastore` API. Data is stored as a BZip2-compressed file of JSON data. This archive format is compatible across Python versions, and is, at least in theory, programming-language agnostic. Validation is done with ``bw2data.validate.bw2package_validator``. The data format is: .. code-block:: python { 'metadata': {}, # Dictionary of metadata to be written to metadata-store. 'name': basestring, # Name of object 'class': { # Data on the underlying class. A new class is instantiated # based on these strings. See _create_class. 'module': basestring, # e.g. "bw2data.database" 'name': basestring # e.g. "Database" }, 'unrolled_dict': bool, # Flag indicating if dictionary keys needed to # be modified for JSON (as JSON keys can't be tuples) 'data': object # Object data, e.g. LCIA method or LCI database } Perfect roundtrips between machines are not guaranteed: * All lists are converted to tuples (because JSON does not distinguish between lists and tuples). * Absolute filepaths in metadata would be specific to a certain computer and user. .. note:: This class does not need to be instantiated, as all its methods are ``classmethods``, i.e. do ``BW2Package.import_obj("foo")`` instead of ``BW2Package().import_obj("foo")`` .. py:method:: _create_class(metadata, apply_whitelist=True) :classmethod: .. py:method:: _create_obj(data) :classmethod: .. py:method:: _get_class_metadata(obj) :classmethod: .. py:method:: _is_valid_package(data) :classmethod: .. py:method:: _is_whitelisted(metadata) :classmethod: .. py:method:: _load_obj(data, whitelist=True) :classmethod: .. py:method:: _prepare_obj(obj, backwards_compatible=False) :classmethod: .. py:method:: _write_file(filepath, data) :classmethod: .. py:method:: export_obj(obj, filename=None, folder='export', backwards_compatible=False) :classmethod: Export an object. :param \* *obj*: Object to export. :type \* *obj*: object :param \* *filename*: Name of file to create. Default is ``obj.name``. :type \* *filename*: str, optional :param \* *folder*: Folder to create file in. Default is ``export``. :type \* *folder*: str, optional :param \* *backwards_compatible*: Create package compatible with bw2data version 1. :type \* *backwards_compatible*: bool, optional :returns: Filepath of created file. .. py:method:: export_objs(objs, filename, folder='export', backwards_compatible=False) :classmethod: Export a list of objects. Can have heterogeneous types. :param \* *objs*: List of objects to export. :type \* *objs*: list :param \* *filename*: Name of file to create. :type \* *filename*: str :param \* *folder*: Folder to create file in. Default is ``export``. :type \* *folder*: str, optional :param \* *backwards_compatible*: Create package compatible with bw2data version 1. :type \* *backwards_compatible*: bool, optional :returns: Filepath of created file. .. py:method:: import_file(filepath, whitelist=True) :classmethod: Import bw2package file, and create the loaded objects, including registering, writing, and processing the created objects. :param \* *filepath*: Path of file to import :type \* *filepath*: str :param \* *whitelist*: Apply whitelist to allowed types. Default is ``True``. :type \* *whitelist*: bool :returns: Created object or list of created objects. .. py:method:: load_file(filepath, whitelist=True) :classmethod: Load a bw2package file with one or more objects. Does not create new objects. :param \* *filepath*: Path of file to import :type \* *filepath*: str :param \* *whitelist*: Apply whitelist of approved classes to allowed types. Default is ``True``. :type \* *whitelist*: bool Returns the loaded data in the bw2package dict data format, with the following changes: * ``"class"`` is an actual Python class object (but not instantiated). .. py:attribute:: APPROVED .. py:class:: CSVImporter(filepath) Bases: :py:obj:`ExcelImporter` .. autoapi-inheritance-diagram:: bw2io.CSVImporter :parts: 1 :private-bases: Generic CSV importer .. py:attribute:: extractor .. py:attribute:: format :value: 'CSV' .. py:class:: CSVLCIAImporter(filepath, name, description, unit, **metadata) Bases: :py:obj:`ExcelLCIAImporter` .. autoapi-inheritance-diagram:: bw2io.CSVLCIAImporter :parts: 1 :private-bases: Generic CSV LCIA importer .. py:attribute:: extractor .. py:attribute:: format :value: 'CSV' .. py:class:: DatabaseSelectionToGEXF(database, keys) Bases: :py:obj:`DatabaseToGEXF` .. autoapi-inheritance-diagram:: bw2io.DatabaseSelectionToGEXF :parts: 1 :private-bases: Export a Gephi graph for a selection of activities from a database. Also includes all inputs for the filtered activities. :param \* *database*: Database name. :type \* *database*: str :param \* *keys*: The activity keys to export. :type \* *keys*: str .. py:attribute:: data .. py:attribute:: database .. py:attribute:: filepath .. py:attribute:: id_mapping .. py:attribute:: unfiltered_data .. py:class:: DatabaseToGEXF(database, include_descendants=False) Bases: :py:obj:`object` Export a Gephi graph for a database. Call ``.export()`` to export the file after class instantiation. :param \* *database*: Database name. :type \* *database*: str :param \* *include_descendants*: Include databases which are linked from ``database``. :type \* *include_descendants*: bool .. warning:: ``include_descendants`` is not yet implemented. .. py:method:: export() Export the Gephi XML file. Returns the filepath of the created file. .. py:method:: get_data(E) Get Gephi nodes and edges. .. py:attribute:: data .. py:attribute:: database .. py:attribute:: descendants .. py:attribute:: filename .. py:attribute:: filepath .. py:attribute:: id_mapping .. py:class:: Ecospold1LCIAImporter(filepath, biosphere=None) Bases: :py:obj:`bw2io.importers.base_lcia.LCIAImporter` .. autoapi-inheritance-diagram:: bw2io.Ecospold1LCIAImporter :parts: 1 :private-bases: .. py:attribute:: data .. py:attribute:: format :value: 'Ecospold1 LCIA' .. py:attribute:: start .. py:class:: ExcelImporter(filepath) Bases: :py:obj:`bw2io.importers.base_lci.LCIImporter` .. autoapi-inheritance-diagram:: bw2io.ExcelImporter :parts: 1 :private-bases: Generic Excel importer. See the `generic Excel example spreadsheet `__. Excel spreadsheet should follow the following format: :: Project parameters , , , metadata Database, , Parameters , , , metadata Activity, , Exchanges , , , , , , Neither project parameters, parameters, nor exchanges for each activity are required. An activity is marked as finished with a blank line. In general, data is imported without modification. However, the following transformations are applied: * Numbers are translated from text into actual numbers. * Tuples, separated in the cell by the ``::`` string, are reconstructed. * ``True`` and ``False`` are transformed to boolean values. * Fields with the value ``(Unknown)`` are dropped. .. py:method:: get_activity(sn, ws) .. py:method:: get_database(data) .. py:method:: get_database_parameters(data) .. py:method:: get_labelled_section(sn, ws, index=0, transform=True) Turn a list of rows into a list of dictionaries. The first line of ``ws`` is the column labels. All subsequent rows are the data values. Missing columns are dropped. ``transform`` is a boolean: perform CSV transformation functions like ``csv_restore_tuples``. .. py:method:: get_metadata_section(sn, ws, index=0, transform=True) .. py:method:: get_project_parameters(data) Extract project parameters (variables and formulas). Project parameters are a section that starts with a line with the string "project parameters" (case-insensitive) in the first cell, and ends with a blank line. There can be multiple project parameter sections. .. py:method:: process_activities(data) Take list of `(sheet names, raw data)` and process it. .. py:method:: write_activity_parameters(data=None, delete_existing=True) .. py:method:: write_database(**kwargs) Same as base ``write_database`` method, but ``activate_parameters`` is True by default. .. py:method:: write_database_parameters(activate_parameters=True, delete_existing=True) Same as base ``write_database_parameters`` method, but ``activate_parameters`` is True by default. .. py:attribute:: data .. py:attribute:: extractor .. py:attribute:: format :value: 'Excel' .. py:attribute:: start .. py:attribute:: strategies .. py:class:: ExcelLCIAImporter(filepath, name, description, unit, **metadata) Bases: :py:obj:`bw2io.importers.base_lcia.LCIAImporter` .. autoapi-inheritance-diagram:: bw2io.ExcelLCIAImporter :parts: 1 :private-bases: Generic Excel LCIA importer. See the `documentation `__. .. py:attribute:: data .. py:attribute:: extractor .. py:attribute:: format :value: 'Excel' .. py:attribute:: strategies .. py:class:: Migration(*args, **kwargs) Bases: :py:obj:`bw2data.data_store.DataStore` .. autoapi-inheritance-diagram:: bw2io.Migration :parts: 1 :private-bases: .. py:method:: load() .. py:method:: validate(*args, **kwargs) .. py:method:: write(data, description) Write migration data. Requires a description. .. py:attribute:: _intermediate_dir .. py:attribute:: _metadata .. py:property:: description .. py:class:: MultiOutputEcospold1Importer(*args, **kwargs) Bases: :py:obj:`SingleOutputEcospold1Importer` .. autoapi-inheritance-diagram:: bw2io.MultiOutputEcospold1Importer :parts: 1 :private-bases: Import and process mutli-output datasets in the ecospold 1 format. Works the same as the single-output importer, but first allocates multioutput datasets. .. py:class:: SimaProCSVImporter(filepath, name=None, delimiter=';', encoding='latin-1', normalize_biosphere=True, biosphere_db=None) Bases: :py:obj:`bw2io.importers.base_lci.LCIImporter` .. autoapi-inheritance-diagram:: bw2io.SimaProCSVImporter :parts: 1 :private-bases: .. py:method:: get_db_name() .. py:method:: match_ecoinvent2(db_name) .. py:method:: write_database(data=None, name=None, *args, **kwargs) .. py:attribute:: format :value: 'SimaPro CSV' .. py:attribute:: start .. py:attribute:: strategies .. py:class:: SimaProLCIACSVImporter(filepath, biosphere=None, delimiter=';', encoding='latin-1', normalize_biosphere=True) Bases: :py:obj:`bw2io.importers.base_lcia.LCIAImporter` .. autoapi-inheritance-diagram:: bw2io.SimaProLCIACSVImporter :parts: 1 :private-bases: .. py:attribute:: data .. py:attribute:: format :value: 'SimaPro CSV LCIA' .. py:attribute:: start .. py:class:: SingleOutputEcospold1Importer(filepath, db_name, use_mp=True, extractor=Ecospold1DataExtractor) Bases: :py:obj:`bw2io.importers.base_lci.LCIImporter` .. autoapi-inheritance-diagram:: bw2io.SingleOutputEcospold1Importer :parts: 1 :private-bases: Import and process single-output datasets in the ecospold 1 format. Applies the following strategies: #. If only one exchange is a production exchange, that is the reference product #. Delete (unreliable) integer codes from extracted data #. Drop ``unspecified`` subcategories from biosphere flows #. Normalize biosphere flow categories to ecoinvent 3.1 standard #. Normalize biosphere flow names to ecoinvent 3.1 standard #. Remove locations from biosphere exchanges #. Create a ``code`` from the activity hash of the dataset #. Link biosphere exchanges to the default biosphere database #. Link internal technosphere exchanges :param \* *filepath*: Either a file or directory. :param \* *db_name*: Name of database to create. .. py:attribute:: db_name .. py:attribute:: format :value: 'Ecospold1' .. py:attribute:: start .. py:attribute:: strategies .. py:class:: SingleOutputEcospold2Importer(dirpath, db_name, extractor=Ecospold2DataExtractor, use_mp=True, signal=None, reparametrize_lognormals=False) Bases: :py:obj:`bw2io.importers.base_lci.LCIImporter` .. autoapi-inheritance-diagram:: bw2io.SingleOutputEcospold2Importer :parts: 1 :private-bases: Base class for format-specific importers. Defines workflow for applying strategies. Takes a database name (string) as initialization parameter. Initializes the SingleOutputEcospold2Importer class instance. :param dirpath: Path to the directory containing the ecospold2 file. :type dirpath: str :param db_name: Name of the LCI database. :type db_name: str :param extractor: Class for extracting data from the ecospold2 file, by default Ecospold2DataExtractor. :type extractor: class :param use_mp: Flag to indicate whether to use multiprocessing, by default True. :type use_mp: bool :param signal: Object to indicate the status of the import process, by default None. :type signal: object :param reparametrize_lognormals: Flag to indicate if lognormal distributions for exchanges should be reparametrized such that the mean value of the resulting distribution meets the amount defined for the exchange. :type reparametrize_lognormals: bool .. py:attribute:: db_name .. py:attribute:: dirpath .. py:attribute:: format :value: 'Ecospold2' .. py:attribute:: signal .. py:attribute:: start .. py:attribute:: strategies .. py:class:: UnlinkedData Bases: :py:obj:`bw2data.data_store.DataStore` .. autoapi-inheritance-diagram:: bw2io.UnlinkedData :parts: 1 :private-bases: .. py:method:: validate(*args, **kwargs) .. py:attribute:: _intermediate_dir :value: 'unlinked' .. py:attribute:: _metadata .. py:function:: backup_data_directory() Backup data directory to a ``.tar.gz`` (compressed tar archive) in the user's home directory. Restoration is done manually. .. rubric:: Examples >>> bw2io.bw2setup() >>> bw2io.backup.backup_data_directory() Creating backup archive - this could take a few minutes... .. py:function:: backup_project_directory(project: str) Backup project data directory to a ``.tar.gz`` (compressed tar archive) in the user's home directory. :param project: Name of the project to backup. :type project: str :returns: **project_name** -- Name of the project that was backed up. :rtype: str :raises ValueError: If the project does not exist. .. seealso:: :obj:`bw2io.backup.restore_project_directory` To restore a project directory from a backup. .. py:function:: bw2setup() .. py:function:: create_core_migrations() Add pre-defined core migrations data files .. py:function:: create_default_biosphere3(overwrite=False) .. py:function:: create_default_lcia_methods(overwrite=False, rationalize_method_names=False, shortcut=True) .. py:function:: es2_activity_hash(activity, flow) Generate unique ID for ecoinvent3 dataset. Despite using a million UUIDs, there is actually no unique ID in an ecospold2 dataset. Datasets are uniquely identified by the combination of activity and flow UUIDs. .. py:function:: install_project(project_key: str, project_name: Optional[str] = None, projects_config: Optional[dict] = get_projects(), url: Optional[str] = 'https://files.brightway.dev/', overwrite_existing: Optional[bool] = False, __recursive: Union[bool, None] = False) Install an existing Brightway project archive. By default uses ``https://files.brightway.dev/`` as the file repository, but you can run your own. :param project_key: A string uniquely identifying a project, e.g. ``ecoinvent-3.8-biosphere``. :type project_key: str :param project_name: The name of the new project to create. If not provided will be taken from the archive file. :type project_name: str, optional :param projects_config: A dictionary that maps ``project_key`` values to filenames at the repository :type projects_config: dict, optional :param url: The URL, with trailing slash ``/``, where the file can be found. :type url: str, optional :param overwrite_existing: Allow overwriting an existing project :type overwrite_existing: bool, optional :param __recursive: Internal flag used to determine if this function has errored out already :type __recursive: bool :returns: The name of the created project. :rtype: str .. py:function:: lci_matrices_to_excel(database_name, include_descendants=True) Fake docstring .. py:function:: lci_matrices_to_matlab(database_name) .. py:function:: load_json_data_file(filename) .. py:function:: restore_project_directory(fp: str, project_name: Optional[str] = None, overwrite_existing: Optional[bool] = False) Restore a backed up project data directory from a ``.tar.gz`` (compressed tar archive) in the user's home directory. :param fp: File path of the project to restore. :type fp: str :param project_name: Name of new project to create :type project_name: str, optional :param overwrite_existing: :type overwrite_existing: bool, optional :returns: **project_name** -- Name of the project that was restored. :rtype: str :raises ValueError: If the project does not exist. .. seealso:: :obj:`bw2io.backup.backup_project_directory` To restore a project directory from a backup. .. py:data:: activity_hash .. py:data:: migrations .. py:data:: normalize_units .. py:data:: unlinked_data