cox.readers module

class cox.readers.CollectionReader(directory, log_warnings=True, mode='r', exp_filter=None, skip_errs=False)

Bases: object

Class for collecting, viewing, and manipulating directories of stores.

Initialize the CollectionReader object. This will immediately open each store in directory and see which table are available for viewing.

  • directory (str) – Path to directory with stores in it. The directory should contain directories corresponding to stores.
  • log_warnings (bool) – Log warnings if tables with the same name have different schemas
  • mode (str) – mode to open stores in. Default ‘r’ (read only), if you want to write you will need to make the mode ‘a’ (append only) or ‘w’ (write).
  • exp_filter (method) – Call exp_filter on the experiment id of each store, excludes store from collection if it returns false.

Closes all the stores opened by the collection reader.

df(key, append_exp_id=True, keep_serialized=[], union_schemas=False, exp_filter=None, skip_errors=False)

Makes a large concatenated PD dataframe from all the stores’ tables matching this table key.

  • key (str) – name of table to collect
  • append_exp_id (bool) – if true, append corresponding experiment id to each row.
  • keep_serialized (list of strings) – list corresponding to column names. If in this list, do not unserialize the string within the column name and make it a python object within the pandas table.
  • union_schemas (bool) – If true, union columns of all collected tables, otherwise error out.
  • exp_filter (method) – If function of exp_id returns false, ignore this store. Otherwise include.
  • skip_errors (bool) – If true, skip an experiment upon error occurs.

Concatenated dataframe of all corresponding tables in the dataframes matching the key.