What’s new¶
A catalog of new features, improvements, and bug-fixes in each release. Follow links to the relevant GitHub issue or pull request for specific code changes and any related discussion.
Dataportal v0.2.2¶
Dataportal v0.2.1¶
API Changes¶
- get_images (as alias for Images) added for consistency with other function names
- SubtractedImages removed; prefer PIMS pipeline feature.
Dataportal v0.2.0¶
API Changes¶
- DataBroker[] for slicing by scan ID(s) or recency
- DataBroker() for building queries from keyword arguments
- get_events return events generator
- get_table return DataFrame
- Header, vastly simplified: it is merely Document with a dedicated constructor that accepts a Run Start Document
Dataportal v0.0.6¶
Enhancements¶
- A new StepScan interface acts like DataBroker but immediately returns tabular data as a DataFrame in one step. (GH136)
- Look up scans by the name of a detector or motor used. For example, to get all scans that measured ‘Tsam’, use DataBroker.find_headers(data_key='Tsam'). (GH88, GH107)
- Look up scans using the first few characters of its unique ID, like DataBroker['aow23oif']. To be clear this is the ophyd-provided uid, not the mongo _id. (GH130, GH131)
- Replay remembers settings when flipping between scans, and it retains these settings between sessions. (GH114)
- DataMuxer.to_sparse_dataframe returns all data with one Event per row. (GH134)
- DataMuxer.plan.bin_on and DataMuxer.plan.bin_by_edges explain the planned operation of DataMuxer.bin_on and DataMuxer.bin_by_edges for a given data set and given arguments. (GH134)
API Changes¶
The Event documents are reorganized to be more intuitive and require less typing. Formerly, event.data returned a dictionary of (value, timestamp) tuples.
event.data = {'motor1': (value, timestamp), 'motor2': (value, timestamp)}
Now, event.data is dictionary of the data
event.data = {'motor1': value, 'motor2': value}
and event.timestamp is a dictionary of the timestamps.
event.timestamps = {'motor1': timestamp, 'motor2': timestamp}
All functions that return Documents, including Headers, Events, and everything stored in metadatastore and filestore, now return Python generators, iterable objects that load data one element at a time. To convert these to normal Python lists, simply use list(gen). (GH127)
The output of DataMuxer.bin_* functions is indexed by bin number (0, 1, 2...). The Event time is given as a column.
Metadatastore and filestore require configuration settings. They look in the following locations, in increasing order of precedence. Use #3 or #4 to customize your own metadatastore and filestore.
- CONDA_ENV/etc/name.yaml (if CONDA_ETC_ env is defined)
- /etc/name.yaml
- ~/.config/name/connection.yml
- reading environmental variables formatted like MDS_DATABASE or FS_DATABASE
For example, in ~/.config/metadatastore/connection.yaml
host: localhost port: 27017 database: my_metadatastore timezone: US/Eastern
and likewise with filestore/connection.yaml. (Filestore does not need a timezone field, however.) If no configuration can be found, they will raise an error on import. We avoid defaults so that experimental data cannot be accidentally saved to an unsafe destination.
Metadatastore configuration also requires a timezone field, which is uses to interpret human-friendly datetimes.