Data Reduction and Compression
Work Package Leader:
D. Spruce (MAXIV), D. Pennicard (DESY)
Thanks to improvements in sources and beamline instrumentation, photon science experiments are producing exponentially increasing quantities of data.
WP 7 focuses on reducing and compressing this data, in order to help users to analyse their results more quickly and to reduce the cost of data storage. A variety of different approaches will be explored, such as data triage (rejecting "bad data"), preprocessing data, and both lossy and lossless compression methods. This will be done both through new research within LEAPS-INNOV (based on a survey of light sources' needs) and by improving sharing of ongoing work. Knowledge exchange with industrial partners will help ensure that these methods are compatible with commercially-available detectors, and that we make efficient use of new computing hardware.
Information on work package organisation, code and data from the work package, can be found on Gitlab here:
DESY, ULUND, ALBA-CELLS, DIAMOND, ELETTRA, ESRF, EUXFEL, HZB, HZDR, PSI, SOLARIS, SOLEIL