https://lepy.github.io/sdata/

Structured data format (sdata)

Design goals

• open data format for open science projects
• self describing data
• flexible data structure layout
  • hierarchical data structure (nesting groups, dictionaries)
  • (posix path syntax support?)
• extendable data structure
  • data format versions
• platform independent
• simple object model
• support of standard metadata formats (key/value, ...)
• support of standard dataset formats (hdf5, netcdf, csv, ...)
• support of standard dataset types (datacubes, tables, series, ...)
• support of physical units (conversion of units)
• transparent, optional data compression (zlib, blosc, ...)
• support of (de-)serialization of every dataset type (group, data, metadata)
• easy defineable (project) standards, e.g. for a uniaxial tension test (UT)
• (optional data encryption (gpg, ...))
• change management support?
• Enable use of data structures from existing tensor libraries transparently
• (single writer/ multiple reader (swmr) support)
• (nested data support)

df = pandas.DataFrame({"a":[1,2,3]})
import sdata
data = sdata.Data(name="my_data", table=df, comment="A remarkable comment")
data.metadata.add("my_key", 123, unit="m^3", description="a volume")
data.metadata.add("force", 1.234, unit="kN", description="x force")
data.to_xlsx(filepath="my_data.xlsx")
print(data.metadata.df)

          name                             value  dtype unit description
key                                                                     
name      name                           my_data    str    -            
uuid      uuid  08222ca66e5047808bdc3b35d8f17224    str    -            
my_key  my_key                               123    int  m^3    a volume
force    force                             1.234  float   kN     x force

Howto

• Das sdata-Format - slides

Demo App

• test the demo app with editor

Try to paste some Excel-Data in the forms ...

Metadata

Attributes

• name
• value
• dtype
• unit
• description
• label
• required

dtypes for attributes

• int
• float
• str
• bool
• timestamp (datetime.isoformat with timezone)

paper

• Das sdata-Format

  • Ingolf Lepenies. (2020). Das sdata-Format (Version 0.8.4). http://doi.org/10.5281/zenodo.4311323
  • slides, html, pdf temperaturmessung-001.json temperaturmessung-001.xlsx

• sdata

  • Ingolf Lepenies. (2020, December 8). sdata - a structured data format (Version 0.8.4). Zenodo. http://doi.org/10.5281/zenodo.4311397