From 5e87f569e2a8c085fe942c6774d22039a985056f Mon Sep 17 00:00:00 2001 From: Justus Kuhlmann Date: Fri, 7 Nov 2025 09:43:58 +0000 Subject: [PATCH 1/3] update TODO --- TODO.md | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/TODO.md b/TODO.md index 4153fc3..85074f8 100644 --- a/TODO.md +++ b/TODO.md @@ -1,14 +1,17 @@ # TODO ## Features -- implement import of non-datalad projects -- implement a way to use another backlog repo as a project - -- find a way to convey the mathematical structure of what EXACTLY is the form of the correlator in a specific project - - this could e.g. be done along the lines of mandatory documentation -- keep better track of the versions of the code, that was used for a specific measurement. - - maybe let this be an input in the project file? - - git repo and commit hash/version tag - +- [ ] implement import of non-datalad projects +- [ ] implement a way to use another backlog repo as a project +- [ ] make cache deadlock resistent (no read while writing) +- [ ] find a way to convey the mathematical structure of what EXACTLY is the form of the correlator in a specific project + - [ ] this could e.g. be done along the lines of mandatory documentation +- [ ] keep better track of the versions of the code, that was used for a specific measurement. + - [ ] maybe let this be an input in the project file? + - [ ] git repo and commit hash/version tag + - [ ] implement a code table? +- [ ] parallel processing of measurements +- [ ] extra SQL table for ensembles with UUID and aliases ## Bugfixes - [ ] revisit the reimport function for single files +- [ ] drop record needs to look if no records are left in a json file. From 085256857d5f3cefc413a6a4364354b886f1e419 Mon Sep 17 00:00:00 2001 From: Justus Kuhlmann Date: Thu, 20 Nov 2025 17:07:16 +0100 Subject: [PATCH 2/3] add test notebook to ignore --- .gitignore | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.gitignore b/.gitignore index 1530e48..e7385f6 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,4 @@ pyerrors_corrlib.egg-info __pycache__ -*.egg-info \ No newline at end of file +*.egg-info +test.ipynb From 73d7687359b4d8f1b5e8351f6700fff8be0a6b4e Mon Sep 17 00:00:00 2001 From: Justus Kuhlmann Date: Thu, 20 Nov 2025 17:08:03 +0100 Subject: [PATCH 3/3] update TODO and README --- README.md | 9 +++++++++ TODO.md | 4 ++++ 2 files changed, 13 insertions(+) diff --git a/README.md b/README.md index 976ae57..0f6c9a3 100644 --- a/README.md +++ b/README.md @@ -5,3 +5,12 @@ This is done in a reproducible way using `datalad`. In principle, a dataset is created, that is automatically administered by the backlogger, in which data from differnt projects are held together. Everything is catalogued by a searchable SQL database, which holds the paths to the respective measurements. The original projects can be linked to the dataset and the data may be imported using wrapper functions around the read methonds of pyerrors. + +We work with the following nomenclature in this project: +- Measurement + A setis of Observables, including the appropriate metadata. +- Project + A series of measurements that was done by one person as part of their research. +- Record + An entry of a single Correlator in the database of the backlogger. +- \ No newline at end of file diff --git a/TODO.md b/TODO.md index 85074f8..ba32ec9 100644 --- a/TODO.md +++ b/TODO.md @@ -15,3 +15,7 @@ ## Bugfixes - [ ] revisit the reimport function for single files - [ ] drop record needs to look if no records are left in a json file. + +## Rough Ideas +- [ ] multitable could provide a high speed implementation of an HDF5 based format +- [ ] implement also a way to include compiled binaries in the archives.