diff --git a/Data-management.md b/Data-management.md index 5e5fff7..ddf3cda 100644 --- a/Data-management.md +++ b/Data-management.md @@ -1,4 +1,4 @@ -#### Disclaimer: this page is adapted from the internal wiki of Lars Pastewka's group +*Disclaimer: this page is adapted from the internal wiki of [Lars Pastewka's group](https://pastewka.org/)* # Datasets @@ -154,3 +154,5 @@ Datasets will be removed by an administrator or an automated script once their e ## Invalid datasets You may want to delete a dataset because you made an error in the simulation script and it now interferes in your postprocessing workflow. Since the dataset will stay on the storage for a while, make it explicit that the dataset is invalid: `dtool tag set invalid`. When you select datasets for postprocessing, make sure it has not been marked as invalid. With the mongo query interface, this can be achieved by adding `{"tags": {"$not":{"$in": ["invalid"]}}}` ([mongodb doc](https://docs.mongodb.com/manual/reference/operator/query/in/#use-the-in-operator-to-match-values-in-an-array)) in an and clause. In python, you can check the dataset instance is valid with `not "invalid" in dataset.list_tags()`. + +# Git LFS