tds2stac

A STAC catalog creator from Thredds data server

These details have not been verified by PyPI

Project links

View statistics for this project via Libraries.io, or by using our public dataset on Google BigQuery

Project description

https://img.shields.io/pypi/v/tds2stac.svg

STAC specification is a method of exposing spatial and temporal data collections in a standardized manner. Specifically, the SpatioTemporal Asset Catalog (STAC) specification describes and catalogs spatiotemporal assets using a common structure. This package creates STAC metadata by harvesting dataset details from the Thredds data server. After creating STAC Catalogs, Collections, and Items, it imports them into pgSTAC and STAC-FastAPI.

Free software: EUPL-1.2
Documentation: https://tds2stac.readthedocs.io.

Installation from PyPi

pip install tds2stac

Installation for development

git clone https://codebase.helmholtz.cloud/cat4kit/tds2stac.git
cd tds2stac
python -m venv venv
source venv/bin/activate
pip install -r requirements_dev.txt

Installing using Docker

For runnig by docker use this repository.

Usage

Use case:

You can use the following template for creating STAC catalog from the TDS web service for your project.

You can change configuration of PgSTAC in config_pgstac

from tds2stac.tds2stac import Converter

converter = Converter("http://172.27.80.119:8088/thredds/catalog/regclim/raster/global/era5/sfc/catalog.html",
                     stac=True, stac_dir="/path/to/save/stac/catalogs/",
                     stac_id = "sample",
                     stac_description = "sample",
                     web_service = "iso" or "ncml",
                     datetime_filter=["2020-02-18T00:00:00.000Z","2020-02-22T00:00:00.000Z"],
                     stac_catalog_dynamic = True)

output:

     Start Scanning datasets of http://172.27.80.119:8088/thredds/catalog/regclim/raster/global/era5/sfc/catalog.xml
     |__ http://172.27.80.119:8088/thredds/catalog/regclim/raster/global/era5/sfc/catalog.xml |  Number of branches:  5
     |_______ http://172.27.80.119:8088/thredds/catalog/regclim/raster/global/era5/sfc/static/catalog.xml |  Number of data:  1
     |_______ http://172.27.80.119:8088/thredds/catalog/regclim/raster/global/era5/sfc/monthly/catalog.xml |  Number of data:  246
     |_______ http://172.27.80.119:8088/thredds/catalog/regclim/raster/global/era5/sfc/daily/catalog.xml |  Number of data:  360
     |_______ http://172.27.80.119:8088/thredds/catalog/regclim/raster/global/era5/sfc/climatology/catalog.xml |  Number of data:  7
     |_______ http://172.27.80.119:8088/thredds/catalog/regclim/raster/global/era5/sfc/aggregated/catalog.xml |  Number of data:  1
     615 data are going to be set as items
     5 data are going to be set as items
     Start processing:  http://172.27.80.119:8088/thredds/catalog/regclim/raster/global/era5/sfc/static/catalog.xml
     5 / 5 STAC catalogs are created
     1 / 615 STAC items are connected to the related catalog
     100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 12.70it/s]
     Start processing:  http://172.27.80.119:8088/thredds/catalog/regclim/raster/global/era5/sfc/monthly/catalog.xml
     5 / 5 STAC catalogs are created
     247 / 615 STAC items are connected to the related catalog
     100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 246/246 [00:47<00:00,  5.17it/s]
     Start processing:  http://172.27.80.119:8088/thredds/catalog/regclim/raster/global/era5/sfc/daily/catalog.xml
     5 / 5 STAC catalogs are created
     607 / 615 STAC items are connected to the related catalog
     100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 360/360 [01:12<00:00,  4.95it/s]
     Start processing:  http://172.27.80.119:8088/thredds/catalog/regclim/raster/global/era5/sfc/climatology/catalog.xml
     5 / 5 STAC catalogs are created
     614 / 615 STAC items are connected to the related catalog
     100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:00<00:00,  7.52it/s]
     Start processing:  http://172.27.80.119:8088/thredds/catalog/regclim/raster/global/era5/sfc/aggregated/catalog.xml
     5 / 5 STAC catalogs are created
     615 / 615 STAC items are connected to the related catalog
     100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:23<00:00, 23.93s/it]
     Start processing:  http://172.27.80.119:8088/thredds/catalog/regclim/raster/global/era5/sfc/catalog.xml
     5 / 5 STAC catalogs are created
     615 / 615 STAC items are connected to the related catalog
     0it [00:00, ?it/s]
     ./catalog_regclim_raster_global_era5_sfc_static/collection.json
     |____ ./era5_sfc_0.25_static_ERA5_Landsea_mask.nc/era5_sfc_0.25_static_ERA5_Landsea_mask.nc.json
     ./catalog_regclim_raster_global_era5_sfc_monthly/collection.json
     |____ ./era5_sfc_0.25_monthly_ERA5_monthly_ws10_2018.nc/era5_sfc_0.25_monthly_ERA5_monthly_ws10_2018.nc.json
     .
     .
     .
     |____ ./era5_sfc_0.25_daily_ERA5_daily_sp_1982.nc/era5_sfc_0.25_daily_ERA5_daily_sp_1982.nc.json
     |____ ./era5_sfc_0.25_daily_ERA5_daily_sp_1981.nc/era5_sfc_0.25_daily_ERA5_daily_sp_1981.nc.json
     ./catalog_regclim_raster_global_era5_sfc_climatology/collection.json
     |____ ./era5_sfc_0.25_climatology_ERA5_climatology_ws10_1981_2016.nc/era5_sfc_0.25_climatology_ERA5_climatology_ws10_1981_2016.nc.json
     |____ ./era5_sfc_0.25_climatology_ERA5_climatology_tp_1981_2016.nc/era5_sfc_0.25_climatology_ERA5_climatology_tp_1981_2016.nc.json
     |____ ./era5_sfc_0.25_climatology_ERA5_climatology_t2min_1981_2016.nc/era5_sfc_0.25_climatology_ERA5_climatology_t2min_1981_2016.nc.json
     ./catalog_regclim_raster_global_era5_sfc_aggregated/collection.json
     |____ ./era5_sfc_0.25_aggregated_ERA5_daily_tp_1979_2018.nc/era5_sfc_0.25_aggregated_ERA5_daily_tp_1979_2018.nc.json
     STAC Catalog has been created!

Copyright

Licensed under the EUPL-1.2-or-later

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the EUPL-1.2 license for more details.

You should have received a copy of the EUPL-1.2 license along with this program. If not, see https://www.eupl.eu/.

Project details

These details have not been verified by PyPI

Project links

View statistics for this project via Libraries.io, or by using our public dataset on Google BigQuery

Release history Release notifications | RSS feed

3.1.24

Jan 19, 2024

3.1.23

Jan 19, 2024

3.1.22

Jan 13, 2024

3.1.21

Jan 13, 2024

3.1.20

Jan 11, 2024

3.1.19

Jan 6, 2024

3.1.18

Jan 6, 2024

3.1.17

Jan 4, 2024

3.1.16

Jan 3, 2024

3.1.15

Jan 3, 2024

3.1.14

Jan 2, 2024

3.1.13

Dec 29, 2023

3.1.12

Dec 26, 2023

3.1.11

Dec 22, 2023

3.1.10

Dec 20, 2023

3.1.9

Dec 19, 2023

3.1.8

Dec 19, 2023

3.1.7

Dec 15, 2023

3.1.6

Dec 12, 2023

3.1.5

Dec 11, 2023

3.1.4

Dec 11, 2023

3.1.3

Dec 11, 2023

3.1.2

Dec 11, 2023

3.1.1

Dec 9, 2023

3.1.0

Dec 4, 2023

3.0.4

Nov 17, 2023

3.0.3

Nov 17, 2023

3.0.2

Nov 17, 2023

3.0.0

Nov 17, 2023

2.2.3

Sep 9, 2023

2.2.0

Aug 20, 2023

2.1.6

Aug 16, 2023

2.1.5

Aug 16, 2023

2.1.4

Aug 15, 2023

2.1.3

Aug 15, 2023

2.1.2

Aug 15, 2023

2.1.0

Aug 15, 2023

2.0.14

Aug 8, 2023

2.0.13

Aug 6, 2023

2.0.12

Aug 5, 2023

2.0.11

Jul 30, 2023

2.0.10

Jul 25, 2023

2.0.9

Jul 22, 2023

2.0.8

Jul 21, 2023

2.0.7

Jul 20, 2023

This version

2.0.6

Jul 19, 2023

2.0.1

Jul 18, 2023

2.0.0

Jul 6, 2023

1.9.2

Feb 8, 2023

1.9.1

Feb 7, 2023

1.6.0

Feb 6, 2023

1.0.4

Dec 6, 2022

1.0.2

Nov 21, 2022

1.0.1

Nov 21, 2022

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

TDS2STAC-2.0.6.tar.gz (48.1 kB view hashes)

Uploaded Jul 19, 2023 Source

Built Distribution

TDS2STAC-2.0.6-py2.py3-none-any.whl (24.5 kB view hashes)

Uploaded Jul 19, 2023 Python 2 Python 3

Hashes for TDS2STAC-2.0.6.tar.gz

Hashes for TDS2STAC-2.0.6.tar.gz
Algorithm	Hash digest
SHA256	`0e604e092772df2206c72ce248cef9e370a452a07632e00d4453a5515ab0515c`
MD5	`dfe335b3ca13f241fb18887d1579a90f`
BLAKE2b-256	`f1018e290041c11061d370c34f175666b7a78d37f3897ebe44331ab82e05e71b`

Hashes for TDS2STAC-2.0.6-py2.py3-none-any.whl

Hashes for TDS2STAC-2.0.6-py2.py3-none-any.whl
Algorithm	Hash digest
SHA256	`7332ed867a7dd4bcaf12d947a9685985181a987f33397efc95b3eec55d356fac`
MD5	`1b976fc96d1be566404325f082a73e95`
BLAKE2b-256	`558e0be40caa859aa1c1f094a4b3e9cdb2580457ae33c1d5dddc44161eb74b16`