Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GeoPandas ShapeFile, Feather, and Parquet Reader and Writer #504

Draft
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

JoJo10Smith
Copy link
Contributor

@JoJo10Smith JoJo10Smith commented Oct 28, 2023

WIP: Adding the GeoPandas FIle Read and Write Class to Hamilton.

Changes

geopandas_extensions.py --> Adding the read and write class
requirements-test.txt -> Adding additional requirements that are needed for the classes to work
test_geopandas_extensions.py --> Adding test for each of the files that are outputted
h_geopandas.py --> Added this file so that I could create DAGs with Geopandas functions
notebook.ipynb --> Examples of usage
geometric_functions.py --> Examples of usage

How I tested this

Added the classes and showed that they output data in a local notebook. I also wrote a single test to check that all the geodataframses that are outputted (dbf, shp, shx) which are all the memory files are the same as the original example file I used.

more info: https://en.wikipedia.org/wiki/Shapefile

Notes

The output dag in the examples is pictured below

image

Checklist

  • PR has an informative and human-readable title (this will be pulled into the release notes)
  • Changes are limited to a single goal (no scope creep)
  • Code passed the pre-commit check & code is left cleaner/nicer than when first encountered.
  • Any change in functionality is tested
  • New functions are documented (with a description, list of inputs, and expected output)
  • Placeholder code is flagged / future TODOs are captured in comments
  • Project documentation has been updated if adding/changing functionality.

…, Tests for that class, and additional requirements
@sweep-ai
Copy link
Contributor

sweep-ai bot commented Oct 28, 2023

Apply Sweep Rules to your PR?

  • Apply: Leftover TODOs in the code should be handled.
  • Apply: All new business logic should have corresponding unit tests in the tests/ directory.
  • Apply: Any clearly inefficient or repeated code should be optimized or refactored.

@JoJo10Smith
Copy link
Contributor Author

Tests are still failing for python 3.7. The earliest version of geodatasets is 0.1.0 (where the example datasets are collected) requires python version 3.8.
https://pypi.org/project/geodataset/0.1.0/

@skrawcz
Copy link
Collaborator

skrawcz commented Oct 31, 2023

Tests are still failing for python 3.7. The earliest version of geodatasets is 0.1.0 (where the example datasets are collected) requires python version 3.8. https://pypi.org/project/geodataset/0.1.0/

yeah you can gate this with the pytest check I showed.

Otherwise some dataclass issue with the current tests failing? Can't see anything obvious on first glance.

@JoJo10Smith JoJo10Smith changed the title GeoPandas ShapeFile Reader and Writer GeoPandas ShapeFile, Feather, and Parquet Reader and Writer Nov 18, 2023
@skrawcz
Copy link
Collaborator

skrawcz commented Nov 18, 2023

@JoJo10Smith BTW we removed support for 3.7 -- so no need to special case anything for that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants