Skip to content
This repository has been archived by the owner on Apr 20, 2023. It is now read-only.

Lazy iterator for post discovery #127

Open
stefco opened this issue Jun 19, 2021 · 0 comments · May be fixed by #125
Open

Lazy iterator for post discovery #127

stefco opened this issue Jun 19, 2021 · 0 comments · May be fixed by #125

Comments

@stefco
Copy link

stefco commented Jun 19, 2021

Is your feature request related to a problem? Please describe.

When iterating through a Profile's posts, one does not always know how many posts one needs beforehand. Since posts are fetched iteratively and with high network latency, it makes more sense in such cases to lazily iterate through posts. This also simplifies the implementation of the post fetcher by pushing control flow up to the caller, where itertools.islice and the like can shape the data properly with zero performance penalty versus the existing solution (and indeed with a significant speedup in cases where the required number of results is not known ahead of time). This is particularly useful when performing many differential updates on accounts with large numbers of posts, where a lazy iterator provides a natural way of expressing a performant solution.

Describe the solution you'd like

I would like to merge my iter_posts method, which implements a lazy iterator over posts. It additionally re-implements the eager get_posts method with far less code as an eager collect on the iterator returned by the new iter_posts method.

Describe alternatives you've considered

Writing convoluted (and still suboptimal) iterator logic that would be more confusing and unmaintanable than the presented iter_posts method.

Additional context

N/A

@stefco stefco linked a pull request Jun 19, 2021 that will close this issue
4 tasks
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant