Experience with using Xarray in production?

[CPC464]

Hi all,

I'm building an accounting platform, and Xarray could be handy for a lot of the reports I need to run. But I'm struggling to find out if it is suitable for being deployed to production, or?

Any thoughts and specific expereinces/examples/dos/donts highly appreciated!

//Thomas

[rabernat]

There are hundreds of orgs using Xarray in production, from NASA down to tiny startups and everyone in between.

If you can provide more detail about your usage pattern, we could provide more specific guidance.

[CPC464]

@rabernat Thank you so much for the reply! I'm building an accounting platform, and I am considering to use Xarray for all reporting/analytics. I.e. both for generating P&L, balancesheet and cashflow statements, but also for more granular analytics and metrics. What are your thoughts on that? Any caveats I should be aware of?

[rabernat]

That sounds super interesting. We're aware lots of folks using Xarray in quantitative finance applications.

"In production" means many things to different people. If you're running a batch job, e.g. nightly, that works great.

If you're building a live API which queries Xarray Datasets and aiming for low latency, one thing to keep in mind is that Datasets can have a considerable initialization time due to setting up of indexes. So it's good to persist them over many requests, rather than re-initializing them on every request.

[CPC464]

@rabernat The specific flow would be something like this:

• API request for a specific report (e.g. a P&L for 24 months)
• Database query to get the monthly totals for each P&L account
• The resulting time-series for all of the accounts would get stored in a Xarray so we end up with a 3D array
• Calculation of subtotals and totals using Xarray methods
• Creation of a new 3D return_array

Re. the indexing - does it help that the overall time series will always be the same for a given tenant (but could be filtered to fewer months)? And that the 'gross list' of accounts is also static (but a given report will only use a subset of those)? I.e. could I iniitalize the indices once, when a user logs into the application (and accept that overhead), but then they would be ready when the user chooses to run a report?

[rabernat]

This sounds totally reasonable. I'd recommend just giving it a try using the defaults. Don't overthink it and don't prematurely optimize! 😉 Unless you need some ultra-low-latency or high-frequency thing, you'll be totally fine.