If you want a specific answer, you’re going to have to ask a specific question.

Sorry for taking so long to get back to you on this, but I’m not always on Lemmy. There’s always more code to be written - you know how it is, I’m sure.

Given the constraints you outline, one other avenue of attack could be to consider the time-sensitivity of product updates and the relative priority thereof. If it’s acceptable for updates to products to lag somewhat, you can at least perform them at a lower rate over longer time, thus reducing hardware load at any given time. If the periodic updates are make to the same per-product values, you could even potentially get smart and replace queued updates not yet performed, if they’re superseded by a subsequent change before they’re actually committed thus further reducing load.

“Why would you go for that opening?! What’s wrong with you?”

To paraquote H. L. Mencken: For every problem, there is a solution that’s cheap, fast, easy to implement – and wrong.

Silver bullets and magic wands don’t really exist, I’m afraid. There’s amble reasons for DBA’s being well-paid people.

There’s basically three options: Either increase the hardware capabilities to be able to handle the amount of data you want to deal with, decrease the amount of data so that the hardware you’ve got can handle it at the level of performance you want or… Live with the status quo.

If throwing more hardware at the issue was an option, I presume you would just have done so. As for how to viably decrease the amount of data in your active set, well, that’s hard to say without knowledge of the data and what you want to do with it. Is it a historical dataset or time series? If so, do you need to integrate the entire series back until the dawn of time, or can you narrow the focus to a recent time window and shunt old data off to cold storage? Is all the data per sample required at all times, or can details that are only seldom needed be split off into separate detail tables that can be stored on separate physical drives at least?

“Oh no. Our automatic door seems to have suffered a unfortunate malfunction while you were trying to leave the premises.”

Eh, tried it. Doesn’t seem to support regex or even wildcard matching, which is… suboptimal. The preview window also often fails to show the content of the current file correctly.

Hah, okay. That’s pretty cool. If people are going to be writing farming bots anyway, one might as well make it the formal objective.

You… grinds teeth may… have a point.

I have a confession to make: Unless shell script is absolutely required, I just use Python for all my automation needs.

A mixture of a combined sub-7k audio sequencer and softsynth, an OpenGL 3.3 PBR renderer and a small BRep CSG modelling DSL.