2022-11-14 05:55:03.268322+01 by Dan Lyke 5 comments
Two decades ago I delicately and carefully wrote a high performance threaded database core using pthreads. Now I spend a lot of time debugging race conditions because people just throw queues and timers around without engineering in threads from the start. Sigh.
[ related topics: Theater & Plays Databases ]
comments in ascending chronological order (reverse):
#Comment Re: Two decades ago I delicately and made: 2022-11-14 19:48:16.865439+01 by: markd
The new Swift "Structured concurrency" async/await syntax helps out some with that. But I noticed that that makes it easy to add an extremely expensive operation in the middle of something. "it's really cool I can trivially add a round-trip to the dropbox API to get some missing metadata, but I'm trivially adding a round- trip to a slow API..."
#Comment Re: Two decades ago I delicately and made: 2022-11-17 00:06:26.687389+01 by: Dan Lyke
Yeah, the problems I seem to be running up against, and sometimes it's hard to debug, are "these things that take no time are thrown off into a background thread, but this thing that round-trips via JavaScript in a web view has to happen on main and kaboom.
The other place I've run into it is a C++ situation I haven't debugged yet where Qt is forking processes for audio decoding and there's some other deadlock.
In both cases, I think having a threading and process strategy that got religiously adhered to, rather than "woohoo, we've got Promises, let's rock!", would make this debugging less... everything.
#Comment Re: Two decades ago I delicately and made: 2022-11-19 01:08:51.071795+01 by: TheSHAD0W
You'd think it would be easy, do things asynchronously until you can't; and turns out all the important stuff is tied together in a way that doesn't allow for it, and coding to prevent that is ridiculous.
In a couple of things I wound up doing multiprocessing instead of threading, which was a lot more resource intensive but didn't result in things locking up.
#Comment Re: Two decades ago I delicately and made: 2022-11-21 23:51:14.853924+01 by: Dan Lyke
Yeah, one of the big lessons back in the days when I was writing in pthreads was that at that point sockets were roughly the speed of memcpy, so processes were a smarter way to go.
Unfortunately, everyone wants to use "promises" and similar abstractions these days, and rather than letting us ponder good architectures it seems to be a lot of "hey, let's throw some shit at this wall and see what sticks", and...
Anyway, two separate projects, lots of pain.
#Comment Re: Two decades ago I delicately and made: 2022-11-22 00:13:08.179477+01 by: TheSHAD0W
Oh yeah, on the Bittorrent fork I managed, I wound up ripping out threading completely and running everything in a single process. Admittedly, python threads aren't real threads so there was no chance of a performance loss.