indeed, I was thinking about the same. How do you even escape the hatch with e.g. Atlas if your migrations folder is autogenerated? What if you modify /migrations but not your entities?

you do what I do!

1. Generate the migrations and commit them to git

2. Have your codegen use diff3 so users can modify the generated files, or fill in details to partially generated code

3. Give users a way to indicate they did this so you don't break things later with codegen

Most often the gap is about indexes or wierd database quirks than fields on an entity.

Also, make sure you are generating the migrations AND the entity implementations in code from the same source-of-truth. Similar to how people commit the migration history to git, I have hof commit the data model history to git (and the codegen'd output without changes, double files are worth the bloat). By doing this, you can also generate the code that moves data between schema or api versions. (i.e. data lenses a la ink&switch)

cool!

Interesting, will check it out.

Hey, I actually made a silly mistake in my post, indeed you first activate the environment and then install stuff in it. Fixed!

I disagree though it is activated immediately, or at least to me with venv I always have to activate it explicitly.

Sorry dang, didn't know the practice + got a bit emotional haha. I agree with the remark above, my message is rather on how easy it is to run python scripts with dependencies (without mutating the state.

No worries!

It's just standard practice here. See https://news.ycombinator.com/newsguidelines.html.

see my reply above

That's still clickbait, although less tropey. The common definition of clickbait is intentionally omitting information that incentives the user to click.

Still clickbait if you don't say what the feature is.

Indeed, you're right ;).

Neat!

Sure, but what about non AI / business logic pre and post? Think RAG calls, guardrails, ...? Or do they fly compared to LLM inference itself?

Probably depends on the specific case. Basic initialization is stupid fast compared to any stupid training exercise. If your training is going to take anything longer than a minute you are probably very very safe keeping init/postprocess in python and well if you're doing anything intense consider importing a faster way of doing it. Python is like a motherboard for soldering components.

agree, it depends, always benchmark, but my question is rather generic i.e. I am looking for a perspective.

Ok my perspective of pursuing something like this in Rust and continuing the motherboard analogy, is that you take on the full burden of developing everything in your laptop onto a single chip (sounds pretty awesome really) but your boundaries can end up blurred as everything is just a single etch. Forcing boundaries by making components with hard won decisions creates a marketplace of exchangeable components. Force everyone to agree on that API by dropping across a hard language boundary.

Your impact is measured by raw performance masked by time in market. Pure Rust or whatever will have higher raw performance but lower overall (my guess) impact because it misses out on a wider market. Python API will have in general a very slightly less raw performance but much wider time in market.

can't disagree (a tradeoff).

It's getting very philosophical, without a (perhaps impossibly complex/fraught) model of real-world programmers using all available languages doing just what you ask there is no way to find the sweet-spot. I've given some rules of thumb why python still rules and why things in general favor standard forms easily reviewable by humans and peppered with magic only understandable by geeks.

edit: you've removed a line while I was resonding like: "maybe go is a sweet-spot"

I think if you are looking to replace python as a runtime you'd maybe be better off arguing for safety, python is much more easily corruptible so if maybe you don't trust your machine(s) you are training on and don't want to be mislead by someone hacking your initializations then arguably running a compiled program with certain guarantees is safer.

I agree Python is hardly replacable. Btw, I only mean inference, not training. Training imho should stay pure Python, you can achieve mega throughout with it for batch processing.

> Btw, I only mean inference

D'oh! Yea that changes things. I would be considering UI integration from the inference side.

When you say RAG calls are you talking about requests to the (usually vector db) external datastore, or the repeated calls to the LLM?

“Guardrails” are often just calls to one or more (usually smaller) classification/moderation models.

call to external db, and then to llm with retrieved context.

also business rules, no?

> call to external db, and then to llm with retrieved context.

Right, neither of those are CPU intensive in a different way than LLM inference itself (the latter is LLM inference itself.)

> also business rules, no?

Business rules can vary quite a bit in content and complexity, but either tend to be simple enough that they won’t impose much additional load, or complex enough that you are probably going to want to simply use an existing rules engine (many of which, regardless of their implementation language, have Python bindings) which are going to behave the same way no matter what language you call them from.

Fair points!

Good perspective. No-GIL should make things better (shared memory parallelism), but it's not bulletproof.

If I could boot up Python with --no-gil I might give it a shot but it looks like a hassle to install a no-GIL Python now. And after all that hassle I'd expect it to take years for the details to get worked out.

One reason Java was successful with thread safety was its xenophobia. Sure you can load libs with JNI but it feels like putting your hand in a toilet. If you add threads to POSIX you will always have old libraries not built with threads in mind, if you start out with threads the whole ecosystem is built on the assumption it should work right with threads.

Indeed, my point was that Go and Rust could lead optimizing the non-AI code, which often begs to be coupled with AI code (think guardrails).

Also, what's the benefit of Python then in this case? Ergonomically, Go isn't shabby, Rust is another story though.