Final Elixir/SQLAlchemy frustration post
I’m going to stop complaining about Elixir and SQLAlchemy now. Not because I suddenly realized how great it is, but because I gave up.
For anyone not familiar with these things, SQLAlchemy is “the Python SQL toolkit and Object Relational Mapper” and Elixir is “a declarative layer on top of the SQLAlchemy library, a fairly thin wrapper, which provides the ability to create simple Python classes that map directly to relational database tables”. Sound nice enough.
But the last days I’ve been tearing my hair out trying to get the sessions and rollbacks to work and today I finally gave up. No sessions for this system! Now it’s all auto-commits and manual rollbacks and things are working! This (not so pretty) solution took about one hour to complete, compared to the looong days of frustration I could’ve avoided if I’d just done it the easy way from start.
OK. So Elixir is like bad magic. No concern about databases at all for the developer, and no concern about sessions. It’s a hell to debug and as soon as you want to do something outside what Elixir was primarily built for, prepare to cry.
Elixir uses the scoped SQLAlchemy session, which means that there is one global session that’s supposedly thread local. My problem was that all child processes kept getting the same session returned from the scoped session and there wasn’t really much to do about it.
There are a few ways to manually tell Elixir what session to use, but only on a module level. I needed my sessions on a thread level as the system runs multiple parallell processes and that isn’t doable to my knowledge. SQLAlchemy has support to add objects manually to any session, but as soon as you start bypassing Elixir and go SQLAlchemy instead you might as well just skip Elixir. When removing the default session handling from Elixir, all convenient mappings are gone and those are pretty much what Elixir is made for.
The conclusion is that if I’d built this system from scratch I probably wouldn’t have used Elixir.
2 CommentsSnowed in
It’s been snowing quite a lot in Stockholm lately, and despite Sweden being a country where this snow thing happens every year, it’s now nearly impossible to get anywhere unless you live in the inner city or are prepared to walk. The subway trains now only function below ground (which means only in the inner city), the buses have problems braking at some stops because of icy roads and many buses are being used to replace the subway traffic. There aren’t nearly enough buses for this though, as one full subway train would need 10 buses as replacement!
Combine this with a temperature lower than city people are used to (-20 C yesterday) and we have chaos. Or at least that’s what media are trying to make us believe. We’re actually being told to stay at home if possible.
I’m not sure that I can get to my office by bus, but it’s only a 20 minute walk anyway, so I might go there. My plans for the day are debugging Elixir (oh the joy) and then finish moving a couple of sites from my old vps playground to EC2. I’m starting a new project, one of my own ideas, and was going to upgrade the vps a bit. I run Ubuntu on it and haven’t touched the server in a while.
Problem number one: Making a snapshop of the vps
The snapshot function is provided by Glesys, the server hosting company, but every time I try to make one the system stalls. I then have to ask them what’s going on, after a while they do something, and two hours later I have my snapshot. Their vps system can only handle one task at a time, so as long as the snapshot is (not) being made, I can’t do anything else.
Problem number two: Not enough memory
The snapshot functionality would of course be great if it worked, so would the possibility to temporarily add more memory to the vps. This is something that is necessary for the dist-upgrade as I don’t pay for more memory than I normally need. The problem? As long as I have the extra memory I can’t do anything else with the system from Glesys’ admin, such as rebooting my server.
Last time I tried to do an upgrade something went wrong, the ssh server died and I couldn’t access anything anymore. Neither could I reboot the system from the admin (or log in to it for that matter). I’m pretty sure the upgrade problems were vps related, as a starter the /tmp folder was mounted noexec, which I had to change. I’m guessing other filesystem settings could cause problems as well. I’m not saying it’s impossible to upgrade Ubuntu on a Glesys vps, I’m just saying it takes a lot of unnecessary time.
The solution: EC2
The current project I’m working on uses EC2, and my boyfriend has been talking a lot about Amazon’s cloud services (I think he might be a little bit biased after having met Amazon’s CTO though).
EC2 is something I’ve considered for a while, especially since the pricing would be about the same as for the vps, and with everything being so frustrating and unnecessarily complicated with the vps I decided to give it a try yesterday. Three hours after making this decision, my site was up and running at Amazon and today I’ll get feminetik.se up and running as well.
Hopefully this will be a much better solution, and if my future sites need more server capacity, it’ll be extremely easy to fix.
No commentsSQLAlchemy? Really?
The last days have been very frustrating, as I’ve been trying to get SQLAlchemy to do as I want.
The problem?
Completely useless error logs, that’s the problem!
Turns out both of the problems that made me want to throw my laptop out of the window was related to different kind of keys and indexes, but this was by no means something the log would give me a clue to. The log typically told me where in the SQLAlchemy python code there was an error, but this information had no connection what so ever to my code or the actual error. Combine this with SQLAlchemys very unpredictable behaviour when adding data to tables where the primary keys of the baseclass and subclass didn’t match (SQLALchemy had generated these tables without complaining ffs!) and we have frustration galore.
ORM can be sweet and quick, I can understand that. But really? All the frustration when something goes wrong, and this way of only thinking “objects”? The resulting database tables after introducing polymorphism and inheritance to my model aren’t pretty. I also wonder exactly how optimized the resulting queries will be? Maybe it’ll work for simpler applications without heavy load, but this is more frustration than fun right now.
No commentsLonely coding
Coding all alone can be a problem. There are plenty of developer communities online, but sometimes I get stuck and just need a fresh pair of eyes to take a look at my code. I like working alone, but I also believe in code reviews and pair programming and that’s often not an option for me these days.
Yesterday I’d been struggling with a stupid SQLAlchemy error for hours and the error logs basically said “doing a rollback, something went WRONG”. The more the feeling of resignation came crawling, the smaller the chance of effective debugging. Before freelancing, when I always had a bunch of developers around, I would simply have asked one of them.
Luckily I have in-house expertise, living with a developer, but how do freelancers normally solve this problem?
1 comment
Networks
