It started with the best of intentions, as these things are wont to do. What had begun as a test script, nothing more than the stub of an eventual server side implementation to catch data coming in from an embedded device, was quickly morphing into an actual project. Hm… getting serious and valuable: time to take this to the next level.
The Incident
As it had grown out of a little command line utility to test firmware, the code was messy and not even under version control. On this day, the third in a row of waking extra early to get on track with a looming deadline, I decided to set things right.
A quick git init and time for an initial commit. But this is Python, and it’s leaving these stupid .pyc files strewn about hither and thither every time I run it, so I’ll just clean that up. A .gitignore? Agh, I’ll do it in a sec. For now:
find /my/path -name “*.pyc” | xargs rm
and goodbye bytecode! Oh, and those damned __pycache__ directories… yeah, get those too. UP, CTRL-left CTRL-left CTRL-left… But which files are actually present, here, anyway? backspace, enter.
Wait… uhm… Enter?
ENTER!?!
So that was it, I’d just issued
find /my/path -name “*.py” | xargs rm
thereby scanning all subdirectories for .py files and deleting them.
All. My. Source.
Gone.
I seem to blunder into a situation that arouses this feeling–the feeling that you’ve just been unexpectedly pushed out of a plane at 10,000 feet–every 5 or 10 years. Thankfully, it’s usually only related to work but even so isn’t the start of a stellar day.
I froze. All those wonderful files, unlinked and floating in limbo, degrading with every passing second as I hear the hard-drive purring, whichever of the 20 programs I’ve got running just doing its thing, flushing its cache, maiming my code. Obliterating my work.
Backup? I do backups as a regular part of my day, right? Yes, my last backup is there. It’s from 2 days ago? Ok, I’m not dead, the project’s not dead, it’s not the end of the world… but these have been long and arduous days… wtf, man??
Shutdown, hard shutdown.
Undelete
This is when you take stock and realize exactly how exposed you are. I’ve got a two-day old backup. There are ways to restore files from an ext4 filesystem, right?
Searching on a mobile sux. I’ll have to setup a rescue USB key on some other computer, come back. There’s foremost that can be used to undelete… but it scans file contents to rebuild them, will it deal with Python? Do I have to make some entry in foremost.conf? There’s extundelete, that looks more appropriate… I set it up, run it… it doesn’t recognize my linux partitions? dammit.
Wait, what am I doing?
I’ve got tons of goodies on my drive–my own projects, circuit designs and source code to which I’ve sold the rights, confidential client information, etc.–and I’m paranoid about this stuff.
So if my hard-drive winds up in the wrong hands it’s pretty useless as everything of interest is strongly encrypted, notably the workdir in which all source code lies. Which means foremost and “scanning contents” are completely out of the question. Even if I recover blocks of anything, I’ll probably never, ever, ever be able to piece the puzzle back together, even with the decryption keys in hand. Faaaaaak.
Eclipse, my friend
So, I take a break and grab a coffee as the machine reboots, and I have a thought. I’d been using Eclipse on this as I find PyDev’s code completion more useful than atom or anything else I use.
I’d noticed, a good while ago, that eclipse had some sort of function to track changes, even when no versioning is setup.
Though I hadn’t given it more attention than a passing “hm, that’s pretty neat”, it did stay lodged somewhere in my brain. Through some unrevealed and magical process, my mind used the coffee break as the occasion to dredge up that tasty morsel and present it as something I might, perhaps, consider noteworthy given my current predicament.
Change tracking? As in, “implies history”? As in… “eclipse restore from local history”!
Yes. Yeeees. Right-click that top level source dir, “Restore from local history…” and, huzzah, there they are.
It’s not perfect. I’m not certain what the schedule is and any files you copied in or edited in vi aren’t there or are out of date, and more still are older than you’d expect even if edited in eclipse (maybe it was my hard shutdown). Still, a good 95% of my work was resurrected in moments.
Thank you Eclipse!
Lessons and procedures
Doesn’t happen too often, but I lose a whole lot of HP every time so it’d be nice to avoid a repeat. The lessons and rules, which I should’ve been following all along, are are:
- Everytime you start a project, or just a dumb little script just for testing, or whatever: git init right now. Period.
- Stop leaving some project backups to the human-rsync. Just set up duplicity for all projects and be done with it. Doing that right now.
- Thank you Eclipse (expect my financial support in the next 20 minutes, and my gratitude for a very long time :) )