Flutterby™! : Why XML and IT staff !@!K 2010-02-26 00:52:36.191529+00

Why XML and IT staff !@!K

2010-02-26 00:52:36.191529+00 by meuon 7 comments

I just got back from a users conference and saw a thing that makes me cringe: an engineer using excel to read in huge very nested XML files, and then use a macro that cut and pasted the data from one sheet, to another, to make it a simple one line per data reading spreadsheet, so that he could export it as a CSV, so the billing system would read it. A little over 10k records, 4 times a day, and it has to be run by a human, at least once a day. The macro takes an hour plus to run.

I replaced it in under 100 lines of simple basic perl, gave him the perl code and a link to ActiveState Perl for Windows. He e-mailed that to his IT staff... and they won't let him install Perl, and now he's in trouble for even asking to. His work laptop is locked down hard. The director of IT has e-mailed me legalese to fill out before he'll even talk to me about it and is accusing me of subverting his organization (I am).

All because I sat next to this nice guy at the conference while he parsed these files for his employer while in a lecture and I felt his pain.

Back when I still did the office temp thing regularly, I did a little data entry for a small non-profit who kept track of things in Access. A couple times a year, they'd pay three or four people for a week or two to correlate big stacks of signed paper documents with registrations entered via a basic web form. About 70% of the job, in terms of time, consisted of typing a number visible in one part of the Access app into a field visible elsewhere on the screen.

I'll bet that just about everyone who's worked with computers in the last 20 years has a similar story. This stuff is everywhere. What's depressing is that I'm now one of the folks in charge of automating the bureaucracy at a company with way more resources (one full of technologists, even) and I can just about guarantee you that somewhere in the building where I work, people are doing a bunch of equally numbing and pointless things in lieu of asking for real automation.

Of course, I'm also pretty sure that it will be years before I can get around to providing all the automation I've already been asked for, so maybe I should just keep my mouth shut...

#Comment Re: made: 2010-02-26 16:10:09.08443+00 by: meuon [edit history]

I know just enough vbscipt, and this isn't that hard I'd give it a try of there was an M$-machine within spitting distance to play with. Hmmm...

Things like this are why I'm glad I work in a place where central IT is weak or nonexistent. Sure, there are economies of scale for centralizing/standardizing processes, and god knows I've wished for more standardization - but I also don't have to deal with the "Mordac, Denier of Information Services" mindset which seems to infect all IT departments over a certain size.

Hrm.. I'm wondering if you could have end-run them by using vbscript instead of perl. Then no new install would have been required.

Back in '96 or '97 working at a Web hosting company (one of the first in South Florida I do believe) and one of our customers wanted to make reports available to their sales reps via their website. Despite having the go-ahead from the CEO, their CTO was not thrilled at all. So we ended up receiving (via FTP) a huge report that we had to re-parse into our own database to serve up the information (and yes, the format on the report would “mysteriously” change from time to time).

And then I wonder, how many jobs would be eliminated if computers were used to their full potential; without all the stupid work-a-rounds?

I'm seeing a number of places where a half a day of Perl and some web forms plus a bit of DB could move mountains, but concerns about support and lifecycle and the whole bit mean that such things will only happen if they're contracted to outside companies with long-term support contracts.

Thought about getting in on such things, but it's not worth doing myself if I have to go through the bidding process, so the whole thing remains stagnant.

Yep, I understand the processes that lock organizations into this sort of thinking, I see how they get there, and it makes me want to lay about with a cricket bat.

I used to work at a Large Consumer Online Company. There was a "tiger team" in one of the call centers that concentrated on particularly bad problems. As calls came in, they'd fill out a form on-line, which generated an email, which someone then pasted into an excel spreadsheet, and then much later someone would process that data (most likely shlepping it into Access).

After hearing a friend complain about the process, I tried to get some kind of db-backed system set up. No-go. The DBAs demanded a schema definition (no assistance for me, who was a db-noob), plus cost- center jazz, etc.

I ended up running a Solid-brand db on my workstation with a simple webserver front-end for a year and a half. Made the tiger team folks *very* happy, since I could quickly throw together slice-and-dicing, and made it easy for them to make custom info gathering templates. They finally got it officially supported and moved to the main Sybase instance on my last day of employment, when they really had no choice.