Flutterby™! : My day started by discovering that I'd

Next unread comment / Catchup all unread comments User Account Info | Logout | XML/Pilot/etc versions | Long version (with comments) | Weblog archives | Site Map | | Browse Topics

My day started by discovering that I'd

2014-05-20 23:05:06.069597+00 by Dan Lyke 5 comments

My day started by discovering that I'd screwed up billing on almost 20k accounts. Gulp. I think I've dug out now. How's your day been?

comments in ascending chronological order (reverse):

#Comment Re: made: 2014-05-20 23:17:16.767183+00 by: meuon

I've done a couple of thousand that the invoices actually reached customers for... Usually it gets caught and fixed before anyone knows. The print and/or email invoice logic is a very different interface and step, because we learned that lesson. In that case where they got printed and mailed/emailed, a human verified the significantly unusual amount billed as correct, after the interface put red error messages all over the place that this was more than a 20% variance.

the funny part: few people noticed and complained, most just paid. Even some -large- customers with bills > $100k USD. The corrected invoices sent a few days later with credit adjustments on them caused more calls and issues.

#Comment Re: made: 2014-05-20 23:41:38.225799+00 by: Dan Lyke

Yeah, part of the digging out was trying to decide which customers we silently fixed and which ones we notified and how we notified to cause the lowest resulting support load.

Ack.

#Comment Re: made: 2014-05-21 00:12:19.568898+00 by: spc476

I found out why the SIP messages from point A (my laptop) to point B (a development server ... somewhere ... in the US) were not getting through (some network device on the path was helpfully chewing them up and spitting them out somewhere else).

#Comment Re: made: 2014-05-21 17:16:50.621006+00 by: Dan Lyke

Oh great! One of those "on the phone with XO or Level 3 all day" sort of happy fun times?

#Comment Re: made: 2014-05-21 20:24:48.355502+00 by: spc476

No, more like a week of "okay, a test program that sends ping-like messages via port 5060 works, but anything dealing with SIP doesn't, I'm not going mad, am I? Oh, you want me to reboot my machine 'just in case' followed by a 30 second conversation with the Right Person that explained everything" happy fun time.