Flutterby™! : Optimizing Python

I've been writing more and more Python code lately. Being an old C hack, I find Python to be a lot of fun. I can write essentially the same code with fewer syntactical errors due to the over-use of punctuation in C. But Python isn't C and my resulting code tends to be pretty slow. For instance, I recently wrote a function to return the next occurrence of a character in a string that isn't inside double-quotes. In C, it's pretty straight forward:

int findUnquoted(char * s, char f) {
  int q = 0;  // Means "not inside quotes"
  
  for (int i = 0; i < len(s); i++) {
     if (!q && *s == f) return(i);
     
     if (*s == "\"") q = (q + 1) % 2;

     s++;
  }

  return(-1);
}

This is pretty snappy because of the way C let's you work with memory pointers. Of course, this is the source of the worst kind of C bugs. This code assumes 's' points to a valid, nul-terminated string. If it doesn't, this could easily become an infinite loop. It also assumes that len(s) is less than MAX_INT, which is pretty safe on most modern systems. The code is parsing XML and you're not likely to be parsing really long XML files on an 8-bit processor. My initial Python version looked very similar:

def findUnquoted(s, f):
  q = 0
  for i in range(0,length(s)):
    if not q and s[i] == f:
      return(i)

    if s[i] == '"':
      q = (q + 1) % 2

  return(-1)

But this was really, really slow. What's much faster is to rely on the string functions in Python (which are implemented in C):

def findUnquoted(s, f):
  p = s.find(f)

  if p < 0:
    return(p)    # Not found

  q = s[:p].count('"')

  if not qc % 2:
    return(p)    # Found with even number of quotes preceding

  # Special case of being inside quotes
  nq = s[p:].find('"')

  p = p + nq + 1 + findUnquoted(s[p + nq + 1:], f)
  
  return(p)

Which is much, much faster but still has some poorly handled conditions, like if f is found inside quotes but never outside. But I am parsing XML files and looking for the closing '>' so I don't expect that condition to ever occur in valid XML. What is interesting is having to think of the problem on a higher level of abstraction. In C, it's a matter of walking through the characters in the string and monitoring my state. In Python, I am considering what the quotes mean relative to the search. In fact, I probably wouldn't have used the modulus operation in C if I hadn't done this Python version first. This is a fairly subtle example of how Python encourages higher abstractions. But when you combine the higher abstractions with the efficiencies inherent in an interpreted, dynamically-typed language, you start to experience quite noticeable boosts in productivity. And carefully written Python need not be much slower than good C code. Next on the to-do list is to play with map() and list comprehensions. }