Dalke Scientific Software: More science. Less time. Products
[ previous | newer ]     /home/writings/diary/archive/2007/05/23/oe8bitimage_to_png

OE8BitImage to PNG

I want to make an web-based image server for OEChem using TurboGears. OEChem is a chemistry library supporting chemical informatics, molecular modeling and similar fields. It's in C++ with bindings for Python, Java and perhaps others. For the work I do it is the most powerful toolkit available. My major complaint is that the API isn't as graceful as I want. My next complaint is that the documentation needs work, hopefully by someone with documentation writing experience.

OpenEye's 2D chemical structure layout and depiction/rendering system is called Ogham. The name comes from the name for a celtic alphabet. Until recently it was distributed seperately from OEChem. I believe they are now distributed together, although you still need different licenses for them. Previously you had to download each library, which tended to cause version mismatch problems.

There is some documentation and demo code for Ogham but it's incomplete. Take a look at the example mol2gif.cpp or if you have the distribution look at wrappers/python/examples/depict/mol2gif.py for a Python equivalent. Actually, take a look at both because each uses a few API calls not used by the other.

I wanted to depict capsaicin, which has a SMILES of "C1(=C(C=CC(C1)CNCCCCC=CC(C)C)O)OC". I pulled out the parts of mol2gif.py that I understood and tried it out. After fixing typos, and using OEWriteGIF instead of the non-existing OEWritePNG, I got some output. Wrong output, but still output.

Checked through the code and found I had forgotten to generate 2D coordinates. Fixed that, done. The output functions OEWrite* (GIF, RGB, PPM, EPS, BMP) write the depiction to an output stream and not to a string. I want the string because that's the easiest way to get the data in a form that I can pass back to TurboGears.

OpenEye includes the "oeosstream" class for exactly this purpose. Following the C++ convention this is an OpenEye ("oe") output ("o") string ("s") stream ("stream"), meaning that the output is saved to an in-memory structure that implements the output file API. This is similar to the (c)StringIO modules.

The final code is this:

from openeye.oechem import *
from openeye.oedepict import *

smiles = "C1(=C(C=CC(C1)CNCCCCC=CC(C)C)O)OC"
mol = OEMol()
if not OEParseSmiles(mol, smiles):
    raise AssertionError("oh, no!")

OEAssignAromaticFlags(mol)
OESetDimensionFromCoords(mol)
OESuppressHydrogens(mol)
OEAddDepictionHydrogens(mol)
OEDepictCoordinates(mol)
OEMDLPerceiveBondStereo(mol)


view = OEDepictView(300, 250)
view.SetSuperAtoms(False)
view.SetAromaticCircles(True)
view.SetAromaticDashes(False)

view.SetColorOnBlack(False)
view.SetBackColor(255, 255, 255)
view.SetForeColor(0, 0, 0)

view.SetShowHydrogens(True)
view.SetDativeBonds(False)


img = OE8BitImage(view.XRange(), view.YRange())

view.SetMolecule(mol)
view.RenderImage(img, True, 0, 0)

# Could also use an oeofstream() here
# and avoid writing to a temporary string.
ofs = oeosstream()
OEWriteGIF(ofs, img, -1)
s = ofs.str()
open("capsaisin.gif", "wb").write(s)
In this case I'm not returning the string to TurboGears so I could have written to the "capsaisin.gif" directly. I'm being pedagogical.

Converting internal image data

I really would rather have a PNG image. Even more, I want to use PIL to further modify the image. For example, I may want to use better fonts, or include semi-transparent overlays. OpenEye's library can't do those, and nor should a computational chemistry company try to provide a general purpose graphics library.

Take a look at the definition of OE8BitImage in include/oedepict/bmpimage.h .

class OE8BitImage : public OEDepictBase
{

private:
  void AssignDefaultColors();

public:
  unsigned char *ptr;
  OEDepictColor col[256];  
  int colcount;
The "ptr" almost certainly points to the image data and the "col" is a list of color indicies. They are available to Python, thought you need to do some tricks to access the raw data.
>>> img.ptr
<Swig Object at _00be8801_p_unsigned_char>
>>> img.col
<openeye.oedepict.OEDepictColor; proxy of C++ OEDepict::OEDepictColor instance at _70a08801_p_OEDepict__OEDepictColor>
>>> 

What's the problem? The img.ptr is a raw Swig-wrapped pointer. It has no properties or methods to retrieve the p_unsigned_char as a string. All I can do is get the pointer value

>>> int(img.ptr)
25738752L
>>> 
Python has the ctypes module, which lets Python access C variables and call C functions directly, instead of through a specially compiled extension. The easiest way to get the data is with "string_at".
>>> s = string_at(int(img.ptr), img.GetXSize() * img.GetYSize())
>>> s[:10]
'\x05\x05\x05\x05\x05\x05\x05\x05\x05\x05'
>>> 

What does that block contain? It looks like position contains an index into a color palette. To verify that, I'll check that there are no more than img.colcount different fields in the text.

>>> sorted(set(s))
['\x00', '\x01', '\x02', '\x03', '\x04', '\x05', '\x06', '\x07', '\x08']
>>> img.colcount
9
>>> 
Looks like 9 different colors.

What are the colors? According to the header file

class OE8BitImage : public OEDepictBase
{

private:
  void AssignDefaultColors();

public:
  unsigned char *ptr;
  OEDepictColor col[256];  
  int colcount;
so it should be the "col" attribute. In Python I'll take a look at the "img.col" field.
>>> img.col
<openeye.oedepict.OEDepictColor; proxy of C++ OEDepict::OEDepictColor instance at _70708a01_p_OEDepict__OEDepictColor>
>>> img.col.r
26
>>> img.col.g
181
>>> img.col.b
229
>>>
That seems right, except that there's only one color. I should have had to say "img.col[0]" to get the first color. There must have been some problem in the wrapping code. I've emailed OpenEye about it.

The data is still accessible. Ctypes to the rescue again. OEDepictColor is a Swig wrapper class to the underlying pointer. To get the Swig handle use "img.col.this", and to get the pointer value use "int(img.col.this)". I'll cast the pointer to a new data structure which can access the r, g and b components.

>>> class DepictColor(Structure):
...   _fields_ = [("r", c_uint8), ("g", c_uint8), ("b", c_uint8)]
... 
>>> cols = cast(c_void_p(int(img.col.this)), POINTER(DepictColor*256))
>>> cols.contents[0].r
26
>>> cols.contents[0].g
181
>>> cols.contents[0].b
229
>>> cols.contents[5].r
255
>>> cols.contents[5].g
255
>>> cols.contents[5].b
255
>>> 
From earlier I saw that color 5 was almost certainly the background color, and this verifies that that field is white.

I'm going to create a PIL Image from the raw image data. PIL supports "fromstring" and "frombuffer". The latter can be faster because it may reference the memory block passed in rather than allocating a new chunk of memory. As it turns out, fromstring and frombuffer have two different origins: lower-left and upper-left, respectively. Because OE8BitImage also uses the lower-left it was simpler to use fromstring.

w, h = img.GetXSize(), img.GetYSize()
pil_img = Image.fromstring("P", (w, h), 
                           ctypes.string_at(int(img.ptr), w*h))

(I might have been able to convert the integer pointer value into a buffer object that frombuffer could use, although I couldn't figure out how. Supposing I did, I would still need to "transpose(FLIP_TOP_BOTTOM)" the image, which in PIL creates a new image. I wouldn't gain any memory or performance.)

Last is to copy the palette over. The PIL Image class has a "putpalette" method which takes a single string of the form "rgbrgbrgb...", with 256 colors in the form (r,g,b) and where each color component is an unsigned byte. That happens to be exactly the same as how OE8BitImage stores the palette information. I can get the palette data using "string_at" to fetch 256*3 bytes and pass that string directly to "putpalette":

pil_img.putpalette(ctypes.string_at(int(img.col.this), 768))

Putting it together, the code to convert from an OE8BitImage to a PIL Image, then save the result to PNG format is:

import ctypes
import Image

w, h = img.GetXSize(), img.GetYSize()
pil_img = Image.fromstring("P", (w, h), 
                           ctypes.string_at(int(img.ptr), w*h))

pil_img.putpalette(ctypes.string_at(int(img.col.this), 768))

# Presto!
pil_img.save("capsaisin.png")
Simple and clean.

Note that OpenEye does not document this interface nor describe it anywhere other than bmpimage.h. If you use this conversion code be aware that the internal format could change in the next release, or never. You should write test cases that can catch changes. For example, write to (1,0) and make sure that the resulting image is as expected.

Coventions

The OE8BitImage -> PIL Image code is clean because programmers use conventions a lot. Colors are rgb, in that order. 0,0,0 is black and 255,255,255 is white. Sure, there's an object interface, with r(), g() and b() methods so the order shouldn't matter, but by staying with the convention even in the underlying data structure means I can copy one memory block to another and skip the API. (Heresy!)

Similarly, there are several ways to lay out the image data: a block of memory in row major form (C convention), or a block in column major form (Fortran), or list of pointers to individually allocated rows or columns. Allocating a block is easier, and OE and PIL use the C convention, so the major ambiguity is the location of the origin and that's easy to handle. There are other ways to lay out the data, but it doesn't make sense to do it another way.

It wasn't quite as simple as I make it out. The Ogham library could have used three blocks of memory, with a block each for red, green and blue. That is, width*height of 8 bits of red, then another width*height of 8 bits of blue, and then of green. I tested for that by looking at the next couple of memory blocks. I could see they contained code data (there were function names in the text), hence weren't in the image.

Suppose I DrawPoint(x=1, y=0). I would expect to see the image data change. Here's the image data before the edit

>>> string_at(int(img.ptr), 10)
'\x05\x05\x05\x05\x05\x05\x05\x05\x05\x05'
>>> string_at(int(img.ptr)+img.GetYSize(), 10)
'\x05\x05\x05\x05\x05\x05\x05\x05\x05\x05'
>>> string_at(int(img.ptr)+img.GetXSize(), 10)
'\x05\x05\x05\x05\x05\x05\x05\x05\x05\x05'
>>>
and here's the change and the result
>>> img.DrawPoint(0, 1, 2)
>>> string_at(int(img.ptr), 10)
'\x05\x05\x05\x05\x05\x05\x05\x05\x05\x05'
>>> string_at(int(img.ptr)+img.GetXSize(), 10)
'\x02\x05\x05\x05\x05\x05\x05\x05\x05\x05'
>>> 
That tells me how memory is laid out with respect to the coordinate system. What I did next then was to generate the image and see if the result needed to be flipped.

I didn't have to extract a string from the image. I could have used ctypes to access the memory block directly. In that case there will be no memory copy and changes made by Python affect the actual OE8BitImage. Here's how I figured out how to do it, though there may be a cleaner way.

>>> arr = cast(c_void_p(int(img.ptr)), POINTER(c_uint8*img.GetXSize()*img.GetYSize()))
>>> arr.contents[0][1]
5
>>> arr.contents[1][0]
2
>>>
Using the array interface I'll show that changing the array contents changes the underling image
>>> string_at(int(img.ptr)+img.GetXSize(), 10)
'\x02\x05\x05\x05\x05\x05\x05\x05\x05\x05'
>>> arr.contents[1][0] = 4
>>> string_at(int(img.ptr)+img.GetXSize(), 10)
'\x04\x05\x05\x05\x05\x05\x05\x05\x05\x05'
>>> 

Star Trek

This bit of work reminded me of a scene from The Galactic Whirlpool. It's a Star Trek novel by David Gerrold and was my favorite of the various novels. I used that link because it shows the cover image I remember. Todd Richmond wrote a better review:

Star Trek novels are great for answering all of the questions that we have during episodes which are never answered. What exactly is Uhura doing when the Captain orders her to hail an alien vessel? Gerrold tells us.
I specifically remember that scene. Uhura receives a signal from the ship and has to figure out how to decode it. Does it contain audio? video? both? something else? Encoded as AM, FM, polarized, or something else? How are video signals encoded? And so on. It's a complicated job. And what I did this afternoon was a small subset of that.

Of course in the case of the book the RF source was from a human ship and if the Enterprise had decent computers they should already be configured to recognize any known/standard transmission format. I hope if that situation should ever occur the computer would say "it's broadcasting TV on VHF channel 11". But the book was, after all, written in 1980 and people had very little experience with computers.

The series Enterprise did a better job of explaining the communications officer role. Hoshi Sato was a trained linguist, which was especially important in the pre "universal translator" days. Looking at Memory Alpha just now I'm reminded of why I stopped watching Enterprise after the first season.


Andrew Dalke is an independent consultant focusing on software development for computational chemistry and biology. Need contract programming, help, or training? Contact me



Copyright © 2001-2013 Andrew Dalke Scientific AB