Category: General

QuickFIX/J and Odd Memory Leaks

This is more a case of posting here to remind myself, but maybe it will help someone out. I just ran into a case where QuickFIX/J seemed to be stacking up endless messages in a LinkedHashMap object (messageQueue) in SessionState.java (discovered using a heap dump and the helpful Memory Analysis Tool from Eclipse) and spent a few hours banging my head on the wall trying to figure out why. After I bothered to read the QFJ code, I understood why, but it’s non-obvious.

Essentially, QFJ’s Session class parses each incoming message, and then attempts to parse any “queued” messages from the aforementioned LinkedHashMap before doing it all again. The contents of the LinkedHashMap (in my case, built up to 500+ megabytes before bringing the JVM to flaming ruin) are only messages that result from a “sequence number too high” error. So if your QFJ application throws, for example, a repeated NPE while handling FIX messages, in certain cases this can trigger a “sequence number too high” problem and will result in FIX messages stacking up in that QFJ-side queue.

I’m not sure if this is really a bug in QFJ or not — I should have checked the QFJ event log sooner, and I might have solved this without noticing the pileup. I suspect that’s why there isn’t a lot of information to be easily found online about the issue – other people just debug their applications correctly.;) That said, it’s non-obvious to me that anything I do in my QFJ application should result in some map inside QFJ’s core classes growing out of control. Hence, noted for myself and anyone else who may care/be curious.:)

Your Zodiac Sign Didn’t Change.

In the interests of combating ignorance, I’ll make a note about this “zodiac change” everyone has been going on about.

YOUR ZODIAC SIGN DIDN’T CHANGE!!!

I’m an interested non-believer in astrology. I find it amusing, systematic, good for entertainment purposes and a reasonable psychological self-inventory system at times. I don’t really believe it has predictive or prescriptive power, but I’m a general agnostic so I won’t categorically deny that it might not for the same reason that I won’t deny the possible existence of a deity. I’m apathetic about either one. What I’m not apathetic about is people in positions of intellectual authority who spread misinformation in some misguided attempt to drive the dark ages of superstition out of the masses, or whatever nonsense.

The very short reason that your zodiac sign didn’t change: you’re most likely a Westerner, accustomed to the not-very-accurate zodiac divisions laid out in newspaper astrology. These dates are aligned to a tropical geocentric zodiac — what this means is that the start of the zodiac is defined by the moment of the vernal equinox, the start of spring. This is usually sometime between March 20-22. This first sign is called Aries, because once upon a time, it was noted that the sun appeared to be in Aries from our vantage point here on earth at that time of the year. The visual change over thousands of years is the “precession” that the linked article refers to. While the background starscape has shifted since the zodiac “time segments” were named, some fundamental scientific facts haven’t changed.

The vernal equinox is the point of the earth’s orbit around the sun where the earth’s axis is exactly halfway from being inclined away from the sun. When the incline of the axis points directly away from the sun, it’s the winter solstice, and the coldest part of the year for the northern hemisphere. Since the earth’s orbit is circular, there are 360 degrees in the orbit. Divided by 12, these roughly align to the 365 days in a year (not quite, which is why the actual start date for zodiac signs shift from year to year), starting at the vernal equinox which is STILL called Aries in Western astrology.

As a result, based on the system that newspaper astrologers here in the West use, you’ll STILL be Aries if your birthdate falls in the March 20-April 20 range, give or take a couple of days. If you believe that the stars are more important than the much closer planets as astrological influences (or you just like your “new” sign better), you can go look up Vedic/sidereal astrology, which uses the apparent position of the sun against the starscape instead of the earth’s orbit around the sun to define the signs. Otherwise, you are still a Virgo or Cancer or whatever, and there is no such thing as Ophiuchus.

New Novel Posted – FourWar

Just a quick note to mention that I finally posted my second novel (finished right at the end of 2005!) on this website under my writing section. FourWar is a bit (okay, a lot!) less fantastic than Alice, trading off the dreamlike qualities for something best described as a sort of sociological cyberpunk scifi blend, but I’ll let my readers tell me exactly what it is.;)

Additionally, I’ve posted a reformatted version of Alice, which should be somewhat more readable than the first version (single-spaced now, Verdana instead of Courier, etc.).

Enjoy!

Music Update

I’ve decided to go ahead and make all of my completed (and one mostly-complete) tracks available here under a CC-BY-SA License. There are 13 tracks so far, all trance-influenced electronica, some with female vocals by Naomi. Any and all feedback is appreciated, but mostly — enjoy!

On a related note, I’m making decent headway on a new track which I’ll hopefully release before the end of the year if I manage to get enough breathing room around my upcoming web-based game project which is now running a private alpha. I’ll convert that project to public alpha shortly, so keep an eye out for a post announcing this!

FreeSwitch Directory and Lua

I’ve had occasion to play around with FreeSwitch lately and was looking at implementing a directory of extensions on the main menu, and so I set up Brian Snipes’ example Lua directory from the FreeSwitch wiki. I didn’t like the fact that the extension list was hardcoded, so I replaced the names{} table with the following bit of code. It’s ugly, but it’s my first 2-3 hours of messing with Lua. Anyway, it basically makes a system call to “grep” to pull the extensions out of your FS directory XML files using the effective_caller_id_* values which you may or may not have in each listing in the directory. Maybe this will help someone avoid having to reinvent the wheel completely.

Note: this presumes that each entry in the directory lives in its own users_[firstinitial][lastname].xml file. For example, I have users_nsimpson.xml containing the following two lines:

<variable name="effective_caller_id_name" value="Nate Simpson"/>
<variable name="effective_caller_id_number" value="1000"/>

You can hack the following code to match the particulars of your own installation. Also, I think string.gfind() has become string.gmatch() in Lua 5.1, so you may need to tweak that as well.


-- NS: build the names table from grep
directory_path = "/usr/local/freeswitch/conf/directory/default/"

namesearch = io.popen("grep effective_caller_id_ "..directory_path.."user_*.xml")
namelist = namesearch:read("*a")
namesearch:close()

names = {}

for thisuser, thisname in string.gfind(namelist, "user_(%a+)\.xml:%s+

SIMILE Timeline and XML String Datasources

I’ve recently been playing around with the SIMILE Timeline visualizer for XML data, which is pretty neat. I had one minor little gripe with it, which is that I wanted to be able to use data from an XML string in certain cases where I already was working in the context of an XSL stylesheet instead of hitting the server again just to present the same data in a different XML structure for the timeline. So I downloaded the code, told it NOT to use the bundled code in timeline-bundle.js by editing appropriately in timeline-api.js, and started experimenting with the sources.

What seems to have worked for me is the following modification [I added it just before the line starting: Timeline.DefaultEventSource.prototype.loadXML = function…] to sources.js (pardon the messy formatting here – you can correct the line breaks in the text editor of your choice 🙂 ):

Timeline.DefaultEventSource.prototype.loadXMLText = function(sometext) {
return Try.these(
// For Firefox, etc.
function() { return new DOMParser().parseFromString(sometext, ‘text/xml’); },
// For IE
function() { var xmldom = new ActiveXObject(‘Microsoft.XMLDOM’); xmldom.loadXML(sometext); return xmldom; },
// As a last resort, try loading the document from a data: URL. This is supposed to work in Safari. Thanks to Manos Batsis and his Sarissa library (sarissa.sourceforge.net) for this technique. (NS: – got this from timeline-helper.js in the GWT Timeline build)
function() { var url = “data:text/xml;charset=utf-8,” + encodeURIComponent(sometext); var request = new XMLHttpRequest(); request.open(“GET”, url, false); request.send(null); return request.responseXML; }
);
};

and in your calling script, replacing the line:

Timeline.loadXML(“example1.xml”, function(xml, url) { eventSource.loadXML(xml, url); });

with the following:

eventSource.loadXML(eventSource.loadXMLText(xmlstring,), “http://path.to.script/goes/here”);

Replace “xmlstring” with either a quoted string of XML or a string variable, and edit your path accordingly. Hope it works for you – it seems to be working fine for me!

MicroReviews: Gravity’s Rainbow, Artemis Fowl, Parable of the Sower

Yes, as you can see, I’ve been reading both the highbrow and the low. A little of everything for the win!

To start: Thomas Pynchon’s Gravity’s Rainbow is a pretty fun read. A large mix of history, science, the occult, and Pynchon’s very odd brain, this one took me a bit longer to read through than usual due to its near-800 page length. An interesting jaunt through Europe and elsewhere set largely in the culmination and aftermath of WW2, laced with original songs, intricate descriptions, silly stories, myriad external references, and obscenity to rival Team America, it’s easily one of the best books I’ve ever read. Recommended, but only if you have a strong stomach and aren’t easily offended.

Octavia Butler’s Parable of the Sower is an interesting dystopian story that seems a bit like the anti-Anthem (Ayn Rand) to me. Lauren Olamina is a girl gifted with quite a bit of foresight, paranoia, and a special talent which is also an Achilles’ heel of sorts, and she copes with the destruction of her family and everything she’s known. In the course, she comes up with something of a new philosophy/religion aimed at a more communal and earth-friendly existence. Interesting read, pretty violent at points. I’d read other novels by this author, to be sure.

And finally, Eoin Colfer’s Artemis Fowl and its sequels are fun reads aimed at kids. They read a bit like tame versions of ground trodden by Jim Butcher and Laurell K. Hamilton (less sex and violence), with more science fiction thrown in. Still amusing if you can get past the technological implausibilities (tracing an MPEG through analysis of radioactive properties??) and focus on the characters. I won’t read them again, but it wasn’t an utter waste of time. Evil child geniuses with a touch of conscience are fun!

Coupla Things

First, I’ve finally bothered to add some of my artwork, so it’s in one place instead of spread out over a buncha blog postings.

Which brings me to a related update – I’ve started pencilling a basic comic-type storyline which I expect to be a bit of horror-comedy, or so I’m thinking now. These things never turn out the way you plan them, though. I’ve only drawn a page so far, so after I have 5 or 6 of them, I’ll throw them up here for feedback.

And I’ve finished another track, I think, which I’m putting some final touches on. Hopefully by next weekend I’ll have something to show.

And now back to your regularly-scheduled mundania…

“Don Lapre” Comment Spam Scam

I have no idea who Don Lapre actually is – he’s some sort of infomercial vitamin-selling scammer dude according to wikipedia. Anyway, somebody left a hand-crafted spam comment on my SORBS post today which actually referenced the content of the post, but the format of the signature was a bit strange so I jumped to the linked website (donlaprewilliams.com) before I even knew Don Lapre existed. The site is pretty strange, as it seems to be leaching content through an iFrame from carpeaqua.com and the only actual content on the page is a couple of header tags mentioning Lapre and some sort of salad dressing blurb that makes no sense. There are no actual outgoing links from this page, so I can’t figure out what the s[p/c]ammer is hoping to promote by doing this.

A bit of searching reveals another domain registered on the same day (donlaprejones.com) which is leaching content in an iFrame from sashafrerejones.com (some sort of musician, I think). This site contains the same salad dressing nonsense and no outgoing links. Whois checks don’t reveal anything useful, other than that they’re Go Daddy domains (what a stupid name for a company).

Maybe the s[p/c]ammer is intending to build up the pagerank of these pages and then use them to promote some stupid vitamin-selling scheme? Anyway, nice try, but that comment is history. And perhaps the webmasters at carpeaqua and sashafrerejones might want to do some creative rewrite rules to block the leaching.

Moral of the story: I fucking despise sleazy advertisers.

Update – 06/16/2011

Seems like karma caught up with this dude – just spotted a story saying he’s been charged with fraud to the tune of $52 million. Why does that not surprise me?