SSH proxying via Apache

Been working out of a customer site in the past few weeks and their firewall is a bit finicky. One day it would let me ssh into my office fine and the next day it would just kick me out right after the initial handshake. It doesn't actually refuse the connection at the onset. It would connect and then immediately drop the connection. Their IT guys tried to tell me it's my server, not their firewall that was the culprit. Hey, I'm an IT guy too, buddy! (amongst other things). If I can connect fine from my home network and two other guys can also connect from their home, I'm no Sherlock but something tells me it ain't my server. As improbable as it may be, because I know you think your firewall is perfect and that you haven't made any change to it that might cause this. My friend, I'd hate to tell you this but, it's your firewall! ;-)

Right. Next time, you try telling your customer's IT guy that his network firewall is faulty. See if that will get you anywhere.

Oh well. At least this gave me the chance to look into enabling our web server to allow SSH proxying. Found a useful article here: Tunneling SSH over HTTP(S).

This is essentially what you need to add to your httpd.conf on the Appache server:


# HTTP Proxy for SSH
AllowCONNECT 22
ProxyVia On
<ProxyMatch (192.168.1.1)> # Internal IP of your SSH server
Order deny,allow
Deny from all
### External (customer) sites allowed to connect
Allow from 199.243.1.61
Allow from 74.100.102.21
</ProxyMatch>


Then, to connect from the remote site, configure your PuTTY Connection host name to 192.168.1.1 (the internal IP address of your SSH server), set Connection>>Proxy setting to use HTTP proxy, enter in the public hostname and port of your Apache server.
That's it. Painless.

A slight diversion from real work, but this will come in handy the next time I'm at a customer site that blocks out all ports except port 80. I need my network to follow me everywhere I go. I'm effectively crippled without it.

Unless...I wonder, what would happen if the customer's network itself uses a proxy server to get out to the Internet...
Oh well. Not my problem to worry about right now. Will deal with it when I run into it.

Google and YouTube


From YouTube.

This Everly Brothers' song is a classic--brings back old memories of high school nights, sitting by the radio, doing my homework.

It wasn't until today when, by chance, I watched the video clip of this song, that the impact of Google's YouTube acquisition finally hit me: it's so totally consistent with their operational patterns ever since Google's inception in 1998.

In 2001, Google acquired the archives of web based UseNet search service called DejaNews, and as the result, surfers can search UseNet archives dating back to 1981. By the way, my "official" birth date as a netizen is Mon, Jun 21 1993 12:08 pm.

As the world turned, in 2003, Google acquired Pyra Labs, the maker of Blogger.com.

And then now...YouTube.

Microsoft's legacy may have been to make personal computers a common household name. Yahoo's legacy may have been to bring web based email to the consumers. But it may be the case that Google's legacy will be: to preserve memories.

May be I'm giving them all too much undued credits.

Posting from Windows Live Writer

Trying out the new Windows Live Writer Beta, posting to my blogspot blog.  Fingers crossed and hoping that it doesn't mess up my template (I have made a backup, just in case).

I'm liking it.  The user interface for this thing looks pretty simple (I like it simple). Apparently it also supports WordPressTypePad, and a few others. I'll try it out on WordPress next.

VB Interview Question

This is one of the easy-level questions that I give my interview candidates on the written test part of the interview:

Assume that a and b are Integer variables.

i. If a And b Then Call DoSomething()
ii. If (a And b) = b Then Call DoSomething()

Statements (i) and (ii) are equivalent. True or False?


I thought this was a no-brainer, but apparently not. This is often the source of many bugs in VB code.

So how many of you think the answer is True? ;-)

New Look

I recently tried the WordPress.com service. WordPress, the software, is nice, but the hosted service itself is limiting in that you can't customize the layout template--well you can, but i'll cost you 15 credits to upgrade. Blogger.com, on the other hand gives you full freedom to customize the layout as you'd like--hence the new look of this blog. I was thinking of switching to WordPress, because of the lame layouts that Blogger provides, but having discovered this, I think I'll stick with Blogger for a while longer.

Legend of the Pink Tie

UW Math Faculty's web site has an interesting article about the Pink Tie--an icon for UW mathies.

Back in those frosh days, my pink tie got tie-napped during orientation week by those damned artsies. Lousy TLO! Sign me up for the TG. I want my pink tie back, damn it!
:-)

ActiveLock lives on!

It's been over a year since the time I left the Activelock project, due to lack of time, after having coded the baseline for 2.0. I just checked in to have a quick peek a few days ago, and it looks like it's made great strides since. Kudos to Ismail for doing a great job keeping the project alive.

Back then I had to deal with low-level RSA and MD5 bit mangling in pure C/C++. Granted that much of the encryption code was "borrowed" heavily from the PuTTY project, but, boy, was debugging it ever a nightmare.

If I were to do it all over again now, I'd rewrite the core in .NET--It's sooo much easier now with all the encryption facilities built into the core .NET framework.

I've noticed that every time I do any code weaving in .NET, it seems to take 10 times faster to get things done than doing it the old ways. Suddenly, you're presented with many new possibilities.

How far is that extra mile anyway?

We have all heard of the phrase "going the extra mile" when people talk about providing exceptional service to their customers. And I'm definitely a proponent of this mentality.

However, I've also once heard of the phrase "your lack of planning does not constitute my emergency", murmured by a former colleague when referring to a particularly demanding customer, and also think that it makes perfect sense.

At the office, I sometimes get calls from our partners/resellers with requests of the kind "um...I've got this demo in 2 hours, and I need your help to build this integration against this application that I want to show for my demo."

First thought that always came to my mind was that wonderful phrase uttered by the colleague. I mean, c'mon! My day is usually fully planned out and these kind of things really throw a monkey wrench into things.

My following thought would be, well, they are trying to sell our product for us, and consider the alternative: I tell the partner to blow off and tell the customer to reschedule and give us more time. On such close notice, this would make the partner look very bad in front of the customer, not to mention the partner might have made a long trip onsite for this demo--all of which aren't the end of the world, but a lost opportunity nonetheless.

With that thought, I put my regular schedule aside, got online with the partner and in 2 hours, whipped up a prototype demo into shape, in time for them to show the customer. Everyone was happy...well, except may be me! <whine>You've taken 2 hours of my life on something you could have done yourself, and I want it back!</whine>.

So, although I think that my colleague's "lack of planning" speech is absolutely spot-on--just like the theory of communism is absolutely spot-on--unfortunately, just like communism, it's not very practical ;-)...which brings me to the subject of this blog post: how far is that extra mile? I'll get back to you when I have the answer.

Dead Dog's back

I used to love listening to the The Dead Dog Café on CBC RadioOne every Sunday morning. It had some really witty silliness sketches, done from an aboriginal perspective. I'm not aboriginal (heck, I don't even know if I'm original) but I really enjoyed this show, which was why I really missed it when it suddenly disappeared off the airwaves in 2000.

I tuned into CBC's Sounds Like Canada program on the way to work this morning, and noticed that it's back--I guess after 6 years, somebody at the CBC finally decided to respond to popular demands.

Noice!

Ingenuity: striving to stay ahead of the curve

As we've been struggling a bit in the past several weeks, dealing with the slight inadequacies of some of the less user-friendly third-party SDKs that we've had to work with, I'm reminded of the tale that might help boost some creative mojo's for us all. It's a tale about how we created one of our very first plugins for AppConnector four, five years ago, an exercise that attested to the ingenuity of our research team, of which yours truly had the great honour of being an insignificant contributing member.

We needed to build a plugin to integrate with this other application, but it had no documented API into its user interface (AppConnector is all about integration at the UI level), and the vendor had neither the interest nor the intention of helping us find out. Hmm...Sounds familiar? Anyway, our research team had to study it the hard way, devising various experiments to feed it various inputs and observed its responses to come up with some consistent patterns, and let me tell you, ladies and gentlemen, for a bunch of lowly developers and business analysts, our process was so scientific that we could have claimed a SRED grant for it if we wanted to. It was. ;-)

We did run into some snags, through no faults of our own, however, but rather it was a limitation in their product. I'll never forget the conversation with their support guys when we called. On the phone, we told them that we were trying to do this and this, driving the user interface using that and that, and we expected it to behave this way but it behaved that way, blah, blah, blah, and yadiyadiyada, is this a bug, and can you help? At the end, they said they would talk to their developers and get back to us. What happened right after that was totally hilarious, my product manager and I couldn't stop snickering as we listened. They started talking to each other as if we we had already gotten off the line. (sneaky us!). I'm paraphrasing from this point on as it is quite a few years back, but the jist of volleys were like this:

Guy A: Did you get all that? Did you understand what they're trying to do?
Guy B: Yeah, man, it's totally crazy s**t. Did you know that? What they're doing. Did you know that that can be done?
Guy A: No, man, that's pretty wild.
Guy B: ....
(dialogue continues for at least another minute...eventually)
Guy A: Anyway, let me talk to X and see if we can fix this for them. I'll let you know.


So...We figured out how to use the software in a way that the original creator hadn't conceived. We took it to a new level they didn't think possible. If that's not ingenuity, then I don't know what is.

The moral of the story here is this: we need to remind ourselves, from time to time, that we ARE the experts! People look to us for solutions to the problems, not the other way around--because we do things that they think impossible to achieve, and not because we're a small bunch of super geniuses, but because we invest, painstakingly, our blood and sweat, not to mention countless hours, in figuring out things that are too time consuming for them to do had they attempted to do it themselves. And although it is sometimes unavoidable to need some guidance from our customers/partners, most of the time we are the ones to show them how it's done. As little karoras in our own rights, we all have the creative spirits within us to do that, to make a difference, and to be a part of something of revolutionary potentials.

Recent breakthroughs made by one of the team members this week further convinced me that the feeling is mutual throughout our closed knit group. Kudos, MDM! Go get 'em, maestro!

Microsoft shocks world with Longhorn betas

Hmm...I wonder why the world would be so "shocked", seeing how Vista Beta 2 was at least 6 months overdue. :)

Skype Offers Free Calls to Regular Phones

Yep! It's true. (I just tried it!)
You can now skype out to regular North American phone numbers for free. Well, at least until the end of 2006.

Using the .NET Framework Class Library from Visual Basic 6

I stumbled onto this link while reading another blog. Using .NET framework classes from VB6? Now that's just plain crazy talk!
Or is it?

Freetext Search ≠ Meta Data Search ?

Gav reminded me the other day, when we were discussing the problem of searching, that freetext search does not equal metadata search (also known as field restricted search). Gav was right, of course, as it takes no stretch of the imagination to see that doing a (freetext) search, for documents whose content contains the words "John Doe", is not exactly the same as searching for documents whose Author is "John Doe" (metadata search). The problem with the former is that you may get results back with documents having "John Doe" the Author, as well as "John Doe" the Reviewer.

I claim, however, that metadata search and freetext search are computationally equivalent problems. That is, you can use a freetext search solution to solve the metadata search problem, and conversely, you can use a metadata search solution to solve the freetext search problem. In this post, though, I shall endeavour to prove the first part only, since that is the crux of the original point of discussion.

First, let's do a little bit of review in computational theory. We say that a problem A is reducible to problem B if we can apply a series of transformations that turn problem A into problem B, which, we hope, is less complex and more solveable than problem A. In effect, we're saying that if we could find a solution to problem B, then we would know a solution to problem A as well. It is a common approach that academic researchers like to use when dealing with computational complexity involving NP-complete problems, i.e. problems for which no solution has been found that can be computed in polynomial time. Researchers try to prove that a problem is NP-complete by trying to reduce it to another known NP-complete problem.

Well, I claim that the metadata search problem is reducible to freetext search. Or, in more layman's terms, you can satisfactorily simulate metadata searching using freetext searching capability. And below are the transformations that take the problem from the metadata search domain into the freetext search domain.

Let's say you want to index a PDF document with the metadata field: Invoice_Number = ABC100.
For this document, you provide a secondary file which will serve as the "index file" whose content contains some sort of encoding of the above name-value pair, in addition to the link to the real document. Something like:


<DocumentSummary>
<Invoice_Number>ABC100</Invoice_Number>
<A HREF="file:///C:/Accounting_Data/DOC00000018567.PDF">View Document</A>
<HASHCODE>51E88D049A06D7018D38740772FCAA0A<HASHCODE</CODE>
</DocumentSummary>


where 51E88D049A06D7018D38740772FCAA0A is the computed MD5 hash of the text <Invoice_Number>ABC100</Invoice_Number>.

You will then feed those two files to the freetext indexing engine.

Now then, since the MD5 hash is fairly unique, if I were to do a freetext search for the keyword 51E88D049A06D7018D38740772FCAA0A, it would be highly unlikely that I would get a mixed set of index file and real files in the search result, but rather I would only get a list of index files containing Invoice ABC100, which contained the link to the actual invoice PDF file itself.

Hence, we've proven that we can use a freetext search capability to do metadata search.
QED.

GMail ATOM feeds

Hmm...It seems that Google has been providing ATOM feeds into your GMail inbox for a while now. The question is, then, whether I should trust either of Bloglines or NewsGator enough to give them my GMail ID and password, so that they can fetch the mails for me. :-)

AppConnector on Linux?

I've been tinkering a bit with the Gentoo Linux distro, been scouring their documentation site, and I bumped into something interesting. This might make you feel a tiny bit dirty, but apparently you can run IE6 on Linux. So what does this mean? Cross-platform Windows binaries! Wine rocks! Years ago when playing with the earlier Slackware distribution, I got some msdos programs to run in Linux under Wine. But this is way more kewl!

More importantly, it implies that we might be able to get AppConnector running on Linux, too--without a single line of code change! AppConnector uses the MSXML Parser and the MS Scripting Engine which are bundled within IE. So if IE can run on Linux, there's a good chance AC might run on it as well. Imagine the possibilities...being able to mashup KDE and GNOME apps. I've got to try it sometime.

Hmm...I wonder if there's any good MacOS port for Wine.

HOW NOT TO compute the inverse of a colour

I once wrote a simple VB utility that acted like a window decorator, allowing the user to redact any window on the desktop by highlighting on a rectangular region within it. The highlighting code tracked the user mouse movements and drew the selected region as a hollow rectangle with black border.

My highlighting code had a problem: what if the window you're highlighting had a black background? That would render the highlighting rectangle virtually invisible, wouldn't it?

The code really should have used, for its border colour, the inverse colour of whatever the window's background colour is. OK...How the heck do we compute the inverse colour in VB?

Well, some quick googling yielded this link, which seemed to be somewhat useful:

Take any point in the [RGB] cube, then draw a line from this point to the centroid. If you then extend this line through the centroid the same length as between the original point and the centroid, you will have found the inverse colour to that defined by the original point. Inverting an RGB image involves a kind of turning inside out of the colour values.


uhh, yeah...

So, anyway, I did some more googling, and it turned out that there was a simpler way to do this in VB, thanks to this link.

The trick is in using a PictureBox and the vbInvert constant. The PictureBox will serve as the drawing canvas, onto which a snapshot of the actual window is placed for selection. The following simplified pseudocode illustrates the rest:


' Initializes PictureBox drawing settings
Private Sub Form_Load()
' Set drawmode to draw using the invert of the background colour
Picture1.DrawMode = vbInvert
' Set pen width
Picture1.DrawWidth = 2
...
End Sub
...
...
Private Sub Picture1_MouseDown(Button As Integer, _
Shift As Integer, _
X As Single, _
Y As Single)
If Button <> LEFT Then Exit Sub
LeftDowned = True
' Store [X,Y] as [Left,Top]
...
...
End Sub
...
...
Private Sub Picture1_MouseMove(Button As Integer, _
Shift As Integer, _
X As Single, _
Y As Single)
If Not LeftDowned Then Exit Sub

' Clear previous drawn rectangle
...
' Get current drag region [Top, Left,Bottom,Right]
...
' Draw new rectangle
Picture1.Line (Left, Top)-(Right, Bottom), vbBlack, B
End Sub
...
...
Private Sub Picture1_MouseUp(Button As Integer, _
Shift As Integer, _
X As Single, _
Y As Single)
' Re-set [Left,Top,Right,Bottom] = [0,0,0,0]
...
LeftDowned = False
...
End Sub


The moral of the story is this: There is a simple solution out there somewhere, waiting to be discovered. So keep googling, ya lazy bastard!

Google deleting GMail accounts?

Just caught wind of this news on the blogosphere: some people's GMail account got deleted. Read the full story here and here.

Sounds quite disturbing. This incident might send some users flocking back to the other free mail services. And to think that at one point, I had an idea to build a personal imaging app on top of GMail. It sounded like a good idea at the time.

I never was a big fan of GMail. The lack of foldering capability was the breaker for me. And tagging just ain't the same thing as foldering.

I highly doubt it, though, that Google would actually and purposely delete people's accounts without prior warning. More likely, it's a bug in their system--and a fairly big one, if that indeed turns out to be the case.

Weird problem with eMachines DVD-RW drive solved

I've been stumped by this problem with my eMachines M6809 laptop for a couple of years, ever since I first bought the beast. The problem was with the Slimtype SDW-431s DVD-RW drive when run under Windows XP. It would read the contents of the CD/DVD OK, but when I tried to launch a program from it, I'd get an error similar to this:

The procedure entry point blahblahblah could not be located in the dynamic link library blah blah blah.

Notice the weird casing in the procedure name MsgWaitForMultIpleObjects. It should have been MsgWaitForMultipleObjects.

For a long time, I couldn't figure out what the heck that error was about, and had dismissed it as a defective DVD drive and left it at that...At least until this weekend.

Last Friday, at approximately 9:30AM Eastern Standard Time, the hard drive on my lovely eMachine decided to pull a head-crash stunt, sending 5 years worth of my life into /dev/null (a long and painful story for another day), and once again I was motivated to get this damned DVD drive working so that I can do backups with.

This time, I stumbled onto this discussion thread on cdfreaks.com, and figured out what the cause was: it had to do with the Secondary IDE Channel setting used by the DVD device driver on XP. This setting is accessible through the Control Panel Device Manager. By default, the Transfer Mode for Device 0 on this channel property was set to "use DMA if available". I changed it to "use PIO only", and voilĂ ! Everything started working like a charm. Now it can read and run programs off the DVD with no problem.


Prior to finding the solution, I had tried googling like nobody's business, hoping to find some other people who might have experienced similar symptoms, to no avail. So now I thought I'd post my findings here, in hope that it could help some other poor souls out there who might have the misfortune of owning an eMachines laptop and have managed to bump into the same problem.

Mashing up the desktop?

Hey, have you tried out the new 3.0 release of Google Desktop? I've only begun trying it out for a few days now, but I must say that I really like it. Google Desktop can index everything from individual files, to emails stored within Microsoft Outlook or Mozilla Thunderbird.

This is great! Suddenly, my local hard drives have become one huge database that is easily and quickly searchable using one common interface, and the desktop becoming one integrated application.

From a technical level, I'm intrigued that the Google Desktop Bar is able to:

  • monitor new emails from my Microsoft Outlook accounts,
  • monitor new emails from my Mozilla Thunderbird accounts,
  • detect that the web page I'm viewing is a blog page, and monitor that blog for new entries (via its Web Clips feature),
  • notify me of new items from all of those sources using one common notification popup interface.

And then, when I click on the notification popup, it brings up the item in the native application: if it's an Outlook email, GD opens it up in Outlook; new Thunderbird emails get opened using Thunderbird; and new blog entries are brought up in the default browser.

It seems to me that this is an example of what Gavin called "mashing up the desktop".

It also seems to me that AppConnector is well on its way to be considered as a desktop mash-up tool, for the following reasons:

  1. It treats the desktop as one single integrated application.
  2. It is able to re-use and leverage existing applications' GUI as-is, without the need embellish them into its own GUI, thereby avoiding legal implications involving copyleft, copyright, and what-have-you.
  3. It mashes up two otherwise disparate functions (e.g. entering A/P invoices and maintaining electronic copies of PO's), normally provided by two different desktop applications, into one integrated (and more complete) solution.


Desktop, the final frontier. These are the voyages of the toolship "AppConnector". Its continuing mission: to explore new desktop environments, to seek out new ways to mash up new desktop applications, to boldly go where no mash-up tool has gone before.
(Star Trek theme music begins...)

And let the mash-up begin.

Google and China

In the past several weeks, The People's Army of Geeks has slash-dotted the heck out of recent news that Google began censoring searches on google.com.cn.

Give it a break, folks! Google makes it easier for people to access information on the global scale. My prediction: Google's presence is one of the catalysts that will bring about a new kind of cultural revolution, one which will see the end of communism in China without bloodshed.

The voice of democracy cannot be silenced forever. And there will be democracy when the people of China (or Vietnam for that matter) want it so much that their government will have no choice but to give it to them.

REST and the principle of encapsulation

I was playing around with my new Outlook Web Access (OWA) installation last night and it struck me that OWA appears to be a RESTful application. Good on Microsoft!

One of the main concepts in REST is the idea that all resources (in a general sense, not technical sense) in the system are uniquely addressable via a URL. It implies that the URL only contains the business end of the feature (e.g. viewing my inbox) and should have no technology association whatsoever (e.g. this feature was implemented using ASP.NET, which, by the way, OWA was).

In my simple-minded view, I think this encapsulation is very important for the following reasons.

As a (somewhat) alert software user, I couldn't care less if you implemented your software using ASP, ASP.Net, Java, Python, Perl, PHP version 4 as opposed to version 5, or what-have-you. Having a URL containing a tail of .asp, .aspx, .jsp, or .py immediately clues me into what technology you're implementing your software in, which may not be a good thing (see next paragraph). Plus, having technically specific parameters in the URL that has nothing to do with what I'm trying to do, like whether it should use UTF-8 or ISO-8859-1 encoding, is simply annoying.

As a developer, I would want to abstract the underlying implementation language away from my users for many reasons, security being one. If there's a security bug found in the current version of PHP, how do you think the hackers are going to target PHP sites? Yep, you guessed it: crawl the web searchhing for URLs with .php or .php4 extensions. Additionally, if later on, requirements change and it turns out that PHP just can't handle the scalability (sorry for the cheapshot, PHP fans), I can always pull the rug and re-implement the whole system using, oh I don't know, Ruby for instance, and do it without breaking any hyperlink that my users may have bookmarked.

Web URLs should be treated as public interfaces, and if you could change the underlying implementation without changing the interface, that would be ideal.

Accessing your intranet web sites through an SSH tunnel

If, from your home, you're connecting to your company's network via a Linux SSH server, and would like to to be able to access all of the corporate intranet web sites, here's how.

First, some assumptions, your SSH server machine must have the following software installed:

  • an SSH server that support SSH2 protocol
  • SSH daemon running with port-forwarding enabled. RedHat Linux has it enabled out-of-the-box.
  • a web (outbound) proxy/cache daemon. RedHat Linux comes with Squid. Note what port your proxy server is running on. Squid runs on port 3128 by default.


Instructions:
You'll be connecting to the SSH server using PuTTY.

  1. Set up a port-forwarding tunnel as follows: L3128=<your_proxy_server_ip_address>:3128

    View Full Image
  2. Connect to your SSH server using the above new settings
  3. Configure your browser proxy setting to use localhost:3128 as the proxy server.

    View Full Image
  4. Now restart your browser, and try accessing an internal web site, for example: http://shrike.karora.ca:8080/supportwiki/


Internal host names will work, since the browser now uses the DNS server on the proxy server for name resolution. The side effect to this is that you're now surfing the web as if you're doing it from your corporate office's computer. So beware of nosy, sniffy network administrators. ;-)

The whole thing is relatively simple to set up. The one pain point is the tunnel forwarding setup in PuTTY, not too intuitive for the non-techie users.

I've been thinking of writing (if I ever get any down time, that is) a Java Webstart application that does what PuTTY does, but simplifies the setup for the novice users. The administrator would configure the appropriate tunnel setting on the server side, then send a JNLP URL to the user to click on. The JWS application will launch and will take care of all the config stuff. The user won't have to mess with port forwarding settings or browser proxy settings, whatsoever.

If anyone knows anything out there that already does something like this, please let me know.