HTTPS Everywhere

This has been a long time coming.

Errorception is now proudly 100% HTTPS. (Well, nearly 100% – read on.)

It turns out, migrating a site to HTTPS isn't as simple as it seems, especially if you have to do it right. I had a huge checklist to look at and verify for this launch. Here's what else has changed with this update:

• Save for this blog and one route that currently needs to be HTTP, all URLs are now strictly only accessible over HTTPS. HTTP access has been disabled. As for that one route and this blog, they are over cookie-less subdomains, and don't carry any sensitive information.
• Cookies are only set when using HTTPS, and have been marked as secure cookies. HTTP cookies that were set in the past are now meaningless. In fact, I've deleted the entire old session-store to ensure that there can be no session hijacking.
• The encryption is end-to-end. In this case, it means that SSL doesn't just terminate at the load-balancer. The connections between the load-balancer and the app servers are also all SSL. Everything is encrypted. Take that, NSA!
• Cookies will henceforth be inaccessible to client-side code to prevent a large class of XSS attacks.
• There are several other security measures implemented. For example, Errorception now implements HTTP Strict Transport Security, prevents clickjacking where possible, reduces MIME-type security risks where possible, and has force-turned on XSS-filters to prevent reflected XSS attacks.
• All external assets included in the site are now loaded over HTTPS as well, to prevent mixed-content scenarios. All links from communications like emails have been updated to use HTTPS URLs. Links that have been forged in the past will still work, but will be redirected to HTTPS.

Unfortunately, because cookies will have to be recreated over HTTPS and since the session store has been cleared to ensure that old cookies are invalid, it means that you will have to log in again to your Errorception account. It's a minor inconvenience, but it's a small price to pay for the vastly improved security.

I'm by no means a security expert, so if you find any lapses, please feel free to let me know. (I'm rakeshpai at errorception dot com.) Also, security is never really done, so I consider this as only the first step in getting to better security.

Why Throw When You Can Just Push?

I had previously launched a feature to .push your errors to Errorception if you liked. It also required you to throw your errors immediately after the push.

Turns out, many people don't like to throw their errors. There could be several reasons for not throwing your errors – for example, if you want to handle it gracefully, but still want it logged. Also, some frameworks such an Angular and Ember provide you with Error objects, but throwing them might not be what you want to do.

Now with Errorception, you don't need to throw error objects anymore. Simply .push them, and you are done!

I must hasten to add that I would consider this usage advanced. A throw, despite all its weirdness, is the best way to halt a problem immediately. If there's an error that you haven't anticipated, chances are you will want to stop your program's execution. If you are going to deal with errors properly, you don't need to throw them, but then you wouldn't have an error in the first place. This makes the feature only useful for a small set of cases. If you are not throwing your error, I hope you know what you are doing.

If your code was already throwing errors after pushing them, you won't have to change a thing. Don't worry – you won't get duplicate errors. In fact, this behaviour (push, with or without throw, without duplicates) will be supported forever, because of the reasons in the paragraph above.

Stack Traces, window.onerror, and the future

One argument that many people have made (and still make) is that window.onerror doesn't provide sufficient information to track down client-side JavaScript errors. While there's certainly some truth to it, I've always thought of it as a case of worse is better.

Consider the alternative: you'd have to wrap all your code in try/catch blocks. But that's not enough. Because of the nature of the event loop in browsers, you will also have to wrap all your async code in try/catch blocks as well. That includes all DOM event handlers, XHR and family including WebSockets, and setTimeout and family. That's something you'll just have to do. Because this requires modification of your code, it's terribly invasive. This also means that you will almost certainly miss catching errors in several cases, simply due to oversight. And that's still not saying anything about the performance overhead of working with try/catch blocks, not just in Chrome. All that, just to get some additional data.

Errorception lets you pass your error objects to us if you want to, since quite some time now. However, less than 10% of errors at Errorception are recorded using this method. It is obvious that the window.onerror approach works far better, either because try/catch isn't comprehensive, or because it is inconvenient to use.

window.onerror = function(message, fileName, lineno) { ... }

That's all the data you got from window.onerror: the error message, the URL of the script, and the line number. Errorception of course records much more information about the browser and its state for you automatically, so there's already a lot of context available.

But Errorception has sorely lacked a very vital piece of data to aid debugging: stack traces. Stack traces are trivial to extract from the error object you get in the catch block of a try/catch statement. Sure, there were a couple of tricks up our sleeve to get fake stack traces where possible, but those were severely limited.

Obviously this problem wasn't one that just Errorception faced. The web community went to browser vendors and standards bodies with their torches and pitchforks (ok, it wasn't quite as dramatic as that), to ask for some love for window.onerror. A couple of months ago, the HTML spec finally added two new attributes for window.onerror.

window.onerror = function(message, fileName, lineno, colno, error) { ... }

See that last argument there? That's what's exciting. That's the error object you would have otherwise got if you had wrapped your code in try/catch blocks. That's right: You don't need to wrap your code in try/catch blocks anymore to get rich error information. This changes everything!

Browser support

As of this writing, no production browser supports these new attributes. But don't let that dishearten you – the spec is only about 3 months old after all. IE10 does support the colno attribute, but not the error object itself. That's because IE10's release predates the spec. I expect the next release of IE to have the error object supported. (Seriously, let's cut IE some slack. They've been doing some rocking work lately. They certainly took the lead here.) Chrome already rolled this out in Chrome Canary two months ago, so it should be be in a public release soon. Discussions are on in Firefox's Bugzilla, and I expect this to be resolved soon as well. The folk over at WebKit seem interested too, though admittedly progress has been slow.

Errorception and window.onerror

Needless to say, Errorception has now rolled out support for the new attributes on window.onerror. Since Errorception already uses window.onerror to record errors, you literally don't need to change a thing (yes, even though the attributes are new). Errorception will record stack traces for your errors whenever available. In fact, I've already tested this with Chrome Canary, and it works like a charm! Yes, this works for Cross-Origin errors too!

This should finally lay to rest the argument about whether try/catch blocks are better for JavaScript error logging, or if one should use window.onerror instead. There's absolutely no advantage to using try/catch blocks for error logging anymore. It's still useful for handling exceptions, sure, but it isn't useful for logging. And if you are using Errorception, you are already using the best mechanism for error logging. Of course, you should be used to that by now. ;)

As always, if you have any questions or feedback, the comments are open.

Say Hello To CORS

Errorception now uses CORS when available to send errors from the browser to the server. This makes the error POSTing process much more lightweight.

This wasn't the case so far. Error information was encoded into form fields and posted into a hidden iframe. While this largely worked fine, it wasn't without its problems. Serialising data as form fields isn't a big deal, even if slightly wordy. However, iframes are very resource-heavy, and are fraught with performance problems. Also, iframes are essentially just regular page loads, so they fire up the browser's loading indicators in the browser tab and show that little message in the status bar of the browser. All of this happens simultaneously. Obviously, users don't like this.

However, two years ago when Errorception was launched, JSON and CORS wasn't as ubiquitous as it is today. Forms and iframes, despite their problems, were the only workable solution, especially if like me you insist on using POST for posting data. Fast-forward to today, and the world is a different place. JSON and CORS are available in every browser worth their salt.

After weeks of development and extensive testing, I've released a new version of the tracking code to make use of JSON and CORS where possible, to ensure that your users see the least amount of performance degradation at any time. If CORS (or Microsoft's XDomainRequest) isn't available, the code falls back to working as it always has - with form fields and iframes.

Upgrade if you haven't already! This new code is only released to people who are using the latest tracking snippet. The new tracking snippet was released earlier this year, and already implements tons of performance improvements over the old one. You should upgrade if you haven't already. If you signed up after May, you are already on the latest code. If you've upgraded already, you rock!

IE's i18n Strings for Error Messages

After the release last week, there have been some requests to both Raj and me to have access to the raw data used for powering the IE i18n de-duplication, from several people, including from some of my competitors.

After a little bit of email back-and-forth with Raj, and ensuring that we were both ok with making this data available publicly, I'm glad to to have just pushed all the raw data to GitHub. Go ahead, give it a look!

What? You are giving data to your competitors?

Well, firstly, it isn't my data at all. It wasn't even extracted by me. Secondly, this isn't the kind of stuff I want to compete on. Data like this should ideally belong to everyone, so that the entire community benefits from it. This data being locked into any one service is doing disservice to the community at large. In fact, when I started this conversation with Raj, my original intention was to get MicroSoft to publish these resource files publicly so that everyone could benefit from it.

Cheers!

All Your Errors, Now In English

Internet Explorer has this interesting behavior that all its errors are localized to the user's preferred language. I'm not sure if that's a good thing, since English is effectively the lingua-franca of programming, and there are so few programming languages that don't use English keywords. Ah, well. One could argue, that if the user has set their browser locale to a non-English language, the browser should display errors in the user's preferred language. After all, the error message is seen by the user.

However, with Errorception, errors are taken out from the browser and saved on a server for an entirely different audience to see. The fact that the language isn't English causes all sorts of problems. Since the languages are different, it's hard to tell at a glance if two errors are duplicates of each other. As you will have multiple error entries created because of this, it leads to other problems. For example, you can't prioritize correctly, since the number of errors are more than they should be, and number of individual occurrences of these errors are less than they should be. It's such a mess!

I'm proud to announce that this problem has now been cracked in the most comprehensive and cleanest way possible. With a lot of help from Rajasekharan Vengalil, Developer Evangelist at Microsoft, we managed to extract the language resource bundles from Internet Explorer's binaries for 35(!) different languages. (By "with a lot of help", I mean "he did all the work".) Once I had the language files, I could then create a mapping between errors in different languages.

Now, when an error comes to Errorception, the code walks across the language mapping, figures out the language of the error and its corresponding English version, finds the tokens (variable names, generally) in the error message, puts these tokens into the appropriate places in the English version of the error, and thus reconstructs the English version of the error. For completeness, the original error is also stored along with your raw logs.

So, starting some time ago, you will now receive all your errors in English, even if the errors are in browsers with a different language! This also means that the right error messages from IE are grouped together correctly, irrespective of the users' settings! And this works across 35 languages for hundreds of errors – even the latest bunch of errors thrown with the new ES5 support in IE!

Oh, and don't forget to buy Raj a beer when you meet him. :)

Better Script Delivery

I'm in the process of switching CDNs from Amazon's CloudFront to CloudFlare. While CloudFront has been generally good, it fell short in a couple of ways. So far, CloudFlare solves all of these problems and then gives the opportunity to have even more interesting possibilities.

There were two major problems with CloudFront:

• No CNAME support. Not for HTTPS, at least. As a result, I've had to give you a script with a someuglytoken.cloudfront.net URL in it. But the problem is beyond just ugliness. It gives me no control to change CDNs in the future. I've been living with this so far, but it's far from ideal. CloudFlare solves this problem elegantly.
• It's a dumb CDN. Like, really dumb. It doesn't do any content-negotiation for what I'd consider even the most basic things. For example, if you want your scripts to be gzipped, you'd have to upload pre-gizpped scripts to the CloudFront. If a browser comes along that doesn't understand gzip, it's out of luck.

The second issue might seem obvious and trivial, but has its consequences. For example, PageSpeed recommends that every asset on the page should have a Vary: Accept-Encoding header. However, on CloudFront if I provide a Vary header, I'd be lying to the network, because I'm not able to vary the content at all. As a result, all sites that had embedded Errorception's snippet would have seen a slight reduction in their PageSpeed score. Indeed, there have been requests to fix this.

Not only do the two problems above get fixed by the move to CloudFlare, it allows for very interesting possibilites in the future. Since CloudFlare is much more smart as compared to CloudFront, and since it effectively works as a caching-proxy, the possibilities are limitless! There are features I simply couldn't roll out on CloudFront which become trivial to do on CloudFlare.

Action needed

Unfortunately, this change requires you to change the script snippet that you've embedded on your site. I know it's a pain to have to do this. I apologize. The last time I asked you to do this was over a year and a half ago, so I assure you that these kinds of changes don't happen frequently. Please go to your Settings > Tracking snippet to get the new snippet.

Upgrade plan

The current script on Amazon's CloudFront will be supported for a couple of months more. Over this period, if there are any critical bugs, I'll be making releases to both the CloudFront and CloudFlare CDNs. Feature releases will not be rolled out to CloudFront, so you will not get any new features if you are using the old snippet. It's highly recommended that you upgrade as soon as possible to ensure that you have the very latest features.

Get your JS error notifications wherever you like!

Just three days ago, I had launched WebHooks support so that you can view your site's client-side JS errors anywhere you prefer. The trouble with vanilla WebHooks is that everyone has to implement the same sets of hooks for integration with the same services. For example, if two companies need the error to be posted to HipChat, they'd both have to independently develop HipChat integrations on top of Errorception's WebHooks, and then manage and maintain their implementations themselves. Such a pain! So, today, there are two announcements to help ease the pain.

Firstly, using the same underlying stack I'm using for WebHooks, I'll host integrations for most popular services myself so that you don't have do a thing. I'm calling this service hooks, blatantly copying the name from GitHub. Simply go over to your settings, and click on Service Hooks to find the list of services you can already integrate with today. You don't have to write any code to implement the hook. Just fill a form, and you are ready to go.

Currently, I'm launching Campfire, HipChat and PagerDuty integration. Integrations with these services have been asked for before, so I decided to start with these. I expect this list to expand further.

   

What an error looks like on Campfire (left) and HipChat

As interesting as that is, what's even more exciting is how it's implemented, and how it matters to you.

<3 GitHub, <3 open-source

All the code that makes these service hooks possible is all open-source. Check it out on GitHub. Even the documentation on Errorception for these services is driven off README.md files from GitHub! That's just awesome!

There are two reasons I wanted this to be open-source. One of the reasons of course is that I like the transparency with open source. When I'm asking you to enter API tokens for, say, your HipChat account, I want to make it clear to you that I'm not going to misuse it. What better way to convince you than to show you the code itself!

Secondly, and probably more importantly, if you want to see a new service integrated with Errorception, you will be able to do that yourself! Just fork the repo on GitHub, write a small little JS function to implement the integration, and send me a pull request. I'll have it merged in no time, and will immediately be available to everyone. It's really simple! And when I say it's a small little function, I mean it - Have a look at the WebHooks implementation as an example.

I've had a lot of fun planning and coding this release. I hope you make the most of it. And if you want to see your favorite services integrated with Errorception, feel free to give it a shot on GitHub. I'll only be glad to help you along.

Announcing WebHooks

Though the Errorception HTTP API is awesome for browsing your errors, it has so far been very hard to get real-time error notifications. There was the hacky solution of polling the API of course, but that's terribly inefficient, and just feels dirty. Today, this gets fixed.

You can now configure WebHooks in your settings, which Errorception will POST to whenever it encounters an error on your site. You can choose if you want to receive POSTs for every occurrence of every error, of the very first time an error occurs.

This has been made available to all projects in Errorception, just like every other feature. Head over to the WebHook docs to learn how to make the most of this feature.

As usual, feedback always welcome! I can't wait to see what you'll be pulling off with this. :)

Error Object Compatibility Table

I've been spending some time lately studying JavaScript's Error object. These are some of my notes on Error object compatibility across browsers, in case anyone else finds this useful.

Property	Google Chrome	Safari	Opera	Firefox	MSIE
name	Yes	Yes	Yes	Yes	Yes
message	Yes	Yes	Yes	Yes	Yes
stack	Yes	Yes	Yes	Yes	Yes (IE10+)
toString()	Yes	Yes	Yes	Yes	Yes
type	Yes	No	No	No	No
columnNumber	No	No	No	Yes	No
fileName	No	No	No	Yes	No
sourceURL	No	Yes	No	No	No
line	No	Yes	No	No	No
lineNumber	No	No	No	Yes	No
number	No	No	No	No	Yes

Notes:

• Though file name is available in the .stack property, only Firefox and Safari provide this as an explicit property, and that too with different property names (FF: .fileName, Safari: .sourceURL).
• All browsers string-format the stack subtly differently. There's no standard regarding stack formatting. Firefox has the least informative stack property. I'm going to cut Firefox some slack though, since they were the first to expose this property. That said, this might change in the future. Worth keeping an eye on this discussion.
• Firefox doesn't provide column numbers in the stack at all. However, it does provide a .columnNumber property which is only useful for the first stack frame.
• The .number property (IE) is practically useless. It points to IE's internal representation of errors.
• .line (Safari) and .lineNumber (Firefox) properties give the line number of the first stack frame of the error. No one else provides a similar property, though this data is available in the .stack everywhere except Firefox.
• The .toString() formatting seems consistent, and similar to the formatting of the error message in window.onerror. That is, it uses the format name + ": " + message. The only exception to this, of course, is that window.onerror formats errors differently when the source file has x-domain restrictions.
• Column numbers in the .stack property are only available in IE10+ and Chrome. Opera provides a .stacktrace property in addition to .stack that has column numbers (go figure!). No other browser provides column numbers in the stack trace. As mentioned above, Firefox does provide an explicit .columnNumber property that's only useful for the first stack frame.
• No stack support for IE<10. Nothing. Zilch.

Stack Traces and Error Objects

A frequently requested feature has been that of stack traces with errors. However, because window.onerror, doesn't give access to an error object, it has not been possible for Errorception to capture stack traces. That's set to change today.

Starting today, you will be able to pass error objects to Errorception. An example is probably the best way to explain this.

try {
    var myObject = JSON.parse(jsonString); // will break if jsonString is invalid
} catch(e) {
    _errs.push(e);
    throw e;
}

When you pass such errors manually to Errorception, Errorception will now be able to record the stack trace for this error. Undoubtedly, this can be very useful for debugging.

Important

• What you push to _errs should be a valid Error object, that is, it should be an instanceof Error.
• It is important that you throw the error right after passing it to _errs. This is for two reasons. Firstly, you really want your program's execution to stop when you encounter an error, and throw is a great way to do so. Secondly, Errorception internally uses both the Error object and the data from window.onerror to capture error data. If you don't throw the error, Errorception will ignore the error object. Update: Throwing errors isn't required anymore, but is highly recommended.

As an additional bonus, with the exception of Firefox and Safari, you will also get the column number of your error. This is especially important since your JS is likely minified without line-breaks. This column number information is especially exciting — it's likely to guide the future features of Errorception.

Bonus: This works perfectly well with the recently launched ability to record custom information with errors. For example, in Errorception I was recently doing this (yes, Errorception uses Errorception):

try {
    var myObject = JSON.parse(jsonString);
} catch(e) {
    _errs.meta = {json: jsonString};
    _errs.push(e);
    throw e;
}

Your JS Errors Are Now An API Call Away

It always annoys me when data gets caught in inaccessible silos, yet that's exactly what I had ended up building with Errorception. Today, that gets rectified.

Today I'm announcing the first cut of the Errorception API. As far as I know at least, it is the first API in the wild tailored specifically for JS errors in your site.

Being a huge fan of simplicity, the API is really simple to use too. In fact, I've embedded a curl example right here in this blog post:

$ curl -i https://api.errorception.com/
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Content-Length: 77
Connection: keep-alive

{"links":[{"rel":"projects","href":"https://api.errorception.com/projects"}]}

You can view detailed API documentation on the API docs page.

I'm excited to see what you will do with this data. I'm hoping to evolve this API based on what your experience is like. As usual, feel free to mail me at rakeshpai at errorception dot com with any suggestions or feedback.

Improving Daily Emails

The daily notification email, which had been turned off for some time now due to a snag, has now been turned back on.

Previously, all the emails used to be sent at the turn of the day on the server. This was problematic in at least two ways.

• The load on the DB to sift through the large volume of errors to generate emails was turning out to be a bit much. The server had begun to protest about the load.
• All users got emails at the same time, irrespective of which timezone they belonged to. I would personally prefer to get my error notification in the morning when I'm starting my day, so that I can schedule time for fixing it. A mail that lands up in my mailbox when I'm about to go to bed is useless at best, and worrisome at worst.

Turns out, solving the latter problem solved the former problem as well, because the server load of sending out the email would get more-or-less evenly distributed across the entire day. So, taking inspiration from what Stride did recently, I've rolled out something similar for Errorception.

A small JS snippet notes your timezone every time you use Errorception. This timezone information is passed along to the server and recorded against your records. A cron kicks in every half hour on the server to determine if you should be sent an email to right now, based on whether it is 9:30 AM in your timezone, using the excellent 'time' module by Nathan Rajlich (TooTallNate). It just works!

So, starting about 15 mins ago, you should now receive emails at 9:30 in the morning, irrespective of which timezone you are in, or whether it's daylight savings or not. If you still aren't receiving emails, get in touch at rakeshpai at errorception dot com.

Almost forgot: Wish you a great 2013! Happy debugging!