Alex Bosworth's Weblog

Working with JQueryMobile

I’ve been playing with and building small apps in JqueryMobile since it came out.

I still can’t make up my mind if I like it or not.

On one hand, it is completely against the principles of JQuery: which a simple, non-intrusive, non widget focused library. Jquerymobile on the other hand holds your hand the whole way and is basically an app framework.

However, it is very fast to develop on, and it does try to be un-intrusive.

I really want it to succeed, but there is a lot that worries me about committing to it as a development platform:

1. It was pushed out as ‘final’, but the performance is still pretty bad in a lot of situations, even on my laptop.
2. The architecture is very much designed to make static web apps easy, which is really weird since the rest of the design is very helpful for building dynamic apps, and how often do you need a static app? I guarantee that if you are building a dynamic app on jquerymobile you will either have to give up their multi-platform benefits to some degree, or spend a lot of time fighting with their automatic caching stuff.
3. The overloading of data attributes is an interesting idea, but you start to get a lot of style markup in your page content, and it can be frustrating to write so many data attributes
4. It doesn’t help you with the difficult offline stuff - it assumes you are building an always online app
5. Some features are added because they are cool, but don’t actually work very well: fixed position title bars for example.
6. It’s not designed with targeting tablets or the web in mind - it’s a false premise that anything can automatically handle all screen sizes without you worrying about that.
7. I would like it to target more modern browsers and aim where the market is going: the jquery ‘A-grade’ experience browser list is really broad

It’s so fast and easy to create apps on jquerymobile though I think it will have some good success, particularly in intranet situations - what is more reassuring is that it seems like an independent, truly open source effort, which should prevent potential lock-in problems in the future.

Fear and Civilization

“The only thing we have to fear is fear itself.” - FDR summed it up nicely. 

Society has a flaw: paranoia. At the core of human psychology is a processing error, a bug: our ability to logically assess risk is out of whack. 

We are living in a golden age: from the Renaissance to modern times, the embrace of reason has carried us on a rapid wave of advances that have extended our lifespans beyond our twenties, has put a universe of information in our pockets, has enabled instant communication between anyone, anywhere in the world.

In computers, there has been a golden age: microchips. Every generation, like clockwork - smaller and smaller, faster and faster, cheaper and cheaper. But this golden age has been slowly winding down, because it is hitting fundamental limitations: the speed of light, available power, quantum uncertainties at microscopic scales all mean that the focus is now on low power chips and multiple cores, the well of infinite computing plenty has started to run dry.

The golden age of civilization is powered by human reason and civilizations across the world that support reason. At a level above capitalism vs communism, democracy vs dictatorship, there is realism. But in modern societies, progress has started to hit a limit to growth: our world has grown so complex, it is hard for regular people to understand what is going on, uncertainty is fear, so regular people are becoming afraid of dangers real and imagined. 

The problem is that psychologically speaking, people are likely to put equal weight on clear and present dangers that they would on imagined dangers or unlikely dangers. Very many people are killed by hamburgers, the flu and car crashes. Very few people are killed by sharks, airplanes and terrorists. Movies and politicians and newspapers and schools play into these fears, because that is what people want to hear about, because of a psychological failure of reason around the unknown. 

If reason is the driving force behind the advance of civilization, then a limit to reason is also a limit to that advance. 

This limit is slowly becoming more apparent as we work to advance within the framework of our societies’ ability to promote reason:

1. SOPA - stop online piracy: there are pirates out there, they threaten established business models. What will happen when these business models change? Regular people and congresspeople don’t understand these issues: an irrational response that seems reasonable is to shut down progress to protect the past.
2. “Shut down your electronic devices” - This is ludicrous even with back of the envelope math. Assuming a tiny fraction of people ignore this, we have a testbed of millions and millions of flights a year that take off and land without issue with an electronic device online. The FAA just approved the iPad for use during take off and landing - but only in the cockpit of the aircraft!
3. “Take off your shoes” - the irony and cynicism of airport security could take ages to lay out. The most horrible example is that the 911 hijackers had no special devices and were actually stopped by airport security, who saw their box cutter and let them through anyway. Free and easy travel to anywhere in the world is progress that is being stopped by fear of an uncertain world.
4. HIPAA - For the thousands of great web apps and smartphone apps and desktop apps that have come out, why is there still no great medical apps? The answer is: fear stopping innovation in this area. I can startup a new online business like Mint and take your bank account passwords directly from you and have the power to take your money, but if I want to store the fact that you have congenitally high blood pressure, I need to follow expensive protocols and pay for an independent auditor to make sure I am following some arbitrary and largely useless security rules.
5. The stop and rollback of new drugs. More and more money has been rolling into the medical field, we are living longer than ever before, and spending huge amounts of money on healthcare. But where is this money going? Developing new drugs is extremely risky because it involves vast, vast sums of money to test them. People are afraid of drugs, so even a drug that saves many people but kills some can be banished from the marketplace, and worse the maker can be sued for all the profits made saving the majority of people for the minority that died, even if they warned of the risk and doctors studied and still recommended the drug.
6. Banking - Even after many many years of telecommunication and the internet, we are still paying huge percentages of every online and credit card transaction to middle-men who essentially skim off the top of every transaction, don’t let small transactions go through at all, and charge usurious rates to the public for small loans. Attempts to innovate like facecash and bitcoin, etc face huge obstacles or get shut out before even getting started for not working within the established system, a new system of free and instant transactions at any level is a scary concept that society is quick to reject.
7. Driverless cars - Google has been proving the concept that computers can drive cars better than people can. In addition to sensing and preventing accidents thousands of times faster than even the most alert person could, computer driven car systems could make highway systems many times more efficient than the current ‘waves of traffic’ problem caused by irrational human drivers. Efforts are in place or underway to ban driverless cars from the road - both by scared people and by car companies who like their established business models.
8. COPPA: children and the internet - The internet: what was supposed to be the best resource for children to learn, is shackled by outsized fear of its dangers. Google and many others simply block children from using their services, because it’s not cost or time effective to comply with societies’ demands that they perform arbitrary and useless measures to ‘protect’ children.

Taking a look at societies’ deficiencies is the job of entrepreneurs and even governments and large businesses: every deficiency represents an opportunity to improve and advance society. But many paths are now or have long been blocked by societal fears, and as society gets more advanced, fears and blocks multiply, and advances in many ‘scary’ areas slows and slows.

The good news is that if people in the US take leadership in fighting irrational fear, they will be rewarded for it, and the rest of the world can be rewarded too. Seeing a scary solution actually work out OK in practice is a great way to fight fear of the unknown and tear down barriers to progress. But some people and some societies always have to be the first to brave their way forward to break the static status quo and help the others get over their fears of the unknown.

Should I use Heroku

To Heroku or not to Heroku, that is the question.

(I should preface this with the info that the CEO / now General Manager of Heroku was my previous boss)

I personally love building web services, but hate configuring Linux boxes, and database and web servers.

That is why when I build new services, I gravitate towards outsourcing the configuration and database/web server stuff, or using a solution where I can control and understand all aspects of how it works because it is either simple or familiar.

Examples of things I like because I can understand how they work and I control them

• Memcache
• Node.js
• JQuery
• Nginx

Examples of things that work by complicated or magic rules and I try to avoid

• Ubuntu, CentOS, etc
• MySQL, Postgres
• Any ORM solution
• Apache

Examples of things I outsource because I don’t want to worry about them

• Amazon EC2 / S3 / SimpleDb
• Tumblr
• GMail
• Github

So where does Heroku fit in for me? I am actually very wary about outsourcing, because there is a good chance that an outsource partner will change the deal on you at some point unexpectedly, not behave as you expect or hope, or just disappear.

I have in fact always been burned by outsourcing:

• Amazon changed their SimpleDB API and said I must change or my App will break. If I control a solution only I get to decide when I must change.
• Tumblr has let me down by not really improving anything that I use on their site. 
• GMail has had progressively worse usability with every update, and they failed to make a decent iOS App because they are putting their Android strategy ahead of their users’ needs.
• Github’s pricing plans don’t match my business needs, they are focused on other kinds of businesses

But all of these negatives don’t mean I want to do everything myself, the world is not a black and white place. Not outsourcing has costs too.

So where does Heroku fit?

In the beginning, I thought the idea of Heroku was wrong: Amazon is already doing for me what Heroku is selling.

But Node.js changed everything for me. Node.js is a framework that is very bare-bones. You have to do a lot yourself. When I switched to Node.js over a year ago, I started work on a scalability system for Node.js where I could reduce process failure vulnerability and increase my ability to easily scale beyond a single machine.

I did this all on my own server, an EC2 instance that cost money every month to run, although thanks to Node.js’s great performance characteristics, was not stressed at all.

But eventually I saw a series of factors that made Heroku the right choice for me:

1. Node.js support (one of the few vendors, even Joyent who invented node.js wouldn’t let me pay them to host my apps on their platform)
2. Salesforce.com owns Heroku which means it’s not going to go bust anytime soon
3. There is incredible community support for Heroku
4. Heroku is free to start, and for me even with tens of thousands of users it’s still free
5. Heroku’s dyno system is almost exactly like the system I built for myself, only more reliable and better
6. Heroku is git based and has a very simple command line interface
7. Heroku is hosted on Amazon EC2, which I already was using and trust
8. Heroku’s pricing model is incredibly friendly to Node.js, almost troublingly so. The model of pricing is based on the idea that web servers ‘block’ on requests - so to get more simultaneous requests you will have to pay more money. But node.js doesn’t block on requests, so it’s like getting thousands of servers for free.
9. Heroku’s customer support is very friendly and good, even though I’m not a big customer
10. Heroku has a nice logging command
11. It takes only a single command to scale up and down

OK, so I made the full switch in August of this year. I said before: every outsourcing solution will burn you. How has Heroku burned me?

1. Heroku makes an implicit and sometimes explicit promise that you can just write code and stop worrying about the server characteristics. This is false, for example if you use too much memory in a single process Heroku will start presenting errors to users, and this threshold of memory is easily reachable in practice.
2. Heroku Apps are blocked in China - I live in China, I want a service that isn’t blocked here. China is projected to become the biggest economy in the world within 5 years, it’s an important market.
3. Heroku’s failure rates have an impact to users: about 0.05% of my requests this month failed
4. Heroku’s add-on system is weird and scary
5. Heroku hasn’t updated their version of node.js in over 6 months and is incommunicative about when they plan to update. 
6. Heroku doesn’t support HTML5 websockets

Given all of these drawbacks, should I still be using Heroku?

Yes, I think Heroku is great. The costs are definitely outweighed by the benefits.

And one of the best features of Heroku is that I could switch away quickly if I ever change my mind.

Do It (Tomorrow) Web in HTML5

We’ve now had 1 million iPhones, iPads, iPod Touches and Androids install our App Do It (Tomorrow) - So we figured we should also bring the App to the Web as well :D

If you are not interested in the dirty details of development, depart. If you are, great: read on. 

Prototype

I often like to develop in a quick style where I put together a prototype and then look and see what I learned. Sometimes I can go directly into a shipping App, sometimes tweaks are required, sometimes I throw the idea out completely.

The prototype revealed a couple major design issues with porting an iOS App to the Web:

• In an iOS App, if you start using it you expect it to save all your data. In a web app, you expect it to lose all your data if you don’t register/login first.
• To get the App to a level of polish that matches the iOS App, I was going to need very modern CSS features, which meant that older browsers were not going to be supported.
• Just like we did a redesign for iPad, yet another redesign was necessary for Desktop - a flexible resolution design for computer monitors is a lot different than a fixed resolution design for iPad or iOS.

Redesign

Login and Register

If you have the smartphone version of DIT (I hope you do!) You notice that you don’t need to register. You can just type in an email and password and it will use that as syncing credentials to make sure we know whose todos are whose.

This actually turned out to be too clever by half, and something that we did as a reaction to Apple’s limited Settings feature, but the net result means that we don’t have a register feature.

This makes for a very weird Web experience, so instead I just added what is essentially a “placebo button” to register. The register button doesn’t really do anything, it just opens the book and sets up your credentials, but it is functionally almost equivalent to login, which also sets up your credentials. 

This also gives you a chance to read a blurb about the App. On the App store you can read a description of the App, but people like to just jump on in when it comes to Web sites. We debated this blurb a lot, but the basic message of Do It (Tomorrow) is represented: “blessed are the procrastinators.” :D

3d rotation transforms

One thing I began experimenting on as a wow factor is something called 3D transforms. Do It (Tomorrow) has always had an ‘opening’ effect, and we put a lot of work into making that look right. The prototype used a 2D CSS transform, but I did research into doing a cover opening in 3D, based on demos I found of this Webkit feature.

Moving divs in 3d space took some trial and error, the way it works is interesting: you set up a “stage” in which you view your transforms. You can set the stage close, so the 3d effect jumps out at you like a cheesy 3d movie, or you can set the stage far away, so the 3d effect looks like it is in the distance.

Once you have established your stage, you set axis of rotation on your div, and you can rotate it around that axis in the 3 dimensions. 

Now the tricky part is animating :D This can also be a trial and error process. To animate the cover, the cover is 3d rotated until it hits an apex, at which point some magic happens and now it’s not the cover rotating, it’s the left page! Important here is to match up all the timings correctly, and to make sure that the functions describing the animation speed are also well matched to each other.

Server architecture

The backend architecture of Do It (Tomorrow) syncing was designed for iOS, with one major goal in mind: be as simple as possible.

The tasks are stored on S3, which means that Amazon can worry about keeping them safe for us. 

To process the tasks, and to serve the web page, I used a small Node.js script that doesn’t do all that much except deal with the mundane work of storage. This script sits behind a nginx reverse proxy.

Most of the logic about tasks actually lives in the client JavaScript - when to move tasks, how to merge sync data and resolve conflicts, etc. 

I am using a S3 library I am writing as I learn Node.js - it is jQuery inspired but my reasons for developing an AWS S3 and SimpleDb library are better for another post.

CSS 3

CSS 3 and HTML 5 features are not just used in the opening transform animation. These features are critical to the polish of the App.

In fact, the CSS file has more lines than all the other project files put together, tomorrow.do is quite a complex CSS beast.

• Media Queries: we felt the App looked weird on very large resolutions. Media Queries add polish details as the browser is resized. (All employees of Adylitica have 27” Apple monitors - btw we are hiring! )
• Multiple Backgrounds: to do a realistic setting and not having messy non semantic elements littering the page, 7 background images are stacked on each other to achieve a layered look.
• Custom Font support: We use a special font for Do It (Tomorrow) - that’s part of the App’s identity. Mozilla makes it a little annoying due to their font-origin policy but it was very important to get in.
• CSS gradients: Obviously these are already having a huge effect on web design. The very good news is that Webkit and Mozilla are coming to agreement on syntax and this looks to be a standard CSS tool very soon.
• ::-webkit-scrollbar - this is something that adds tons of polish and unfortunately Mozilla is religiously against. 

Domain Name

Apps can use any old name you like, for the most part. Web Domains on the other hand are like precious jewels, hoarded by evil dragons that will kill you with a breath. 

To find http://tomorrow.do/ I used an App called http://domai.nr/ - I highly recommend it. I like unconventional TLDs because they look cool to me, but they are usually more expensive and more of a hassle.

To get tomorrow.do took well over a month, after paying $140 for the privilege. The Dominican Republic can move quite slowly on domains as it turns out. 

We had many ideas, but I chose tomorrow.do because I feel like it is easy to remember and notable.

Deployment!

I hope that people enjoy it, I enjoyed making it. One of the strengths of developing a web app is that it is way less stressful than iOS or even Android. I can for the most part control the audience, by only showing it to friends and family, then wider and wider. I can receive also feedback - code a fix or a tweak - and roll it out to the server in the space of 5 minutes - easy!

Buy Bitcoin!

Here at Adylitica we have been paying close attention to the new currency you probably have heard of by now: Bitcoin. 

In a sentence: Bitcoin is a new currency of virtual coins that are limited in number and can easily be traded in a decentralized internet marketplace.

Obviously the concept of Bitcoin is to be used as a medium for commerce, but to a person everyone I have talked to about Bitcoin has been extremely skeptical. 

When I started looking into Bitcoin seriously in January/February of this year, Bitcoin’s exchange to the dollar was headed from about $.60 to $1, already high volatility. But now 4 or 5 months later it is up to $7 to $8.

Half seriously in January we attempted to buy some Bitcoin but it was too much trouble and they were sold out. Looking now, was not buying Bitcoin a huge mistake? Theoretically, in opportunity cost on paper we have already lost a lot of money by not buying when we first heard about it.

When I analyzed Bitcoin in February I came to the following conclusions:

1. Bitcoin is based on a piece of software, the marketplace and transaction hub for Bitcoins. Even though this software is open source and decentralized, it may have bugs or vulnerabilities that have yet to be considered fully or even found.
2. Fixed supply commodities have volatile prices. Part of the reason for governments switching away from a commodity backed currency is that these standards are subject to random whims of the market. (Silver up 100% in 1 year)
3. Trading with Bitcoin will have a lot of resistance until it becomes mainstream, a chicken and egg problem. Trying to actually buy and using Bitcoins, it’s definitely doable, but akin to setting up desktop Linux, it’s just enough of a hassle for a small benefit that it’s not a good sell for ‘normal people’.
4. Bitcoin has absolutely no intrinsic value and no backing whatsoever. Gold and diamonds are shiny, and have brand value at the very least.

This is not to say that it wouldn’t be great if Bitcoin were to take off. Currency and monetary trading today is an entrenched system that is hostile to innovation. Bitcoin’s wide adoption could trigger a www or app store style innovation boom in the money space. A lot of fields could benefit: Micro-loans in poor countries, tips to bloggers, in app purchases, easy funding and purchasing for small businesses and indie developers. 

So buy Bitcoin now? Not to advocate too strongly for it, but definitely buy Bitcoin if you want to help the movement, or at least talk about Bitcoin: awareness of Bitcoin as a currency will actually increase its chances of becoming a currency. This is a big reason the value of a coin has gone up over 10x in the past year: people are talking about it.

Buy it as an investment? No. As a gamble? Yes. Bitcoin is a zero sum game, it’s not like a company’s stock where money can be created from nothing given innovation and hard work. It’s a commodities market, not a stock market. 

The relevant questions:

1. Are people undervaluing Bitcoins?
2. Are people likely to overvalue Bitcoins in the future? 

Not to prognosticate, but Bitcoin by itself is a very intriguing idea and piece of technology that probably deserves mainstream attention. In the future it is likely to be over-valued on that basis alone.

I’m not going to buy myself as I don’t have so much money I feel like I can gamble it, but I would say yes it’s a good gamble if you do: it will probably rise in price when it gets mainstream attention, at which point you can sell for a profit.

Use Bitcoin for actual commerce? Not yet, unless you are prepared to change your prices frequently and only keep limited amounts of Bitcoin on hand, or you don’t care about the risks. Bitcoin is too volatile, there are too few players, it still is a hassle to integrate, and it is risky at this point in its life-cycle. 

Services + HTML5/iOS, a new model for App development

A lot of things are changing with the advent of HTML5 + the rise of iOS/Android.

We are going to have to throw out a lot of how we make service oriented Apps. 

This stuff isn’t going to be done overnight, but the kind of web frameworks that exist now and new web frameworks and languages going forward are going to need to adapt to a new dynamic of smart clients like HTML5 browsers and iOS/Android Apps.

The next steps for service architectures remove HTML/CSS/JS from the equation. Apps will consist of: data store access, business logic, and static file + JSON service endpoints, with separate client code driving interaction and interfacing with the service endpoints. Notice what’s missing? No more “page” classes. Client agnosticism is coming, and it will make service development a bit more abstract but a lot more fun to do in the long term. 

No More HTML Munging

The curse of web development is working with an ugly mess of strings that is HTML. Why do we put up with this? No one really wants to work with putting HTML together, it’s long been just a necessary evil. Template frameworks help but also add complexity and present annoying walls to rapid development.

So celebrate, this is going away, it will be the client’s problem to deal with the interface! Yes HTML will still exist, but JQuery is now a great solution for putting pages together from scratch, you get more dynamic interfaces and importantly interfaces that are clearly separate from server code.

Before, a big reason to do server-side HTML munging was SEO. (Thanks Chris for rescuing that article from my old blog). Today, Facebook and Twitter both use what will likely grow more and more common: Google’s #! syntax. Many apps don’t even need SEO, and Google themselves have moved to JS driven clients.

Now that we have HTML5, killer fast JS engines and and rich plugin resources like JQuery’s, putting all web pages together on the client transforms from a Quixotic time sink into the smart design decision.

Finally: HTML browsers are now just another client for backend services, iOS/Android apps are a major way to consume services - the most cost effective and maintainable solution will move to a common API layer for all clients.

Eliminate Cookies

Developing based on cookies is not a good model for server development. Cookies are a terrible model for maintaining state between the server and client. They add latency, they are extremely limited and a hassle to work with, and are the source of a litany of exploits.

Cookies are going to be less and less relevant. localStorage APIs obliterate the space limitations of cookies. Request header manipulation + cross domain request standards provide security + flexibility as well as natural speed improvements by eliminating the cookies model of sending state variables back and forth with every single request.

I’m through with web frameworks. What’s the point of a web framework when I do my HTML generation on the client, or iOS/Android apps use my service?

That’s a huge reason I’m through with PHP. PHP was a language built from the ground up for web development. It was a great tool for HTML munging, for rapid development, and for scaling to large numbers of page requests. PHP is not fundamentally a good language however, and now that the HTML munging and page request models are fading, it’s time for something else.

Node.js is right now at a great position for leading a new model of service oriented development. There’s no framework yet to make this stuff easy or idiot proof, but the fundamentals of Node are all right on target to drive the next generation of web and smartphone apps.

I’m only about 50% of the way to converting all my code to Node.js, there’s no question to me that it’s still immature but there is also no question that it has huge strengths for developing next-gen services and it’s well designed + fun and deserves the growing momentum it’s getting.