WordPress.com’s Job System – Cron for PHP in Distributed Environment

Colleague Demitrious Kelly (meech, Apokalyptik) earlier this month open sourced the (Unix process) jobs system he (primarily) has been developing for WordPress.com. Not that I really understand it, but “jobs” is described as

A fast, distributed, horizontally scalable system built upon linux, php5.2, and mysql 5.1 wherein work can be stored in a database, and processed outside the flow of script execution. Examples of common things that are part of script execution but not necessary to the rendering of the response to the user might be spam scanning, statistical analysis, email notification sending, processing input data, etc. Also included is an equally distributed cron mechanism to remove single servers as a point of failure for scheduled jobs.

There are a lot of terms I like in there like fast, distributed, horizontally scalable, scheduled. It could probably benefit from robust and fault tolerant. Any others?

Anyway, this crontab-like system for PHP scripts is an essential part of our WordPress.com infrastructure, and I’m really excited to see it open sourced!

Check out code.trac.wordpress.org for other pieces of our puzzle that don’t have pretty project pages.

WordPress.com Still Blocked in China

I’ve seen some reports lately that “WordPress is blocked in China”, including some Mashable articles. To clarify, it’s WordPress.com that is blocked, not all sites that run self-hosted WordPress.

Mashable’s “China Blocks Twitter (And Almost Everything Else)” seems to suggest that the block relates to the 20th anniversary of Tiananmen massacre on June 4th. That may be the case for the other major web publishing and social networking sites, but WordPress.com has spent much of the time since the beginning of 2006 blocked by the Chinese government.

It’s an incredibly frustrating, sad situation.

In “Blogging guru chips away at Great Firewall of China” my boss Matt Mullenweg shares:

After some discussions, Mullenweg realised the site would be allowed back online if he agreed to block certain words or topics and give up information to the Chinese communist government about users.

“I started thinking about the DNA of the company,” he said. “That sort of company is not one I would wake up every day and feel passionate about working in.”

The site remains blocked, although Mullenweg, unable to suppress a giggle, points out that the official Chinese line insists it is freely available.

There was a break in the blocking around the time of the Beijing 2008 Olympic Games, but that ended with the games. That was the longest time I know of when people could consistently access WordPress.com from across China.

Part of the Chinese government’s approach seems to be to sometimes allow western hotels access and other specific points — likely to deceive Westerners and cause confusion.

There does not seem to be a reliable way to automate detection of WordPress.com being blocked, regardless of what tools might promise. I’m not surprised that http://greatfirewallofchina.org/ gave up. This is the message on the site’s homepage:

“Because of the ever stricter measures of censorship China imposes on the Internet, the team of www.greatfirewallofchina.org at present can no longer vouch for the reliability of its test tool. We have therefore decided to take the test tool offline.”

Herdict <http://www.herdict.org/>, a project of the Berkman Center for Internet & Society at Harvard University, seems to be a novel, survey based approach to find out which locations are blocking or censoring web sites.

The tool of choice to let people dodge surveillance is Tor <http://www.torproject.org/>. Please consider your own safety and technology expertise before using any such tools. Reporters without borders <http://www.rsf.org/-Anglais-.html> seems like an excellent resource if you are considering reporting from a dangerous place.

Monday, June 8th Update

A reliable source in China shares that Twitter is back. The same source also shares:

Specific blockages do appear to coincide with major events, like those in Tibet last year and the recent anniversary, but are also part of general and ever-changing blanket censorship.

The Twitter blockage is really the only new story of the last week (apart from 5 second delay foreign television news blackouts) but the news media wanted to turn it into a larger story, including the WordPress angle.

WordPress has been blocked for years now. Youtube has been blocked for the last 3 months. Twitter went done last week.

While the dates for the last two occurred at the same time as sensitive periods or events, I think both are more about controlling new forms of communication and networking than the particular events. Censorship in China is generally a one-way street with periods of activity ramping up around specific times. I think of it in the same way tax auditors get busy around tax season and government ministries around budget time. The difference being once a site is blocked by thePublic Security Bureau (PSB) here in China, it is very difficult to get it unblocked; given the public security connection in an already opaque closed-door administrative system. A good analogy is trying to get someone off a no-fly list in a bureaucratic security system that lacks transparency.

Domestic blogging sites are heavily controlled and censored (though in an ad hoc way). Chinese video sharing sites are coming under increasing regulation and many believe it was only a matter of time before Twitter, as the newest uncontrolled networking site, was shut down.

We Raised $29.5 Million!

Matt, now Ma.tt, and Toni have respectively written “Act Two” and “Automattic fundraising” about us, Automattic, raising a $29.5 million USD round of financing today. Our friend Om has a head start on journalistic coverage with “WordPress.com Creator Raises $29.5M“.

By “we” in the title, I mean I have absolutely nothing to do with the financial matters, so don’t ask.

This development is, of course, a little distracting at the moment, but that will quickly fade.

The fundamentals haven’t changed. We have good, popularly, highly regarded solutions to important, interesting problems. There are many challenges we want to continue to refine our solutions to and many areas we have just started to explore — so many things are too hard, and we know we can help. Most importantly we will stay focused on relationships between real people and being dynamic to your needs — which is really what the web and particularly blogging are all about.

I’m as excited to working for Automattic today as the day I started. Has it really only been just over a year? My job description, like all job descriptions, has changed a lot in this short, and I’m sure it will continue to evolve.

I now spend about half my time working with Raanan, Barry, and Michael and the “major media organization, from the NY Times, WSJ, CNN, Fox, Time, People, and more…” the emerging ones like Giga Omni, Gawker Media’s Gizmodo Live, Blog Talk Radio, Curt Schilling’s 38 Studios and I Can Has Cheezburger.

The rest of my time is focused on participating in WordPress.org.

I love the diversity of challenges my roles expose me to. I love that WordPress is a near universal solution for online publishing, empowering the personal publishers and the major media organizations.

But most of all I love the people. Bloggers and open source participants are the among the most wonderful people I have ever met!

But along the way you still have to feel with some stinker technical challenges and personal conflicts, and that is where my favorite Automattic part comes into play. Every member of the Automattic team I learn from every day and every one of them I would love to call friend.

WordPress 2.3 Heroes

September 24, 2007 we released WordPress 2.3, and a little over a month later, this past Friday, we released WordPress 2.3.1.

Yesterday , we, WordPress won Best Open Source Social Networking CMS.

Who are we?

Continue reading

WordPress.com Happiness Engineer For Hire

Matt linked to our Happiness Engineer job posting:

Our software and services are far from perfect, and when things go wrong people aren’t shy about contacting us asking for help. We consider the support side of the user experience to be vitally important because it’s the person who interacts with our customers most and makes the biggest impression in their time of need. In fact everyone who joins Automattic, regardless of position, does support for 3 weeks. The customers range from the everyday blogger to VIPs like CNN, Flickr, and People Magazine. The job requires:

  • Patience and grace.
  • Excellent writing skills.
  • Working knowledge of WordPress, HTML, and CSS.

Continue reading