Oct 29, 2009

Blogger: Most Popular Posts Widget (Feedburner)

There are several widgets for wordpress to show most popular posts, however, as far as I know, there is no such widget yet for blogger. In this post I explain how to create such a widget for your blogger blog. A prerequisite is rudimentary knowledge of python. The installation is a little bit more intricate than other widgets on this site. I think anyone, with some patience should be able to do it, however if your computer skills are very limited don't try it.

This widget uses feedburner statistics to list most viewed items. If you want to have data based on google analytics please see my popular post widget with analytics.

You'll need a feedburner account. In your feedburner account you need to activate the awareness api (publicize->awareness api->activate). This allows external access to your feed traffic data.
activate awareness api in feedburner

If you don't have one already, you need to create a free account on google app engine. Hosting your files and web applications on google app engine brings you the advantages of a low latency content delivery network (CDN) which can make your site load faster and for this implementation of the most popular posts widget you need an account. You'll have to choose a domain such as yoursubdomain.appspot.com.

If you don't have it already install python. Download the app engine python SDK for python.

Download the source code of the widget. You need subversion for this. The command for checkout is this:
svn checkout http://popular-posts-for-blogger.googlecode.com/svn/trunk/ popular-posts-for-blogger-read-only

Go to the popular-posts-for-blogger/popular-posts-for-blogger subdirectory and edit the getfeedstats.py file (use an editor or IDE that knows python, such as eclipse, vim, or kate). Search and replace myoutsourcedbrain for your feedburner uri.

Upload your code with the google app engine SDK. Still in the same directory type:
appcfg update .

If you did everything correctly you should now be able to see the javascript at http://yoursubdomain.appspot.com/reports/feed.

In your website or blog, where you want the widget to show, paste this code:
<h2>Most Viewed</h2>
<ul><script src='http://yoursubdomain.appspot.com/reports/feed' type='text/javascript'/><noscript>most popular articles by feed views</noscript></ul>


Enjoy. Please leave a comment below for questions and suggestions. If you have ideas for any improvements please tell me.

Oct 28, 2009

Embed and Highlight Source Code in HTML

In articles about programming it is very important to embed source code so that people can see the code and even download it directly. Sometimes I include code snippets in my posts and the code reads much nicer if the syntax is highlighted. There are different ways to embed source code from different programming languages in HTML and in this post I comment on them.

The different methods for embedding source code in internet pages can be distinguished into three classes:
  1. Direct conversion into HTML
  2. Javascript library
  3. External hosting service

Conversion of source code to HTML

This method requires you to format your source code to HTML before posting it. It has the advantage that you don't have to rely on third-party code and you don't need to load external scripts.

I found vim2html (website inactive now) quite useful for this purpose. As the name says, Vim2Html takes advantage of the vim editor, which does syntax highlighting for many different programming languages. Vim2html extracts the highlighting and converts it to html, which I then only have to insert here. Gvim has the option of converting to HTML already in the menu.

Escaping Special Characters

If you want to embed code into HTML (for example in your template) you have to convert HTML special characters such as &, <, >. If you want to embed code into posts you only must convert characters < and > (but are free to additionally convert &).

Try it out how conversion works in the form below. Paste or write your code and see how it converts.


You might want to use the HTML escaper to have more options. The above converter is based on code by bloggerfunda.

To let text standout, you can always use common HTML style attributes such as <b></b> or <i></i>, or color attributes.

Javascript library

This method has the advantage that you can directly insert your source code without any pre-formatting. It is cheap for your web server, because formatting is client-side. You include a javascript library in your pages which converts the source code to HTML and highlights it. Alex Gorbatchev’s syntaxhighlighter Google Code project gives that functionality. It supports the languages C++, C#, CSS, Delphi, Pascal, Java, Java Script, PHP, Python, Ruby, Sql, Visual Basic, XML/HTML, among others.

External Hosting Service

This method has the advantage that it is lightweight for both the web server and the client browser. You have the disadvantage of being dependent on a third-party service. You copy your source code to the service and include an embed script to include it on your web site. Often this embedding allows conversion to plain text and direct download of the code.

Examples for such services include pastie.org, snipplr and DZone Snippets.

Oct 22, 2009

Blogger: Welcome Widget (Re-Search)

On some internet sites, you may be greeted with a message like this:
You searched google for "Blogger Welcome Widget." On our site there are 500 articles corresponding to that query.
I wanted something like this for this site. In this article I present a simple bloger widget that presents a short greeting message containing query and number of search results. Clicking on the message triggers a blogger search in your blog for the query.

I extended this widget to include welcome messages to visitors coming from Entrecard and Twitter. Please see my Greeting Widget for Entrecard and Twitter.

It is fairly easy to extract information about referrer and queries using javascript. I found a search query parser at transarte.net and only changed it slightly so that only a message appears if the search comes from outside (ignoring, say, searches from the blogger navbar or a google widget).
 
Using google's ajax.api search queries can be executed and parsed. I used the simplest form so no new javascript libraries have to be loaded.
 
The query plus number of search results are a hyperlink so that clicking on the message triggers a google search in your blog for the query.
 
If you want the fast installation, just press the widget install button to get it. Make sure you edit the information about your website (address and name of your website). Put the widget where you want it.
 

 
There is one line in the code which checks whether the referral is your own site. You will have to change that line. Just search for myoutsourcedbrain and replace with some adequate string contained in your page URL.
 
It should be fairly easy to customize the message and its look. Enjoy!

Oct 21, 2009

Translation Widget

There are different blogger widgets for translation. They differ in look and functionality. In this post I compare some widgets and publish my own blogger widget code for translation.

The standard google translation widget for blogger and other websites looks like this:
google translation widget

A bit bulky in my opinion.

At google translation tools you can build a pull-down menu like this one:

Select the language of translation:

Another possibility is to have a menu with flags such as the google site translator at widgetbox.com.
translation widget with flag selection

While images may be nice to see, the use of country flags is a faux-pas. There are many countries that share languages and if you prefer to show one country flag over the other, people might become offended. While for USA and England, the problem may not seem so grave, it is potentially conflictious to decide between Taiwan and China. The flag menu choice also neglects minority languages such as French in Canada.

All of these menus, I feel are a little uncomfortable. It should be faster to get to a translation and I don't want people to have to search through many options.

Why not just recognize the language of the browser or user and offer a ready button? I build a translation widget that does just this.

It uses the google translate library and a javascript to recognize the browser language. Google translates the translate button into the browser language, so that the visitor of the site sees a button, most probably in his or her own language. This button leads to the google site translate service with the language pairs English and recognized browser language.

Click the button below to add the widget and place it where you want it to be.




Now, go to your blogger dashboard, design, edit html, and search for </head/>. Just before that copy-paste the following code:
<script src='http://www.google.com/jsapi'/>
<script type='text/javascript'>
// verify browser language
function getLang() {
  var lct=&quot;en&quot;;
  if (navigator.language) {
    lct=navigator.language.toLowerCase().substring(0, 2);
  } else if (navigator.userLanguage) {
    lct=navigator.userLanguage.toLowerCase().substring(0, 2);
  } else if (window.navigator.language) {
    lct=window.navigator.language.toLowerCase().substring(0, 2);
  } else if (navigator.userAgent.indexOf(&quot;[&quot;)!=-1) {
    var debut=navigator.userAgent.indexOf(&quot;;[&quot;);
    var fin=navigator.userAgent.indexOf(&quot;]&quot;);
    lct=navigator.userAgent.substring(debut+1, fin).toLowerCase();
  }
  return lct;
} // end getLang()

    function initialize() {
      var lang=getLang();
      if(lang!=&quot;en&quot;){
        google.language.translate(&quot;Translate!&quot;, &quot;en&quot;, lang, function(result) {
          if (!result.error) {
            var container = document.getElementById(&quot;translation&quot;);
            container.innerHTML = result.translation;
          }
        });
      }
    }
google.load(&quot;language&quot;,&quot;1&quot;,{&quot;callback&quot;:initialize,&quot;nocss&quot;:true});
</script>


This will put a translation link on your page (where you placed the widget). The translation link will be in the browser language and it will not appear if the user language is English. If you want to have translation from any other language (not English) you will have to change en to the ISO correspondent of your language.

The browser language recognition script is based on what I found at stackoverflow and java2s. I tested it for firefox, safari, konqueror, internet explorer, and opera. Please report if you find that it doesn't work for any browser (any browser that supports javascript, that is).

That's it. Enjoy.

Oct 12, 2009

Blogger: Related Posts Widget

There are different methods to list related posts in blogger. I have tried several of them and after reviewing, I think Yu-Jie Lin's blogger related posts is the best for several reasons. It is very fast and updates from feed once a day. Installation is very fast and can be done by installing the widget from Yu-Jie's blogger related post service and adding a code to your blogger template. However, you might want to consider hosting on your own, in order to have your own dedicated find related posts service. This can be setup for free using google low latency content delivery network (CDN), with some basic knowledge of python in about 10 minutes. In this article I explain how.

Introduction

A related post widget is very useful in order to reduce bounce rate and keep people on your blog exploring one or several topics. There is no related posts widget from blogger, however many people have developed their own related posts widget.

Some of these related posts widgets are very good, however they suffer from one or several of the following limitations:
  • very intricate installation procedure
  • high server latency - because your visitors need to contact some servers somewhere far away from them
  • syncronization - update is not frequent enough, so that after a week you still see deleted posts listed under related posts, while other new ones are not listed
  • speed - some take big or convoluted javascript that takes too long to download and execute
  • duplication - Within the listed 3 posts, two are identical.
  • relevance - You don't know how they calculate which posts are related (lack of transparency). Sometimes the choice can be quite arbitrary.

Y-Jie's related posts widget has none of these problems. Aside from being very responsive, fast, and low latency, it has following characteristics:
  • Relevance is based on labels of queried post
  • Allows customization of widget title
  • Fast and easy setup: 3 Lines only for manual installation
  • High update frequency: data cache for 24 hours
  • Lists up to 10 related posts

Problem however is that already many people use Yu-Jie's service at his setup, he is going to run into trouble. There are quota for appengine and he is already at the limit of the quota.

In the next section I explain how you do your own setup for free and in less than 30 minutes. If I can't convince you how easy it is, you can still use Yu-Jie's setup at http://brps.appspot.com or possibly mine at http://findrelatedposts.appspot.com. If you like it, please, as a matter of courtesy, put a link to Yu-Jie's page. This way other people also get to know about this related post widget.

How to Setup Related Posts on Google App Engine

Hosting on your own, can be useful in order to reduce server load and to get highly responsive scripts. If you want really fast responses you would want to host your own related post server. You can of course use any server that supports python. For google app engine you need an appengine account, which is free and takes only about 5 minutes for registration. Here I explain how to deploy your application to the google app engine, however most of the instructions are generic.

Installation walkthrough:
  • Register at google app engine and claim a subdomain, which is not brps (Yu-Jie's) or findrelatedposts (mine).
  • Download the google app engine SDK (you also need python installed).
  • Download the application files from brps project page:
    hg clone https://brps.googlecode.com/hg/ brps
  • Change the application ID from brps to [YourID] in app.yaml
  • Change brps.appspot.com to [YourID].appspot.com in brps.js and in all python scripts (optionally some HTML templates under template)
  • in python, where it loads simplejson, change the lines so that it says:
    from django.utils import simplejson as json
  • Deploy your new application. From the brps source directory, type:
    appcfg update .

The code is released under the GPLv3 license. Remember that you should release your source code if you make any changes. This helps everyone to profit.

If you are interested in other ways to make your webpage or blog faster, please read my article speed up loading of website and blog.

Oct 11, 2009

Blogger: Bigger Template Edit Box

The Blogger template edit box is tiny and you have to scroll constantly to edit the HTML. The input box for editing the template is just too narrow and too short. In this article I explain how you can increase the size.

In a previous article I introduced Greasemonkey and Platypus, Firefox add-ons, which allow to customize web pages on-the-fly and make the changes persistent. In my last article, I explained how to increase the size of the Blogger post edit box. In this article, again, I assume you have Greasemonkey installed. I show how you can increase the size of the template edit box.

If you have Platypus installed (see previous article), you can customize the HTML template box at your gusto using the mouse which only takes several minutes. Just click one of the Platypus buttons at the toolbar, highlight the template edit box by moving your mouse over it, click right, and choose change style, where you can increase the width and height of the edit box.

You can also download a Greasemonkey user script for increasing the size of the post edit box in Blogger from userscripts. You can find many more examples there, however you should take care they work for your version of blogger.

In the configuration menu of Greasemonkey, make sure that all variations of the URL are caught (use * for all, for example http://www.blogger.com/template-edit* or http://draft.blogger.com/html*).

Blogger: Bigger Post Edit Box

An annoying thing with Blogger is the tiny size of the input box for editing a post. When you write a post, the box is too narrow and too short and you have to scroll constantly. Blogger does not allow to change that size, however you can adapt it using augmented browsing, for example a Greasemonkey scripts. Here I explain how.

In my last article I introduced Greasemonkey and Platypus, Firefox add-ons, which allow to customize web pages on-the-fly and make the changes persistent. Here I assume you have Greasemonkey installed. In this article I show examples how you can adapt the size of the post edit box.

If you have Platypus installed (see last article), you can customize the edit box at your gusto using the mouse which doesn't take longer than several minutes. Just click one of the Platypus buttons at the toolbar, highlight the edit box by moving your mouse over it, click right, and choose change style, where you can increase the width and height of the edit box.

You can also download a Greasemonkey user script for increasing the size of the post edit box in Blogger from userscripts. You can find many more examples there, some many not work for blogger beta, though.

In the configuration menu of Greasemonkey, make sure that all variations of the URL are caught (use * for all, for example http://draft.blogger.com/post*).

In a next article, I will explain how to increase the size of the Blogger template box.

Augmented Browsing in Firefox with Greasemonkey and Platypus

Augmented Browsing refers to systems that allow to customize the look and function of web sites. It allows to customize web pages on-the-fly and make the changes persistent, so that every time you enter a website, the site gets automatically adapted to your need. In this article I give a short introduction to augmented browsing.

Augmented Browsing allows you to change the feel and function of pages. The browser can actively modify content in web pages according to user needs. If you don't like the advertising on the page you can just delete it and never see it again. If you want to have a bigger edit window on a page, you just extend it.

A popular example of augmented browsing is Greasemonkey. Greasmonkey is an extension for Firefox, Opera, and Chrome, which can improve the grease of web pages. It takes scripts (in javascript) that change the behavior of pages.

You can install Greasemonkey from mozilla.addons. After restarting firefox, you write your own scripts or find many scripts for many different web pages on user scripts org. You just click on the script and Greasemonkey should automatically pick it up. After that, any time you enter a specific web page, it will look differently (until you delete the script in the Greasemonkey menu).

A great extension to Greasemonkey is Platypus. It is a Firefox add-on which lets you edit a Web page directly from your browser in a What You See Is What You Get (WYSIWYG) fashion. You can save your changes so that Greasemonkey will repeat them the next time you visit the page.

You can install Platypus from mozilla and at the Platypus home page you can find some basics explained about Using Platypus.

In next articles I will show how to use Greasemonkey and Platypus to make the Blogger edit box bigger and the Blogger template edit box bigger.

Oct 8, 2009

Don't Cut back on Spanish Science!

This is a close translation of an earlier article which was written in Spanish by Erola Pairó, as part of an initiative to protest against the Spanish government's decision to reduce expenditure in science and development. The earlier article was called la ciencia no necesita tijeras, literally science doesn't need scissors.

As an answer to the economic crisis, the Spanish government has decided to cut government expenditure on research and development (R&D) by nearly 18 percent [vanguardia newspaper]. Authors of a blog proposed an initiative to protest against this decision. They called out to bloggers to publish a post on October the 7th, presenting arguments why public investments in R&D should not be cut back.

I think there can be many reasons why the science budget should not decrease. The first that occurs to me is that, as a PhD student, my future could depend on what Spain inverts in science. Although this reason seems very valid to me personally, I decided to look for more objective reasons. In the process I found many studies that indicating that in fact there is a causal relationship between investments in R&D and economic growth. In this post I will explain what I found.

As explain Giovanni Dosi, Patrick Llerena, and Mauro Sylos Labini in an article from 2006, the contribution of scientific research to technology has become more and more important since the industrial revolution, with the rates of technological change often drawing from the strength of the science base. This means that science is a motor for the market. A grand part of the science funding comes from public sources. (The relationships between science, technologies and their industrial exploitation: An illustration through the myths and realities of the so-called "European Paradox", Research Policy)

The economy is changing from an economy based on raw materials to an economy based on knowledge. Countries need an economy which permits them to adapt rapidly to changes in the market and to technological advances. At the Barcelona Spring Council European heads of state and ministers of labor, as one of the conclusions of the council, agreed on an objective of R&D investments of 3 percent of GDP by 2010 (where two-thirds should come from the private sector)[2002 conclusions]. It goes without saying that most European countries including Spain will stay short of this target, exceptions being Sweden and Finland.

The amount of investment that a country dedicates to R&D can be expressed as Gross Domestic Expenditure on R&D (GERD), either in local currency or as percentage of GDP. Another indicator, the BERD, Business Enterprise Expenditure on R&D, represents the money that the private sector spends in R&D.

According to this year's report of the journal R&D magazine titled 2009 Funding Forecast and the data about research provided by the European Commission at Eurostat, GERD (in percent of GDP) has increased in Spain during the last decade, but Spain's 1.2 percent of GDP in 2008 stays below the European average 1.8 percent. Comparing Spain with United States or Japan, which both spend more than 2 percent of GDP, the difference looks even bigger.

The report Main Science and Technology Indicators 2009-1, of the Organization for Economic Co-operation and Development (OECD) includes charts which compare different countries by their R&D expenditure.

In the following chart taken from this report, you can observe the GERD in percentage of GDP for many different countries. You can see that there are European Union member states that lead this ranking (prominently Sweden and Finland) together with United States, Korea, and Japan. Other countries, among which is Spain, let the European average fall below that of the OECD which is little more than 2 percent.


In the next chart taken from the same report you can observe the development of the GERD over the last years in the United States, Japan, and OECD member states. The Europe of 27 is below OECD and Japan. Dosi and others conclude in the article mentioned before that the often-conjured European Paradox, which refers to a notion that the European incapacity to convert their investments in R&D into technological innovation, in fact is baseless. Europe inverts comparatively little in R&D. This makes again clear that Spain, which is below the European average, would have to make an effort to achieve scientific excellence similar to United States, Japan, or Switzerland, to take an example of a geographically closer country.


Below you can see a graph that shows the correlation between GERD, BERD, and GDP of the European Union since 1995 until 2005. As can be seen, the curves for the three indicators are nearly parallel, which shows that the higher GDP, the higher the spending in R&D (source: Creation of The Competitive Environment Concerning Research and Development In The European Union).


However, correlation doesn't imply causality, and therefore I tried to find out if there is evidence that R&D is important for economic growth.

The European Union commissioned different studies to study the relationship and important conclusions are summarized in Key Figures 2005. One of the studies, realized in 2004, (3% d'effort de R&D en Europe en 2010:analyse des consequences à l'aide du model), investigates the effect of increasing the R&D spending to 3 percent of GDP. It concludes that in case of reaching a 3 percent an additional economic growth of 4.1 percent could be expected and a creation of about 3 million jobs by 2015.

This doesn't seem a long time span at all for effects to be taking place.

Another study, again commissioned by the European Commission, undertaken by Roskilde University, Creation of The Competitive Environment Concerning Research and Development in The European Union investigates (among other issues) the relationship between GERD, BERD, and the GDP. One conclusion is that a correlation between the growth of GDP is in fact bidirectionally causal. On the one hand when a country increases its GDP it can spend more on R&D, on the other hand R&D helps the economy to improve.

Does it make sense then to reduce science budget in times of economic crisis? Or would it be better to make an effort to invest in something which could help the economy to grow?

The only rich European country with little investment in R&D (1.8 percent of GDB) is Norway. Norway can do it because it has oil resources. Spain's resources are sun and beach, but it has become abundantly clear that an economic growth based on tourism and construction is not safe in times of crisis. If Spain doesn't want to stay behind in the construction of a knowledge society, the last thing Spain needs is to cut back on its science budget.

When during these days the Nobel prizes will be awarded, it will have to be remembered in Spain that there have been only two Spanish Nobel laureates in science (Santiago Ramón y Cajal in 1906 and Severo Ochoa in 1959) and that Ignacio Cirac, Spanish this year's candidate to a physics Nobel prize is working at Max Planck institute in Germany. Cutting back investments doesn't make it more probable that Spain's situation is going to improve.

Oct 7, 2009

La Ciencia en España No Necesita Tijeras

This article is written in Spanish as part of an initiative to protest against the Spanish government's decision to reduce expenditure in science and development. A translation will follow up as another article, called Don't cut back on Spanish science!

Como respuesta a la crisis económica el gobierno español ha presentado unos presupuestos que prevén un recorte de un 17,7 por cien en I+D+i [vanguardia] . Varios colectivos relacionados con la investigación han mostrado su preocupación por este recorte y desde un blog, la Aldea Irreductible, se propuso una iniciativa para protestar, que se llama la ciencia española no necesita tijeras. Cada blog apuntado a la iniciativa publicará hoy un post exponiendo razones por las que el presupuesto en investigación no se debería cortar.

Creo que se pueden dar muchas razones para no bajar el presupuesto en investigación. La primera que se me ocurre es que, como estudiante de doctorado, mi futuro puede depender de lo que España invierta en ciencia. Aunque ésta me parece una razón muy válida, he decidido buscar alguna más objetiva. En este proceso he encontrado muchos estudios que relacionan causalmente el crecimiento economico con la inversión en I+D y enseguida lo explico.

la ciencia española no necesita tijeras

Como explican Giovanni Dosi, Patrick Llerena y Mauro Sylos Labini en un artículo de 2006, desde la revolución industrial, la contribución de la ciencia en la tecnología ha sido cada vez más importante. La innovación viene muchas veces modelada por una ciencia de base que es el producto del conocimiento producido por la investigación, y que en gran parte se financia con dinero público. Es decir, la investigación, y la innovación como su consecuencia, se han convertido en uno de los motores del mercado. (The relationships between science, technologies and their industrial exploitation: An illustration through the myths and realities of the so-called European Paradox, Research Policy)

La economía está cambiando desde una economía basada en las materias primas, a otra basada en el conocimiento. Los países deben tener una economía que permita adaptarse rápidamente a los cambios del mercado y producir avances tecnológicos. Para sentar las bases de esta nueva economia los jefes de estado de la Unión Europea propusieron en el Barcelona Spring Council en 2002, la idea de llegar a un 3 por ciento del producto interior bruto (PIB) invertido en investigación en 2010. No hace falta decir que España no va a llegar a conseguir implementar este compromiso, de hecho con la excepción de Suecia y Finlandia, ningún país de la Unión Europea va a conseguirlo.

La cantidad de dinero que un país dedica a la investigación, se calcula normalmente mediante un indicador llamado GERD (Gross Domestic Expenditure on R&D), que representa el PIB invertido en I+D calculado en moneda local (millones de euros/dolares) o en porcentaje del PIB total del país. Otro indicador, el BERD, (Business Enterprise Expenditure on R&D), representa el dinero invertido solamente del sector industrial).

Según el informe anual de la revista R&D magazine titulado 2009 Funding Forecast y los datos sobre investigación proporcionados por la Comisión Europea en el Eurostat es verdad que el GERD (en porcentaje del PIB) ha aumentado en España en la última decada, pero, el 1,2 por cien de 2008 de España del PIB aún queda por debajo de la media europea que ronda el 1,8 por cien. Si comparamos España con Estados Unidos o Japón, países que le dedican más de un 2 por cien de su PIB, el hueco es aún más grande.

El informe Main Science and Technology Indicators 2009-1, de la Organización para la Cooperación y el Desarrollo Económico (OCDE) incluye unas gráficas de los diferentes indicadores de la inversión en investigación, y la comparación entre los diferentes países miembros. En la siguiente gráfica, sacada de este informe, se puede observar el GERD en porcentaje del PIB para muchos de los países miembros. Se puede ver como hay países miembros de la U.E. (el caso de Suecia y Finlandia que hemos mencionado antes) que lideran este ranking junto con Estados Unidos, Corea y Japón. Otros países, entre los que se encuentra España, hacen que la media de la Unión Europea baje hasta estar por debajo de la media de los países de la OCDE que es un poco superior al 2%.

comparison of science budget of different countries

En esta otra gráfica del mismo informe, lo que se muestra es la evolución del indicador en la Unión Europea, Japón, Estados Unidos y la media de la OECD en los últimos años. La Europa de los 27 está muy por debajo de los otros países. El artículo de G.Dosi y otros mencionado antes, concluyen que la supuesta European Paradox, o la imposibilidad Europea de traducir su inversión en ciencia en innovación tecnológica, no existe y que Europa está por debajo de los países miembros de la OCDE en lo que inversión científica se refiere. España se encuentra por debajo de la media Europea, así que tanto en esta gráfica como en la anterior se hace evidente el esfuerzo en inversión que se debe hacer si España quiere llegar a niveles científicos parecidos a los de Estados Unidos, Japón, o otros miembros de la OCDE como Suiza para citar un ejemplo más cercano.

japan, oecd and europe and their research spending during the last years

Viendo el nivel de desarrollo de los países mencionados antes, parece claro que hay una correlación entre la riqueza de un país, y su inversión en I+D. Como ejemplo abajo se puede ver el gráfico correspondiente a la correlación entre GERD, BERD y PIB de la Unión Europea desde 1995 a 2005 (fuente: Creation of The Competitive Environment Concerning Research and Development In The European Union).

correlation between spending in R&D and GDP

Como es conocido, una correlación no implica una causalidad, así que para aclarar si una inversión en I+D es importante para un crecimiento económico, he buscado estudios que investiguen la correlación entre indicadores.La Unión Europea encargó diferentes estudios para estudiar esta correlación, las conclusiones más importantes se encuentran resumidas en el informe Key Figures 2005. Uno de los estudios, realizado por P. Zagamé en 2004 (3% d'effort de R&D en Europe en 2010:analyse des consequences à l'aide du model), estudia el efecto de aumentar la inversión en I+D hasta un 3% del PIB. Concluye que, en caso de llegar a este porcentaje, se esperaba un crecimiento adicional del PIB del 4,1% y una creación de más de 3 millones de empleos para 2015. El efecto en la economía al invertir en I+D no parece a tan largo plazo, así que me animé a buscar más.

Otro estudio, encargado también por la comisión Europea a la Roskilde University, Creation of The Competitive Environment Concerning Research and Development in The European Union estudia (entre otras cosas) la relación entre el GERD, el BERD y el PIB de un país. La conclusión más importante es que se puede establecer una correlación entre el aumento de PIB de un país y su inversión, tanto porque cuando un país aumenta su PIB puede realizar una inversión mayor como porque un aumento en la inversión en investigación ayuda a mejorar la economía del país.

¿Parece lógico, entonces, disminuir el presupuesto en investigación en época de crisis? ¿O sería mejor hacer un esfuerzo para invertir en algo que pudiera ayudar al crecimiento del país? El único ejemplo europeo de país con un gran nivel de vida y que no invierte demasiado en I+D (un 1,8% del PIB) es Noruega, y le es posible hacer esto porque tiene grandes recursos naturales (petróleo) y su I+D está destinado sobretodo a mejorar las tecnologías para explotarlos.

Supongo que el gran recurso a explotar por España sería el Sol y la playa pero yo creo que España ya ha abusado suficiente de tener un crecimiento basado en el turismo y la construcción, que además ha demostrado ser ineficaz en tiempos de crisis. Si se tiene que cambiar el modelo económico, si España no quiere quedarse atrás en la construcción de una economía basada en el conocimiento, está claro que lo que no necesita la ciencia española son tijeras.

En estos días que se reparten los premios Nobel, no está de más recordar que sólo dos científicos españoles lo han conseguido y que Ignacio Cirac (candidato al Nobel de física) está investigando en el Instituto Max Planck en Alemania. Sin invertir no es probable que la situación en España mejore.

Oct 6, 2009

Elegant Label Navigation: Ajaxlabels

Ajaxlabels is a blogger functionality which allows previewing articles pertaining to labels without leaving the current page. This can be much faster than waiting each time for the label page to load. I found this cool hack at blogger-hacked. Credits go to Deepak Ranganathan. I explain it here, because I update it by a simple fix and because the way I explain you save hosting of javascripts. In this post I'll explain how to install ajaxlabel navigation in four simple steps. I only assume you know how to create standard widgets by point-and-click and how to edit the HTML template.

For this hack to work you need a label widget which will the navigation element and another widget where you want the preview be displayed. Then you'll need to edit the HTML of the template and insert a short code and a stylesheet.

Step 1: Create a Label Widget

If you don't already have a label widget which you want to use for the navigation, in layout-> page elements create one.

I'll assume you'll use the first label widget with the id label1, if it's the second the ID should be label2. If you are not sure which ID your label widget has, edit the widget and see the id in the address bar.

Step 2: Create an HTML Widget

Create an HTML widget where you want the preview displayed and insert this code:

<div id="indicator" style="display:none"><img alt="Indicator" width="16" src="http://myoutsourcedbrain.appspot.com/images/indicator.gif" height="16"/> Loading...</div>
<div id="search-result" style="display:none"><a name="sres"/><div class="search-close" onclick="javascript:Element.hide(&apos;search-result&apos;)"><script type="text/javascript">document.write(' X ');</script></div><div id="show-result"/><div class="search-close" onclick="javascript:Element.hide(&apos;search-result&apos;)"><script type="text/javascript">document.write(' X ');</script></div></div></a></div>


Step 3: Edit HTML Template and Change Label Widget

Go to layout->html. Make a backup by clicking download template. Check expand widgets, search for label1 (or whatever the ID is, see above). The widget code is between <b:widget id='Label1' locked='false'> and </b:widget>. Substitute the widget code for the following:

<b:widget id='Label1' locked='false' title='Browse Topics' type='Label'>
<b:includable id='main'>
 <b:if cond='data:title'>
     <h2><data:title/></h2>
 </b:if>
 <div class='widget-content'>
     <b:loop values='data:labels' var='label'>
            <b:if cond='data:blog.url == data:label.url'>
              <data:label.name/>
            <b:else/>
<a expr:onclick='&quot;javascript:getCat(\&quot;&quot; + data:label.name + &quot;\&quot;,null)&quot;' href='#sres' rel='tag'><data:label.name/></a>
            </b:if>
<b:if cond='data:label.isLast != &quot;true&quot;'>|</b:if>
     </b:loop>
     <b:include name='quickedit'/>
 </div>
 </b:includable>
</b:widget>


Step 4: Insert Code Inside Head

Search for </head> and insert the following code above:

<!-- Begin Scriptaculous includes -->
<script src='http://jsload.net/init'>
scriptaculous
</script>
<!-- End Scriptaculous includes -->
<style type="text/css">
/* SEARCH/CATEGORY STYLES
----------------------------------------------- */
#indicator {
position: absolute;
z-index: 10000;
padding: 15px 0;
top: 22em;
background-color: #FFFFFF;
border: 1px solid #AAAAAA;
width: 176px;
left: 50%;
margin-left: -88px;
text-align: center;
}

#search-result  {
 border: 1px solid #AAAAAA;
 padding: 10px;
 padding-bottom:30px;
 font-size:85%;
}
.search-result-meta {
 background: #EFEFEF;
 padding: 2px;
}
.search-result-meta img {
 border-width:0;
 vertical-align:text-bottom;
}
.search-title {
 font-size:1em;
 padding-bottom:3px;
 font-weight:bold;
 text-align:left;
 text-decoration:underline;
}
.search-cat {
 display:block;
 padding:3px;
 font-size:1em;
 margin-top:5px;
 margin-bottom:5px;
 border-bottom:1px solid #C0C0C0;
 font-weight:bold;
}
.search-close {
 color:silver;
 float:right;
 border:1px solid #F5F5F5;
 margin-top:10px;
 cursor:pointer;
} 
</style>
<!-- Begin AJAX Categories -->
<script type='text/javascript'>
//<![CDATA[
/*****************************************************/
/* AJAX Categories                                   */
/* Coded by Deepak Ranganathan                       */
/* http://blogger-hacked.blogspot.com                */
/* Protected by Creative Commons License             */
/* http://creativecommons.org/licenses/by-nc-sa/2.5/ */
/*****************************************************/
var gCat;
function getCat(cat) {
   var url= '/feeds/posts/summary/-/' + cat + '?max-results=5&alt=json';
   var pars = '';
   gCat = cat;
  
   new Ajax.Request(
       url,
       {
          method: 'get',
          parameters: pars,
          onLoading: function(){Element.show('indicator')},
          onSuccess: showCat,
          onFailure: showFail
       });
}

function showCat(output){
   Element.hide('indicator');
   Element.show('search-result');
   $('search-result').innerHTML="";
   var main = string2JSON(output.responseText);
   var xtitle, xcat, xcontent;
  
   var meta = document.createElement('div');
   meta.className="search-result-meta";
  
   var link = document.createElement('a');
   link.href="/feeds/posts/summary/-/"+gCat;
   link.innerHTML = "<img width=\"16\" height=\"16\" src=\"http://myoutsourcedbrain.appspot.com/images/feed-icon.gif\" title=\"Subscribe to this category\" /> Subscribe To " + gCat;
   meta.appendChild(link);
  
   $('search-result').appendChild(meta);
  
   for (i=0; main.feed.entry[i]; i++) {
      xentry = main.feed.entry[i];
      catdisplay = "<div class=\"search-cat\"><span style=\"color:silver;\"> Categories:<\/span> ";
      for (var j=0;xentry.category[j];j++) {
         catdisplay = catdisplay + xentry.category[j].term;
         if(j<xentry.category.length-1)
            catdisplay = catdisplay + " , ";
      }
      xtitle=xentry.title.$t;
      xcontent=xentry.summary.$t;
      xlink=xentry.link[0].href;
          for (var k = 0; k < xentry.link.length; k++) {
            if (xentry.link[k].rel == 'alternate') {
              xlink = xentry.link[k].href;
              break;
            }
          }

      $('search-result').innerHTML = $('search-result').innerHTML + "<h3 class=\"search-title\"><a href=" + xlink + ">" + xtitle + "<\/a><\/h3>";
      $('search-result').innerHTML = $('search-result').innerHTML +  xcontent + "...";
      $('search-result').innerHTML = $('search-result').innerHTML + catdisplay + "<\/div>";
   }
   $('search-result').innerHTML = $('search-result').innerHTML + "<div class=\"search-close\" onclick=\"javascript:Element.hide('search-result')\"> X <\/div>";
   $('search-result').style.display="block";
   new Effect.Highlight('search-result');
}

function string2JSON(str){
   var n;
   //We have to do exception handling here because eval might not work in all cases
   try {
      eval("n="+str);
   }
   catch (e) {
      n= null;
   }
   return n;
}

function showFail(){
   Element.hide('indicator');
   $('search-result').innerHTML="Sorry! The page cannot be loaded.";
   $('search-result').style.display="block"
}
/**************************/
/* End of AJAX Categories */
/**************************/
//]]>
</script>
<!-- End AJAX Categories -->
<!-- thanks, Deepak Ranganathan -->


Save the template and enjoy!

Final Comments

The javascript includes a fix I included. The links to the articles didn't work for me without it. They used to point to the comment feed instead.

The image indicator.gif is displayed while the query is processed. It is for black backgrounds. If you have a light background use http://bloggerhacked.googlepages.com/indicator.gif instead. If you plan to use this hack for a longer time you should host the image on a free file server, e.g. google sites.

The trick which connects the labels to the preview is this link:
<a expr:onclick='&quot;javascript:getCat(\&quot;&quot; + data:label.name + &quot;\&quot;,null)&quot;' href='#sres' rel='tag'>
. Any label you link this way will be connected to your preview.

Jsload is an API for fast loading of javascript libraries hosted on google app engine.

This script uses prototype.js. If you want to use it with jquery you will have to invoke jquery.noConflict() as one of the first commands after loading. See the Using jQuery with other libraries at jquery docs.

If you ask yourself why I don't ajaxlabels for navigation, it's because I think the site loads fast enough to do without it. See the screenshot below to see how it looks like. Above you see the label navigation widget, below the preview widget.


For more details about CSS and script parameters please see the original article.

Oct 5, 2009

Speed Up Loading of Website and Blog

firebug download cascadeThere are so many widgets, scripts, and multimedia to put on a website! And many of them are so nice. Problem is, if you put too many of them, your site will load very slowly. This will drive visitors away and will let search engines to rank you worse. The faster your site loads, the better. General advice: your site should load in about 10 seconds, otherwise people will get bored. Here I describe several possibilities to reduce loading time of your site.

The image shows how items from a site are downloaded (credit: getfirebug.com). You can see that items are downloaded in a sequence rather than in parallel. This is one source for improvement.

Possibilities to improve performance of your site fall roughly into these two categories:
  • Remove files and optimize files.
  • Migrate files to a content delivery network for lower latencies.

Any intent to accelerate your site should start with a thorough analysis of the bottlenecks. It won't help reducing file sizes if your real problem is the amount of server connections which each wait for response for a long time. In turn, if you have a lot of videos and 80 external files of 2Mb each, no wonder your side will slow down. You should attack the problem at its root.

In this article I will explain how to analyze and remove loading bottlenecks and how to increase the loading speed of your site.

Analyzing the Problem

There are several analysis tools you can use. There are of different levels of sophistication and consequently utility. None of them are difficult to use.

Some web-driven applications give you benchmarks of your site. You only have to type your URL and you'll get a first idea. You'll get feedback such as "too many files" or "this file is too big." You'll see estimates of how long your site will take to download at different bandwidths.

A short collection of sites that provide these online speed tests for free and without registration:

These on-line performance tools are fast and comfortable, however useful only to an extent. They miss information about latency. Latency is often more important than file size. Often it can take longer to wait for a server to respond than to actually download the file.

This information you'll get with firebug. Firebug is a firefox extension for debugging, editing, and profiling of websites. There is a firebug lite plugin for opera and internet explorer, however without the profiling option, so for the analysis you have to load the site in firefox and firebug shows you how long each of the components take. You'll also see that many files download in parallel. To get you started, see the firebug tutorial at ibm.

File Optimization: Reduce File Size, Remove Files

First step to a speed up: get rid of anything you don't need. Delete widgets you don't need. Remove scripts which are slow. Remove superfluous big images and videos. This reduces clutter and can make your site clearer.

Second step: There are techniques to reduce the size of files and to make them load more quickly. You can change HTML code, style sheets, javascripts (js), and images to make your site appear faster. Each of the file types can each be compressed to some extend, let's see how to do that for each filetype in turn.

HTML Optimization

HTML code are all the tags inside the greater than (>) and less than (<) signs. It is the organization of the HTML which can make a file appear load faster. If you move the CSS (inside <style> and <skin>) to the beginning of the header the side will appear faster, because the browser's layout engine doesn't have to wait to know how and where to display. Javascripts (between <script> tags) can have mutual dependencies, which is why browsers tend to download them sequentially. Anyways, the dynamical elements are only useful once the user interact with the page and this is when the page is downloaded. Therefore it is recommended to move as much of the javascript to the bottom as you can. HTML can have some clutter which makes your site slow. Tables and images without the <width> and <height> attributes stops appearance of content. Without these attributes the display has to wait until image or table content has downloaded in order to determine size. You should also avoid scaling of images at loading time. Scale the images already to the size you want them displayed at. While we are at it, many <meta> tags you don't need. While we are at it clear it out.

HTML documents typically are not cached because their content is constantly changing. So moving out CSS and scripts can help faster repeated loading if the browser caches your site.

This leads to cache control. The browser loads some files into memory, so it doesn't have to download them again. If the browser caches aggressively, the site will load much faster after the first time. There are some HTML meta tags to give the browser indication cache and server side commands for external files (.htaccess). Setting the file expiration to many days can help a lot. Read more about caching in the Caching Tutorial for Web Authors and Webmasters.

Image Optimization and Compression

Images can often be reduced a lot. Do you really need a file of resolution 1000x1000? Also, all meta information about the file, where it was created and so on, is not important for the viewer of the page. Again: scaling images during loading by width and height attributes costs time. Resizing them to correct size beforehand helps. JPEG and PNG (Portable Network Graphics) formats usually result in smallest files.

There are many free tools for optimizing file sizes and reducing resolutions. Gimp is a visual application similar to photoshop. One of the best command line tools that does everything from converting and editing is ImageMagick. Pngcrush and pngtopnm are optimizers for PNG files. Both perform lossless compression, meaning no quality will be lost. Jpegoptim and jpegtran provide lossless and lossy methods for size reduction of JPEGs.

CSS Minification

CSS minification involves deleting all unnecessary characters, such as whitespaces, and shortening names. There are on-line minifiers, like this css compressor tool. CSStidy CSStidy is a command line tool, which optimizes CSS stylesheets.
A general rule for CSS use is to avoid CSS expressions. They are a way to set CSS properties dynamically. These expression might be evaluated many times during page load and can slow down page loading a lot.

Javascript Minification and Compression

Compress all accessible javascripts, The compressor rater compares compression of javascript by four different methods, JSMin, Dojo shrinksafe, Packer, and the the YUI Compressor, with and without additional gzip compression, and lets you download . Packer (v. 3) is one of the most effective methods.

Compressing files using gzipper has the risk however that some browsers may not support gzip files or not recognize the MIME type. File compression should be done server-side.

Content Delivery Network - Reducing Latency

If you have many files (scripts, images and other multimedia, stylesheets) which have to be downloaded from servers this can cause a problem of latency. It can help to concatenate files to have less files, which means less waiting time for server response. However there is more you can do.

What is better, that visitors can downloaded and view your pages from a single server or from many servers around the world whichever is closest? What is better, that some experienced programmers take care of optimization of servers so that they can take care of even high demands, like a slashdot effect, or that you have to do it? I think the answer is obvious: content delivery network and the best of it is that it is free (until you really hit traffic).

Content Delivery Networks (CDN) are services that provide file servers distributed so as to maximize bandwidth. This means that visitors to a website can be served content from a server nearby. Big names in the CDN business are google, amazon, and yahoo. Google keeps the location of its data centers a secret, but in the image below you can see the server locations of the coral content delivery network, to give you an idea of what a CDN is.


Google's app engine's free service has some access quota, which can be extended for pay. A very good tutorial and introduction is provided at 24ways: using google app engine as your own CDN.

Yahoo performance rules, which provides general tips on performance improvement. It was one of the best resources I found during my research. You can find some more resources in the post Best Blog Tools.