Pattern matching for string and list in erlang

Earlier I wrote a post on how to distinguish string from list. Steve and Justinpointed that my approach will not work for a list of anythingg other than string.

Here is what I am trying to solve. I have a main configuratin file and client specific configuration files. Client specific configuration file can override or add more values from main configuration file. If the value in main configuration file is a string, I want the client configuration to override the value of main configuration. If the value in main configuration is a list, I want to append the list from the client configuration to the main configuration.

For example, main configuration file is

[{blacklist, ["foo.com", "aaa.com"]},
{source_host, "http://www.bar.com"}].

and client configuration file is

[{blacklist, [ "bar.com" ]},
{source_host, "http://www.something.com"}].

After merging above 2 configuration, I want the final configuration to look like,

[{blacklist, [ "foo.com", "aaa.com", "bar.com" ]},
{source_host, "http://www.something.com"}].

The code reading the configuration files and merging them does not know the first element of the tuple ahead of time. The code is generic to read a file of list of tuples and merge the values to generate the final configuration.

Since Erlang implement string as a list, using is_list guard for a function will not work. If anybody has solved this problem earlier, please comment on this blog.

Differentiate string from regular list in Erlang

Erlang is a great language to work with. I need to concentrate on the business logic rather than nifty details about the distributed computing like communication between nodes, handle node failures, restart apps etc. All these details are already taken care by the language itself. Yes.. distributed computing and concurrent programming is a integrated part of the language.

Erlang implement string in terms of list. So it is tricky to distinguish regular string from the list. For example,

foo(C) when is_string(C) ->
   % do something with string;
foo(C) when is_list(C) ->
   % do something with list.

First of all there is no function called "is_string" in Erlang. (I just made it up). But the point here is that how to distinguish regular string from the list. I came up with this solution.

foo(C) when lists:flatten(C) == C ->
   % do something with string;
foo(C) when is_list(C) ->
   % do something with list.

Somewhat inefficient because of call to lists:flatten/1 call. But for small strings, it is not an issue at all.

Shame on you Mr. Manmohan Singh....

Here is a comment from Condoleezza Rice:

US Secretary of State Condoleezza Rice has told Pakistan that there is "irrefutable evidence" of involvement of elements in this country in the Mumbai terror attacks and it had no option but to act urgently "otherwise, the US will act."

Dispute with Pakistan is of India's own. Should Indian government wait for USA to protect it inspite of having one of the biggest and powerful army in the world? Shame on you the respected prime minister of Manmohan Singh. I expected such a statement to come from Indian government as soon as India had enough proof to implicate "Elements within Pakistan" for the attack on Mumbai on 26/11. Why should Indian government present the proof to US government and wait for them to act on behalf of us? Isn't that shameful?

I am not supporting war with Pakistan as it is bad for the people of India as well as Pakistan. The terroists activity in Mumbai has nothing to do with the people of Pakistan. It is some groups freely operating within Pakistan is responsible for it. Punishing the whole country for the act of few people is not a wise idea. But India should have acted against the group immediately. Do what US did in Afghanistan. Ask the government to hand over the people responsible. If they don't, get in and take them out yourself. Don't wait for some other country to do that for you.

Shame.. Shame.. Shame...

A Letter to the Prime Minsiter of India

Today I received this email from one of my colleague. It really captured my attention. Here is the letter.

LETTER TO PRIME MINISTER


Dear Mr. Prime minister


I am a typical mouse from Mumbai. In the local train compartment which has capacity of 100 persons, I travel with 500 more mouse. Mouse at least squeak but we don't even do that.

Today I heard your speech. In which you said 'NO BODY WOULD BE SPARED'. I would like to remind you that fourteen years has passed since serial bomb blast in Mumbai took place. Dawood was the main conspirator. Till today he is not caught. All our bolywood actors, our builders, our Gutka king meets him but your Government can not catch him. Reason is simple; all your ministers are hand in glove with him. If any attempt is made to catch him everybody will be exposed. Your statement 'NOBODY WOULD BE SPARED' is nothing but a cruel joke on this unfortunate people of India.

Enough is enough. As such after seeing terrorist attack carried out by about a dozen young boys I realize that if same thing continues days are not away when terrorist will attack by air, destroy our nuclear reactor and there will be one more Hiroshima.

We the people are left with only one mantra. Womb to Bomb to Tomb. You promised Mumbaikar Shanghai what you have given us is alianwala Baug.

Today only your home minister resigned. What took you so long to kick out this joker? Only reason was that he was loyal to Gandhi family. Loyalty to Gandhi family is more important than blood of innocent people, isn't it?

I am born and bought up in Mumbai for last fifty eight years. Believe me corruption in Maharashtra is worse than that in Bihar. Look at all the politician, Sharad Pawar, Chagan Bhujbal, Narayan Rane, Bal Thackray, Gopinath Munde, Raj Thackray, Vilasrao Deshmukh all are rolling in money. Vilasrao Deshmukh is one of the worst Chief minister I have seen. His only business is to increase the FSI every other day, make money and send it to Delhi so Congress can fight next election. Now the clown has found new way and will increase FSI for fisherman so they can build concrete house right on sea shore. Next time terrorist can comfortably live in those house, enjoy the beauty of sea and then attack the Mumbai at their will.

Recently I had to purchase house in Mumbai. I met about two dozen builders. Everybody wanted about 30% in black. A common person like me knows this and with all your intelligent agency & CBI you and your finance minister are not aware of it. Where all the black money goes? To the underworld isn't it? Our politicians take help of these goondas to vacate people by force. I myself was victim of it. If you have time please come to me, I will tell you everything.

If this has been land of fools, idiots then I would not have ever cared to write you this letter. Just see the tragedy, on one side we are reaching moon, people are so intelligent and on other side you politician has converted nectar into deadly poison. I am everything Hindu, Muslim, Christian, Schedule caste, OBC, Muslim OBC, Christian Schedule caste, Creamy Schedule caste only what I am not is INDIAN. You politician have raped every part of mother India by your policy of divide and rule.

Take example of former president Abdul Kalam. Such a intelligent person, such a fine human being. You politician didn't even spare him. Your party along with opposition joined the hands, because politician feels they are supreme and there is no place for good person.

Dear Mr Prime minister you are one of the most intelligent person, most learned person. Just wake up, be a real SARDAR. First and foremost expose all selfish politician. Ask Swiss bank to give name of all Indian account holder. Give reins of CBI to independent agency. Let them find wolf among us. There will be political upheaval but that will better than dance of death which we are witnessing every day. Just give us ambient where we can work honestly and without fear. Let there be rule of law. Everything else
will be taken care of.

Choice is yours Mr. Prime Minister. Do you want to be lead by one person or you want to lead the nation of 100 Crore people?

What should India do about Mumbai attacks?

During his preliminary interrogation, Qasad believed to have reportedly told police, "I have done right and I have no regrets." He has also admitted to being a member of the Lashkar.

This what Qasad said who was captured by NSG in Mumbai. Should India listen to this and keep quite? I say NO... Make him regret. Whatever means necessary.

If I was given control, I would make him regret by making his loved one loose their life in front of his eyes and set an example for the future. Make people think about the consequences if at all they decide to do something like what happened in Mumbai.

For those who say I am cold, I would say bring it on.

Who is responsible for Mumbai attack?

US expert on South Asia dubs Mumbai attacks as India's "domestic issue"

Here is what he said:

This is a domestic issue. India's domestic problems and long tensions between local communities were at the root of the rise of terrorism in the country

How come when the attack is in US, it is global terrorism and when the attack is on other countries (ofcourse other than the so called friends of US) it is domestic issue.

This guy failed to check that the attackers were not from India, but everyone of them were from Pakistan. More over, Pakistan reverted its decision to send the head of ISI to India to share the information on this terrorist attack.

Mr/Ms. Christine Fair, get your facts right before making any statements.

Read more about it here.

Wah bhai wah!

Parallel assignment

Today I was working on some problem (off course in Erlang). What I am trying to achieve was to execute a function in parallel and assign result of function to a variable.

Func = fun(Arg) ->
          Parent = self(),
          Pid = spawn( fun() -> timer:sleep(1000), Parent ! {self(), Arg + 2} end ),
          receive {Pid, Val} -> Val
       end,
RetVal = Func(),
SomeOtherValue = 10,
% rest of the code
io:format( "~p~n", [RetVal] ),
% more code here

In the above code, calling Func() block the execution until receive loop within Func is done receiving the data from the spawned process. SomeOtherValue is assigned only after the execution of Func.

What I want is not that. I want the execution to continue even if the execution of Func is not finished. I want to block only when I am trying to use RetVal variable.

Currently I am acheiving this by waiting for the response from the spawned process only when I need the data from the process, i.e., same as blocking when the variable is accessed.

Func = fun(Arg) ->
          Parent = self(),
          Pid = spawn( fun() -> timer:sleep(1000), Parent ! {self(), Arg + 2} end ),
       end,
Pid = Func(),
SomeOtherValue = 10,
% rest of the code
RetVal = receive {self(), Val} -> Val end,
io:format( "~p~n", [RetVal] ),
% more code here

Even though it is a trivial code change, I would like if the parallel assignment is allowed and block when I try to use it (if necessary).

Erlang and Garbage collection

In my new project, we did the whole coding in erlang. Yaws is our front-end. When running a load test, we observed that memory usage goes high and ultimately crash the erlang virtual machine. We did a thorough analysis of our code to find any possible memory leak (not possible as erlang uses garbage collection) and also studied the code for any possible recursion instead of tail recursion. We did not find any. It was pretty puzzling. Then I did find one strange thing. This is all about erlang garbage collection works.

In turns out that Erlang's garbage collection works per process. Each process has its own heap and stack. If a process is a long running one and does lot of things for each call, memory gets accumulated and will get released very slowly. When Erlang virtual machine is busy doing things, garbage collection thread gets lower priority and may never get called. Even if it is called, it may not do the whole memory garbage collection. Thus, memory keeps on growing.

In our code, we were running a gen server. This gen server was doing lot of work in the same process. We were running several hundreds of these gen_servers (monitored by a supervisor). Child specs for the supervisor was created dynamically depending on the initial configuration. In nutshell, it was doin the following

-module(foo_server)
-behaviour(gen_server)

% Not outlining all methods exported here, only the required ones

execute(Params) ->
   gen_server:call({run, Params}).

handle_call({run, Params}, _From, State) ->
  some_module:execute(Params). % some_module:execute does most of the work here.

Note that here some_module:execute/1 runs in the same process as gen_server. Idea here was that gen_server is monitored by supervisor and we get free process monitoring. But since same process gets called over and over again, memory accumulates. Running load test was increasing memory usage.

Now Erlang process comes to rescue. Because of functional aspect, it got pretty easy to change the code to run some_module:execute/1 in a different process and return the result to the caller via process message passing. I did the following changes

-module(foo_server)

execute(Params) ->
  MySelf = self(),
  spawn( fun() -> Result = some_module:execute(Params), MySelf ! {MySelf, Result} end),
  receive 
      {MySelf, R} -> R;
      after 1000 -> {error, timeout}
  end.

Eureka! Running the same load test did not result in increased memory usage. Memory was pretty stable even after running the load test for more than an hour.

URL Shortcut in Firefox

Firefox has this amazingly useful but hidden feature of adding shortcut for a bookmark. This is not just restricted to search based urls. Here is how to do this.

1. Bookmark your favorite URL
2. Go to Bookmarks -> Organize Bookmarks... menu
3. Choose the bookmark from (1)
4. In the bottom panel, click on "More"
5. Enter shortcut word or letter in "Keyword" text box
6. Close "Organize Bookmarks" window
7. In the address bar, type the word or letter from (5)
8. Viola!
9. Enjoy the time saved in taking the mouse pointer to Bookmarks menu and selecting the bookmark :-)

Quatum of Solace...

Java is like writing story.

Here is a code to get md5 signature for a text

public static String MD5(String text) 
        throws NoSuchAlgorithmException, UnsupportedEncodingException  {
                MessageDigest md;
                md = MessageDigest.getInstance("MD5");                
                byte[] md5hash = new byte[32];   
                md.update(text.getBytes("iso-8859-1"), 0, text.length());
                md5hash = md.digest();
                return convertToHex(md5hash);
}

private static String convertToHex(byte[] data) {
        StringBuffer buf = new StringBuffer();
        for (int i = 0; i < data.length; i++) {
                int halfbyte = (data[i] >>> 4) & 0x0F;
                int two_halfs = 0;
                do {
                        if ((0 <= halfbyte) && (halfbyte <= 9))
                        buf.append((char) ('0' + halfbyte));
                    else
                        buf.append((char) ('a' + (halfbyte - 10)));
                        halfbyte = data[i] & 0x0F;
                } while(two_halfs++ < 1);
        }
        return buf.toString();
}

Here is code in perl

use Digest::MD5 qw(md5_hex);
my $digest = md5_hex($text);

This is 20+ lines vs. 3 lines. Java seems to make everything over-object-oriented. Why do I need to getInstance(), update and then call digest() to just get the md5 of the string?

Viewing PDF from within Firefox on Mac

I was searching for the firefox plugin to view pdf files within the browser (embedded). I was not able to find the anything useful via google search. But Yahoo! search got me the result, that too first one...

Here is the page

Hurray.. At last I can read PDFs within Firefox without downloading it on to my desktop. No more folder clutter..

How to setup grid using Erlang?

Erlang is a functional programming language with the built-in support for concurrency and distributed computing. Nodes running erlang virtual machine can communicate with each other without any additional coding.

Erlang use light weight process model to achieve the distributed computing. Erlang processes are lighter than linux thread. One erlang node can run more than 10k (sometime more than 100k) processes on a single node.

Erlang process is started using spawn or spawn_link function. A process can be started on the current vm (virtual machine) or on any other vm in the grid by just adding one additional parameter to these functions.

In order for 2 nodes to communicate with each other, both nodes should have the same cookie. In a grid, all erlang nodes should have the same secret (set via -setcookie command line option or using erlang:set_cookie/2 function)

Erlang nodes follow the following naming convention.

name@hostname where name can be any arbitrary name

In this example, I am assuming that I have 2 hosts namely node1.example.com and node2.example.com

Run the erlang vm on each node with short name

On node1.example.com,

erl -sname foo -setcookie mysecret

On node2.example.com,

erl -sname bar -setcookie mysecret

On node1.example.com, ping node2.

net_adm:ping('bar@node2.example.com').
pong
nodes().
['bar@node2.example.com']

Why Microsoft just don't get it?

Today Microsoft's Live Search announced SearchPerks program. Users are given point when they use Live Search. These points can be re-deemed for X-Box games, t-shirts etc. But the catch is you have to use Internet Explorer 6.0 or higher.

To earn value for your searches and join the SearchPerks! promotion you must have Internet Explorer 6.0 or higher to participate

Microsoft just don't get that Internet is a open space and users should be allowed to use whatever browser they are using.

I can do everything in C or in Java..

Here is the philosophy that should be used by programmers

As a programmer you should spend your time expressing your ideas, not jumping through hoops to accommodate a language. If you spend the majority of your time doing the latter, the language you're working in probably isn't very good.

From here

No shadow building

Le Project Triangle is one of those buildings that make us think that we may actually drive flying cars one day. To be completed by 2014 in the Porte de Versailles area in Paris, its most impressive feature is that, according to the architects, it won't cast shadows on adjacent buildings. The trick is the orientation and it's shape: While it looks like a massive pyramid from one side, the other side shows that it really is an ultra-thin triangle resembling a shark's fin

More here

Threads are at last in C++ standard

It's been decades since I am doing thread programming in C++. It was so painful to use POSIX thread APIs. It is like writing a same novel many times. Not anymore... C++0x standard is out there now. Yahoooooo.. Thread is included in the standard now. Read more about it here

Illustration of Boost shared_ptr

Illustration of Boost Shared Pointer - Upload a Document to Scribd