Planet Smalltalk

• an object model from Smalltalk
• compiler optimizations from Self
• types from Strongtalk
• concurrency from Erlang
• syntax from Javascript and C

"One of our key goals with Dart is "toolability", and as the person leading our tooling work, Eric is a key player in the development of Dart. With his extensive Smalltalk background he is a great person to be able to talk to a Smalltalk audience about Dart, the things that are inspired from Smalltalk, and the things it does differently." - Lars Bak, Google Inc.

In the previous post, I described how to use the vmc command-line tools to deploy a Ruby application to VMware’s cloud. In this post I will describe the steps I took to create and use a private cloud (based on the instructions found on github). If you have an account with Cloud Foundry, you can download a pre-built Micro Cloud Foundry, but by following these instructions you will have an environment similar to what I used in this project.

Creating a Micro Cloud Foundry

On a clean MacBook Pro running Snow Leopard (10.6.8), I  downloaded an ISO to install 64-bit Ubuntu Server 10.4.3, then I installed VMware Fusion 4.1.1. I launched Fusion and started the process to create a new virtual machine:

• Introduction: I clicked the “continue without disk” button.
• Installation Media: I chose the disk image for Ubuntu Server.
• Operating System: I left the defaults of Linux and Ubuntu 64-bit.
• Linux Easy Install: I accepted the default user and specified a password (‘swordfish’).
• Tools Download: I downloaded the tools.
• Finish: I accepted the defaults (1 GB RAM, 20 GB hard disk, NAT networking).
• Save As: I named the virtual machine ‘My Cloud’.
• A few minutes later the server started and gave a login prompt.

It seems that Fusion’s easy install of Ubuntu does not get the keyboard properly mapped, so I entered the following command:

sudo dpkg-reconfigure console-setup

I selected the ‘Dell 101-key PC’ keyboard (pressing the <D> key repeatedly since the arrow keys didn’t work), and accepted the other default settings. I then entered the following commands (based in part on the instructions here) to update and install a few needed packages:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install vim
sudo apt-get install openssh-server
sudo apt-get install curl

Next, I entered the command to setup this Ubuntu server as a VMware Cloud Application Platform (VCAP):

bash < <(curl -s -k -B \
 https://raw.github.com/cloudfoundry/vcap/master/dev_setup/bin/vcap_dev_setup)

This process took some time (an hour?) and when it finished it showed a message describing how to start Cloud Foundry:

~/cloudfoundry/vcap/dev_setup/bin/vcap_dev start

We now have the confusion of dealing with two machines (one real and one virtual) and so need to be careful to identify which one we mean in the subsequent instructions:

• Client: A MacBook Pro is my development machine, and a “client” with respect to “the cloud.” For this project, I interact with the client primarily using Terminal (using “a shell on the client”), but also using a web browser (Safari, Firefox, or Chrome) and the Finder.
• Server: VMware Cloud Application Platform (VCAP) runs on 64-bit Ubuntu in a Fusion virtual machine on the Mac, and provides “the cloud services” on a “Micro Cloud.” I interact with the server primarily using ssh in a Terminal session (“a shell on the server”), but occasionally using the console visible in Fusion. This micro cloud offers the same developer experience that a public cloud (such as appfog.com) should provide, so you can practice your deployment without making your application public.

In order to interact with the server using ssh in a Terminal session, we need to know the server’s IP address. On the server console, I entered the following command:

ifconfig eth0 | grep inet

The first line gives the IPv4 address (in my case, 192.168.10.128). On the client, I then entered the following command:

sudo vim /etc/hosts

to edit the client’s hosts file and add the following line:

192.168.10.128   mycloud

You should, of course, use the IP address you got from the server. Then, from a client shell, I entered the following to get a shell on the server:

ssh mycloud

I entered ‘yes’ to add the RSA key and then entered the password I defined when I built the server (‘swordfish’). To simplify my server commands, I added an alias using the following commands on the server:

echo "alias vcap='~/cloudfoundry/vcap/dev_setup/bin/vcap_dev \$1'" >> ~/.bashrc
source ~/.bashrc

On the server, I entered the following command:

vcap start

This gave me some debugging information, and ended with a list of services that all show as “RUNNING.”

Targeting the Micro Cloud

From the client we interact with a cloud using vmc, the command-line interface in a client shell. When we were using the public cloud, the “target” was api.cloudfoundry.com; with the micro cloud we use api.vcap.me as the target. VMware has registered the domain vcap.me and configured the DNS servers to map that domain (and all its subdomains) to 127.0.0.1 (or localhost). Of course, the micro cloud is not at 127.0.0.1 from the perspective of the client, so we need to establish a tunnel from port 80 on the client to port 80 on the server. First, we need to ensure that port 80 is not in use. On my Mac, I opened System Preferences, selected Sharing, and then made sure that ‘Web Sharing’ is unchecked. With port 80 available, I entered the following command in a client shell (replacing USER with the Ubuntu user defined during the initial setup):

sudo ssh -L 80:mycloud:80 USER@mycloud -N

This command establishes a tunnel from port 80 on the client to port 80 on the server, and the shell hangs until the tunnel is terminated (e.g., with <Ctrl>+<C>). To show that the cloud is there and responding, I entered http://api.vcap.me/info in a client web browser and observed the JSON data returned. In another client shell, I entered the following vmc commands (you should provide your own email!):

vmc target api.vcap.me
vmc info
vmc add-user --email jfoster@vmware.com --passwd swordfish
vmc info

With the micro cloud up and running, I tried deploying my trivial Ruby application (described here) using the following client commands:

cd ~/env
vmc push

In response to the questions, I named the application ‘jfoster-env’ and asked for 2 instances; otherwise I accepted the defaults. At this point (see notes below) I was able to see the application in a web browser at http://jfoster-env.vcap.me/ and http://jfoster-env.vcap.me/env and I tried the various vmc commands at the end of this post. The behavior was generally consistent with the public cloud.

Note 1: The first time I tried a push the response was “Error: Application [jfoster-env] failed to start, logs information below. Error 306: Error retrieving file ‘/logs/err.log.” I believe that this means that the application failed to start in time, and that when the system went to look for an error log, there wasn’t one. The application actually was running and responded to HTTP requests. Also, ‘vmc apps’ and ‘vmc stats jfoster-env’ showed the application running and ‘vmc restart jfoster-env’ reported success.

Note 2: The response to ‘vmc logs jfoster-env’ included “Error 306: Error retrieving file ‘logs/startup.log.” I found someone who said, “The file ‘logs/startup.log’ isn’t always generated, so the error returned by vmc can be a red herring.”

Conclusion

We have built an Ubuntu server and configured it as a private micro cloud. We used it as a target for our application and were able to use it in place of the public cloud. Next we will investigate adding a new runtime and framework to Cloud Foundry.

As you may be aware, VMware’s Cloud Foundry is an open source “Platform as a Service” (PaaS) on which you can “deploy and scale your application in seconds.” In anticipation of my presentation at STIC 2012, I have been learning about Cloud Foundry and this blog post describes my initial steps, using material from cloudfoundry.com and from github.com.

I started by registering for an account because I wanted to try deploying on VMware’s cloud (this step is not necessary for our later deployment on a private cloud). The confirmation email took several hours to come, and I’ve read of others who waited for days, so if you register don’t be surprised if it takes a while to hear back.

On a clean install of Snow Leopard (Mac 10.6), I started by creating a very simple Sinatra application in Ruby. (Of course, the goal of this project is to deploy Smalltalk to the cloud, but we will start with things that Cloud Foundry understands, and go from there!) In a Terminal, I entered the following commands (providing my password when prompted):

sudo gem install mime-types
sudo gem install rubyzip
sudo gem install uuidtools
sudo gem install sinatra
mkdir env; cd env

In ~/env/ I created a file, env.rb, consisting of the following text:

require 'rubygems'
require 'sinatra'

get '/' do
    host = ENV['VMC_APP_HOST']
    port = ENV['VMC_APP_PORT']
    "<h1>XXXXX Hello from the Cloud! via: #{host}:#{port}</h1>"
end

get '/env' do
    res = ''
    ENV.each do |k, v|
        res << "#{k}: #{v}<br/>"
    end
    res
end

Then, from the Terminal, I entered the following command:

ruby env.rb

This told me that Sinatra was running a web server on port 4567, so I opened a web browser on http://localhost:4567/env and verified that my application ran and returned the expected values. I used <Ctrl>+<C> in the Terminal to terminate the application.

The next task was to install the command-line tools (vmc) used to interact with Cloud Foundry. In the Terminal I entered the following command:

sudo gem install vmc --pre
The ‘–pre’ option tells ruby to install the pre-release version of vmc. Next, I specified the target cloud for deployment, I logged in using the email address and the password I got when I registered (see above), and I pushed the application to the cloud:
vmc target api.cloudfoundry.com
vmc login
vmc push
The push command prompts a series of questions. I accepted the default (Y) to deploy from the current directory. I gave a unique name for the application (‘jfoster-env’). I accepted the default to confirm that this is a Sinatra application and it can be deployed in 128M of RAM. I specified two (2) instances so I could see the application run on two machines. Finally, I accepted the default to confirm that I did not need any services and I did not need to save the configuration. The tools then proceeded to confirm that the application was pushed and started successfully. In a web browser, I went to the designated location, http://jfoster-env.cloudfoundry.com/, and found that my application was indeed running. I refreshed the page a few times and confirmed that the application was running on two machines (showing different internal IP addresses and ports).
I tried a number of commands to explore what was available through the vmc command-line tools:
vmc help
vmc user
vmc target
vmc info
vmc runtimes
vmc frameworks
vmc apps
vmc instances jfoster-env +2
vmc apps
vmc instances jfoster-env -2
vmc stats jfoster-env
vmc logs jfoster-env
vmc files jfoster-env
vmc files jfoster-env app
vmc files jfoster-env logs
vmc env jfoster-env
vmc stop jfoster-env
vmc apps
vmc delete jfoster-env
vmc apps
vmc logout
In all of this, we have been interacting with the VMware public cloud. The next blog post will explore creating and using a private development cloud.

         

Anatomy of an IDE

Using a few example IDEs we are going to look at what makes an IDE valuable.
Building software is a complex business, software that works and stays in production for years. It is a craft that involves engineering, insight and skill. The tools that we use to build that software are vital enablers to our success.
Between 1997-2004 the dominance of Java and the main vendors’ tools strategies led to something of a stagnation for IDEs. But since then with the return to language diversity and the broadening of platforms there has been a real opportunity to experiment with what an IDE is and means and to look at how it could evolve.
We will look at a range of IDEs including WebVelocity, Cloud9 and Codea and contrast them with more traditional IDEs such as VisualWorks, IntelliJ’s IDEA and Eclipse.
This evening will be more in the format of an overview of the issues and opportunities and then a discussion as I am sure that many of you have a far deeper understanding of many of these IDEs than I do.

Simply due to new work on my libHaru interface app I needed the newest version of libHaru. You may download version 2.1.0 for Windows from the libHaru site.

Building libHaru under Windows is time consuming – strange stuff and lots of people are asking for prebuild libraries under Windows.

I made a build for their 2.3.0 RC2 candidate (software date: 23.01.2012) and it can be downloaded from here.

• Riak allows defining multiple indices per object. DynamoDB has a more limited index model. 
  
• Riak allows storing objects larger than DynamoDB's 64 KB limit. This is particular important when you want to serialize a business object graph and store it as a BLOB in a single object/record in the database. With DynamoDB this approach won't (usually) work; 64 KB is too little to fit a serialized graph. You will be forced give up serializing, break up your entire graph and map it to multiple records. This means manual mapping between objects/database. If the domain is complex, I fear we enter another Vietnam of Software Development.
• DynamoDB only replicates between "availability zones", not "regions". Riak Enterprise allows for replicating between data centers in different regions, e.g. Europe and the US. With Riak you can have your data in Europe and the US to reduce latency for both regions.
• You can run Riak at your own hardware having full control of the data. DynamoDB is only run at Amazon's data centers.

1. Introduction to MorphoPhysics
2. Make a ball
3. A better ball
4. Add a rubber band
5. Rubber band with force

Introduction to MorphoPhysics:

Gofer it
  squeaksource: 'DaliotsPlayground';
  package: 'ConfigurationOfDaliotsPlayground';
  load.

(Smalltalk at: #ConfigurationOfDaliotsPlayground) 
    project lastVersion load: 'MorphoPhysics'.

Make a ball:

"Before creating the BallMorph class"
b1 := EllipseMorph new.
b1 openInWorld.
b1 extent: 40@40.
b1 color: Color green.
b1 borderWidth: 0.
b1 center: (158@225).
"..."
b1 delete.

"After creating the BallMorph class"
b1 := BallMorph new.
b1 openInWorld.
b1 reset.
b1 startStepping.
b1 velocity: 1@0.
b1 velocity: 0@1.
b1 velocity: 0@ -1.
b1 velocity: 1@ -1.
b1 velocity: 20@0.
b1 velocity: -20@0.
b1 acceleratedBy: -20@0.
b1 mass: 40.
b1 applyForceBy: -1000@0.

b2 := BallMorph new.
b2 openInWorld.
b2 mass: 80.
b1 applyForceBy: -1000@0.
b2 applyForceBy: -1000@0.

Gofer it
  squeaksource: 'Pharocasts';
  version: 'DemoMorphoPhysics-hjo.1'; 
  load.

A better ball:

Gofer it
 squeaksource: 'Pharocasts';
 version: 'DemoMorphoPhysics-hjo.2';
 load.

Add a rubber band:

"before creating the RubberBandMorph class"
r := PolygonMorph new.
r openInWorld.
r makeOpen.
r beStraightSegments.
r setVertices: {80@180. 180@200}.
r borderColor: Color black.
r setVertices: {b1 center. b2 center}.
r delete.

"after creating the RubberBandMorph class"
r := RubberBandMorph new.
r openInWorld.
r ball1: b1.
r ball1.
r ball2: b2.
r ball2.
r connectTwoBalls.
r length.
r vectorFrom1To2
Transcript open.
r thickness.

Gofer it
 squeaksource: 'Pharocasts';
 version: 'DemoMorphoPhysics-hjo.3';
 load.

Rubber band with force:

r stress.
r applyForceOppositeDirection: 100@0.
r unitVectorFrom1To2.
r applyForceOppositeDirection: (r stress * r unitVectorFrom1To2).
r applyForce.
r delete.

b1 delete.
b2 delete.

"A little more complex example"
b1 := BallMorph new.
b1 openInWorld.
b1 mass: 80.

b2 := BallMorph new.
b2 openInWorld.
b2 mass: 50.

b3 := BallMorph new.
b3 openInWorld.
b3 mass: 200.

r12 := RubberBandMorph new.
r12 ball1: b1.
r12 ball2: b2.
r12 openInWorld

r23 := RubberBandMorph new.
r23 ball1: b2.
r23 ball2: b3.
r23 openInWorld

r31 := RubberBandMorph new.
r31 ball1: b3.
r31 ball2: b1.
r31 openInWorld.

Gofer it
 squeaksource: 'Pharocasts';
 version: 'DemoMorphoPhysics-hjo.4';
 load.

Hello again,

I have started a new blog at http://simplphp.wordpress.com.

SIMPL/PHP is a complete “Smalltalk-like” online environment for developing Internet applications.

And I will keep this blog alive as well for discussing Smalltalk…

At the weekend 4th/5th of february there will be this years FOSDEM, a big free and open source conference held in Brussels. Some of the smalltalk guys arranged a smalltalk dev room on sunday, 5th of february.

If you are interested in smalltalk don't hesitate to visit us in the smalltalk dev room (room number seems to be AW1.126). If you have interests in smalltalk and REST then you are even more welcomed to visit my talk. It is the first in the schedule (I should have been more careful :) ).

Here is the schedule:

09:30 Norbert Hartl, Take a small REST. Simple approaches for REST in 
      smalltalk
10:00 Stephane Ducasse, Marcus Denker, The next steps for the Pharo Vision
11:00 Laurent Laffont, John Thornton, Amber, the Smalltalk for web developers
12:00 Nick Ager, An introduction to jQuery Mobile
12:30 Stefan Marr, RoarVM, Sly
13:00 David Chisnall, Compiling Smalltalk to fast native Code
13:30 Craig Latta, Spoon
14:00 Stephan Eggermont, Willem van den Ende, Diego Lont, Back to the future, 
      (re)learn smalltalk (till 16:30).

See you @ FOSDEM

Hi guys. During ESUG 2011, at the Awards, I was showing Fuel. The week before such event I was thinking what I could show to the people. This was a challenge because showing a serializer can be plain boring. I was working at home that afternoon, and suddenly I thought: “What happens if I try to serialize a living debugger and materialize it in another image?” After 5 minutes, really, you will see it takes only 5 minutes, I notice that such crazy idea was working OUT OF THE BOX. Even if I knew Fuel supported serialization of methods, contexts, closures, classes, etc…I was surprised that it worked from the first try. I was so happy that I tried to explain to my poor wife what I had just done hahahah. Unfortunately, she told me it was too abstract and that understanding the garbage collector was easier (I promise she really understands what the garbage collector does hahhahaha).

Well….several months has passed, but I would like to show you how to do it because I think it may be of help for real systems   So…the idea is the following: whenever there is an error, you can get the context from it, and such context is what is usually written down into a log file (in Pharo this is PharoDebug.log). I will show you two things: 1) how to serialize a debugger in one image and materialize it another one and; 2) how to write down the context into a Fuel file when there is an error so that you can materialize it later in another image.

Installing Fuel

The first step is, of course, install Fuel. The latest stable release is 1.7 but to have better results with this example, I would recommend 1.8. Fuel 1.8 is not released yet it is because we plan to write some stuff in the website. The code is almost finish, so you should load Fuel 1.8 beta1. In my case I am using a normal Pharo 1.3 image:

Gofer it
url: 'http://ss3.gemstone.com/ss/Fuel';
package: 'ConfigurationOfFuel';
load.
((Smalltalk at: #ConfigurationOfFuel) project version: '1.8-beta1') load.

Once you have finished loading Fuel, save the image. Let’s call it Fuel.image.

Serializing and materializing a debugger

Now its time to do something hacky in the image so that to open a debugger. Or you can just take a piece of code and debug it. In my example, I opened a workspace and wrote the following:

| a |
a := 'Hello Smalltalk hackers. The universal answer is '.
a := a , '42!'.
Transcript show: a.

Then I select the whole code, right click -> “debug it”. Then I do one “step over” and I stop there before the concatenation with ’42!’.

I am sure there could be better ways, but the simpler way I found to get the debugger instance for this example, is to do a Debugger allInstances first   so… be sure not to have another debugger opened hahaha.  Now…let’s serialize the debugger:

Smalltalk garbageCollect.
FLSerializer
serialize: Debugger allInstances first
toFileNamed: 'debugger.fuel'.

After that, you should have a ‘debugger.fuel’ created in the same directory where the image is. Now close your image (without saving it) and re open it. If everything is fine, we should be able to materialize our debugger and continue debugging. So, let’s try it:

| newDebugger |
newDebugger := FLMaterializer materializeFromFileNamed: 'debugger.fuel'.
newDebugger openFullMorphicLabel: 'Materialized debugger ;)'.

So????  Did it work??  are you as happy as me when I first saw it?   if you check this new opened debugger, you will see its state is correct. For example, the instVar ‘a’ has the correct state. You can now open a Transcript and continue with the debugger as if were the original one.

Of course that even if this simple example works, there are a lot of problems. But I will explain them at the end of the post.

Serializing and materializing errors

In the previous example we have serialized the debugger manually. But imagine the following: you have a production application running. There is an error, and PharoDebug.log is written with all the stack. The user/client send you by email the .log and you open your favorite text editor to try to understand what happened. Now imagine the following: you have a production application running. There is an error, and a PharoDebug.fuel is written with all the stack. The user/client send you by email the file and you open an image, and then materialize and open a debugger. How does it sound? magical?

For this example, we will just change the place where Pharo writes PharoDebug.log when there is an error. That method is #logError:inContext:. What we will do is to add just 2 lines at the beginning to serialize the context:

logError: errMsg inContext: aContext

" we should think about integrating a toothpick here someday"
FLSerializer
serialize: aContext
toFileNamed: 'PharoDebug.fuel'.

self logDuring: [:logger |
logger
nextPutAll: 'THERE_BE_DRAGONS_HERE'; cr;
nextPutAll: errMsg; cr.

aContext errorReportOn: logger.

"wks 9-09 - write some type of separator"
logger nextPutAll: (String new: 60 withAll: $- ); cr; cr.
]

Now yes, let’s execute something that causes an error. What I did is to evaluate 1/0. After that, you should see the file PharoDebug.fuel in the same directory where the image is. You can now close the image and reopen it. And then, let’s reopen de debugger:

| aContext |
aContext := FLMaterializer materializeFromFileNamed: 'PharoDebug.fuel'.
Debugger openContext: aContext label:  'This is the new debugger!' contents: nil

Et voilà! Hopefully that worked    Notice that in this example and the previous one, there is nothing in special with the Fuel serialization. You are using the normal API, and all you do is to serialize a debugger or a context as if you were serializing any normal object. Of course, you can also apply this idea to other places. For example, in Seaside you have an error handler. You may want to serialize the error with Fuel there.

Limitation and known problems

• Even if Fuel can fully serialize methods, classes, traits, etc., it is recommended that the image were the contexts/debuggers are serialized and materialized are equal. If you are doing this in a production application, then you can have the same image running locally. The idea is that both images have the same classes and methods installed. This is because, by default, if the object graph to serialize includes compiled methods, classes, class variables, etc., they are all considered as “globals”, which means that we only serialize its global name and then during materialization it is searched in Smalltalk globals. Hence, classes and methods have to be present. Otherwise you have to use Fuel in a way that it serializes classes as well, but that’s more complicated.
• There may be things that affects the debugger which are part of the image and not the serialization, and they may have changed. Imagine for example, a class variable which has changed its value in the image where you serialize. Then it will have a different value in the image where you materialize. Most of these problems also happens if even if you open the debugger later in the same image…some state may have changed…
• The graph reachable from the contexts can be very big. For example, Esteban Lorenzano was doing this for an application and one of the problems is that from the context it was reachable the whole UI…which means lots and lots of objects. In such a case, you can always use Fuel hooks to prune the object graph to serialize.
• Be aware to use exactly the same version of Fuel in both images

Conclusion

All in all, I think that as a very first step, it is very nice that we can serialize this kind of stuff like contexts and debuggers out of the box with Fuel. This could be the infrastructure for a lot of fancy stuff. I don’t think that the debugger materialization can be as reliable as if you were debugging in the original image. I don’t think either that it should replace PharoDebug.log. However, what I do think is that you can add the Fuel serialization just as another way of getting more information about the problem. It’s one more tool you can add to your Smalltalk toolbox