Home :: Download :: Documentation :: Contact :: Blog :: Bugs Top :: Bottom

The Blog

23 Jul 2006: Late binding

Recently one of Nemerle users played a little bit with our macro system and implemented a feature allowing dynamic invocation of methods or accessing properties in a quite clean way (as you probably know, Nemerle is entirely statically typed and forces your code to be compliant with .NET standard object-oriented access to classes). The simplest example is

def getLength (obj : object) late obj.Length System.Console.WriteLine (getLength ("Grey goo")) System.Console.WriteLine (getLength ([1,2,3])) System.Console.WriteLine (getLength (System.Windows.Forms.LinkArea ()))

Here we just want a method to fetch the Length property of given object and late keyword introduced by macro allows us to do so easily. The biggest advantage here is that we do not care about the intefaces and common supertypes of object we use this on, so we can apply this method to ANY object (which is assumed to contain the correct property). The call will be made using .NET's dynamic invocation feature.

The idea is a little bit similar to something called duck typing (though the name is a bit politically controversial in my country at the moment ;p), but in case of our macros it works in a different way. We do not introduce any dummy "types" for objects, which are not type-checked, but we allow user to specify which parts of code should be dynamic (for more detailed specification see a deciated page).

Now the question is why is it really useful? In general I'm always heading towards the perfect world and I prefer my programs to be proved being perfect and safe. Type inference is a powerful Nemerle's feature, which helps here - I do not need to specify types of variables and compiler can infer what I mean; if it can't, it usually means I made some mistake. But here comes the problem, sometimes the code we write is simply not compatible with language / platform type system and compiler complains about my code even though I know it is right. Let us consider another example:

#pragma indent using Nemerle.Late public class Pixel mutable shift = 0 public Draw () : void System.Console.Write (string (' ', shift)) System.Console.Write ("*") public Move (dx : int) : void shift += dx public class Triangle public Draw () : void System.Console.Write ("/^\\") def perform (x : object, with_move = false) late x.Draw () when (with_move) x.Move (5) x.Draw () System.Console.WriteLine () perform (Pixel (), true) perform (Triangle ())

Now as you can see my perform function is quite tricky - it does not call Move on object if I do not specify with_move parameter to be true. So its contract is something like "calls Draw, but if we want move also calls Move", which is not expressible in any simple type system (e.g. .NET). With our late binding macro we just ignore the type system and perform all calls dynamically. :)

One more thing to notice here is that usually we can avoid dynamic calls by using well designed object hierarchy and interfaces (if we are not too lazy of course ;) ). In our example we can easily make the call to Draw statically safe, by introducing the IDrawable interface.

public interface IDrawable Draw () : void public class Pixel : IDrawable mutable shift = 0 public Draw () : void System.Console.Write (string (' ', shift)) System.Console.Write ("*") public Move (dx : int) : void shift += dx public class Triangle : IDrawable public Draw () : void System.Console.Write ("/^\\") def perform (x, with_move = false) x.Draw () when (with_move) late x.Move (5) x.Draw () System.Console.WriteLine ()

Note that now type inference comes in help - the signature perform (x : IDrawable, with_move : bool) : void is automatically guessed by compiler. The only late bound operation now is a call to Move. To solve this we could do two things:

The other cases when late binding might be useful is when you use a platform's feature, which is very dynamic by nature - like using types obtained through COM.

16 May 2006: Nemerle 0.9.3 hits The Outside World

We now require mono 1.1.13 or newer (or MS.NET 2.0 as usual).

Library changes:

Macro library changes:

Backward incompatible library changes:

Fixed issues from BTS:

24 Jan 2006: Nemerle 0.9.2 in the wild

This version brings a bunch of new features and of course several bugfixes. We now require mono 1.1.11+ or MS.NET 2.0. About 180 svn commits were done since the last release.

New language features:

Macros:

Library:

Other:

Bugfixes:

And from mantis:

21 Dec 2005: Recent changes

Yield

Supporting yield was harder than it first seemed. The problem was that Nemerle already supports most of the stuff needed for yield (for example putting local variables in classes so they persist), but unfortunately in a slightly different way, so a reimplementation was required in a few cases. This was very irritating (as doing the same thing twice always is). Fortunately now it works.

There was another problem with finally blocks which has to be run only once (we don't want finally blocks to be run, when yielding a value from inside try), and also when the iterator is prematurely disposed. Fortunately C# doesn't allow yield in try-catch, so we decided to do the same ;)

The C# spec was quite sketchy about this issue though.

List comprehensions

Haskell (as well as Python to some extent) has this really nice nice notation, that comes from math. The idea is to describe a set in terms of all such x that x comes from the set A and is greater than zero. This would be written as:

$ [ x | x in A, x > 0 ]

But this isn't really funny, so let's look at something more complicated, all such pairs (x, y) that x comes from A and y comes from B:

$ [ (x, y) | x in A, y in B ]

We can now further restrict this to x < y:

$ [ (x, y) | x in A, y in B, x < y ]

We can even apply some complex expressions, for example to return set of lengths of elements in A we would use:

$ [ x.Length | x in A ]

There are functions to do all that stuff in Nemerle standard library, but list comprehensions provide a way to combine them in one short expressions. The translation of this stuff to the underling core language is straightforward -- each e in something is translated to a nested foreach loop and each condition is translated to when, details are here.

For example to list all members in all types in all assemblies used by the program one could use:

def allMembers = $[m | a in System.AppDomain.CurrentDomain.GetAssemblies (), t in a.GetTypes (), m in t.GetMembers ()];

To limit this to types from the "System" namespace we could use:

def systemMembers = $[m | a in System.AppDomain.CurrentDomain.GetAssemblies (), t in a.GetTypes (), t.Namespace == "System", m in t.GetMembers ()];

And so on. Note that the following statement would have similar effect:

def systemMembers = $[m | a in System.AppDomain.CurrentDomain.GetAssemblies (), t in a.GetTypes (), m in t.GetMembers (), t.Namespace == "System"];

But it would be less efficient because of calling GetMembers even for types outside System, and only discarding them later.

This is really a nice feature and only one hour to implement :-)

What should come next is something like:

$[(x,y) | x in [1...3], y in [1,3,...,15]]

04 Nov 2005: Nemerle 0.9.1 is out

This release brings the long-awaited indentation syntax and a few bugfixes.

About 150 SVN commits were made since the last release.

Availability

As usual we provide a source code release and a number of binary packages. Starting with this release we provide a generic Unix binary installer, similar to the one Mono has (but text-mode only). RPM and MSI packages are also ready. DEB package is on its way.

Language changes

With the -i option compiler can now recognize indentation based syntax. More info at the wiki page.

Additions

The library

Bugfixes

02 Oct 2005: The Nemerle course for OOP programmers starting on Monday

We'll be lunching the course tomorrow. You have the last chance to subscribe! More info...

And from the other news, I have just prepared a binary installer, much like the one mono has, for people not willing to compile. Step by step guide to get Nemerle compiler running has all the details.

14 Sep 2005: Nemerle 0.9.0 is out

C# 4.0 today!

You read about C# 3.0 and are willing to see how could C# 4.0 look like? More powerful type inference? AST manipulation raised to its limits? Full generics support? Try Nemerle!

In general

The biggest change in this version is switch to the .NET 2.0 assemblies. The compiler now generates code using runtime generics when parametric polymorphism is used in Nemerle. While the language was designed with this switch in mind since the very beginning, the constantly changing and/or incomplete specifications forced us to make several changes to language semantics in this release.

The intention behind the 0.9 version number is that we're now very close to the 1.0 stable release.

We now require either Mono 1.1.9 or MS .NET Aug 2005 CTP. There are still several very serious issues with MS .NET S.R.E. API which may prevent certain features from working. Under mono there are problems with generic type serialization.

The performance of the generic code vary. Mono folks didn't implement shared code yet, which means generic code is JITed for each instantiation. This isn't that bad as it first look though, after some tweaks that are already in the Mono 1.1.9 the performance is comparable to the non-generic version.

In addition we should generate slightly better code overall with this version. The changes are mostly cosmetic though.

There are some breaking changes in standard library API, because we dropped our implementation of some generic classes in favor of .NET library classes. Most previously existing classes are still available though, but they are now subtypes of their BCL counterparts.

About 500 SVN commits was made since the last release.

Availability

As usual we provide a source code release and a number of binary packages. Starting with this release we provide a single noarch RPM package that should work with all architectures assuming mono is installed in /usr. MSI package is ready, DEB package is on its way.

Warning: Since 20:00 yesterday UTC till 11:00 today the 0.9.0 packages were broken. They contained a wrong version of antlr.runtime.dll. This is now fixed.

For details consult the downloads page.

Language changes

New features

Other stuff

Bugfixes

06 Sep 2005: Aug CTP and Mono 1.1.9, completion and forums

Generics status

The good news is that we're able to bootstrap Nemerle compiler with both Mono SVN (thanks to Martin, no patches required anymore!) and Microsoft .NET Aug CTP. There are several limitations in their S.R.E. though, most notably we're unable to emit explicit generic interface implementations on Microsoft platform. Our runtime issues page has all the details.

Unfortunately while MS.NET is able to run Mono produced executables (with some minor glitches), Mono cannot run MS.NET produced ones (also reported). Probably metadata format was changed.

But hey! In general everything seems to go in the right direction :-) The generic specifier and nested generic types issues are now solved. I also took some time to implement wanted features (like default parameters on local functions and partial application++, #pragma warning and with-matching see the NEWS file for details).

Completion engine

Alejandro Serrano is working since some time on C#-usable completion engine. This means it will be possible to link Nemerle.Compiler.dll to a C# application and get code completion services. This is good news for IDE integration.

The engine is now capable of producing tree of all types and members within given set of source files (so you can say display it to the user, make it clickable (go to definition) etc). Some basic code completion is also supported, thanks to support directly in the compiler. It can now complete static and instance members as well as type names. It cannot yet deal with namespaces and local variables.

One way or another, real IDE with completion is coming (be it #D and/or MD).

Forums and online course

We've set up a phpBB forum. It can be used for any kind of Nemerle-related communication. In particular, we would like to use it for free, online Nemerle course coordination. Our irc 2 www gateway is also going to help with course communication.

Michal

08 Aug 2005: Generics branch merged into trunk

We have just fixed last regressions existing in generics branch and decided to move it into the main development branch. So it happened, generics are now supported on trunk.

A short notes about current status:

Other problems with generics should be reported as bugs - in general we would like to support all scenarios you could use in C# and much more (especially in case of type inference).

Short list of features:

The bad news is that we were unable to get compiler working on MS.NET 2.0 (neither Beta 2 nor July CTP). There are numerous bugs, which we regularly report and sometimes manage to workaround. Currently building Nemerle.dll causes various exception stack traces in runtime, but when we copy it from boot/ then rest of the compiler builds fine. So you won't be able to use trunk with MS.NET until (hopefully) the next public release (I guess in September). Also with mono, you will need to use svn version (AFAIK the 1.1.9 release is not going out very soon, but probably it is a matter of month). Sorry for inconvenience.

For anybody, who is unable to run trunk version we have branched out the non-generic version into

http://nemerle.org/svn/nemerle/branches/nongeneric

you can also stay with trunk revision 5546

svn up -r 5546 http://nemerle.org/svn/nemerle/trunk

01 Jul 2005: It boots!

After exactly a month, since we have created generics branch, the Nemerle compiler finally can compile itself generating working generic code. It was harder than we initially thought due to numerous issues in both Mono and MS.NET runtimes. Mono had more issues, but the feedback loop was much shorter, Martin was very quick to fix problems we have found, thank you! #75429 was the last one preventing full bootstrap.

I was somewhat disappointed by poor performance of the generic code. Nemerle uses a lot of generic containers and switching from cast-everything-to-object (type erasure) to real generic code brought us about 10x slowdown. On small example (the standard library) the non-generic code uses 6.4s vs 50s, on a larger one (the main compiler library) it is 41s vs 405s. This is all on amd64 (1.8Ghz).

We will be doing some micro benchmarks shortly, but as I understand it today mono creates a separate copy of machine code for each instantiation of generic type or generic method. Which seems to be a problem with heavily polymorphic code, like the one we use.

But hey, it works! :-)

01 Jun 2005: Nemerle 0.3.2 released!

This version brings a few new features and a bunch of bugfixes. About 350 svn commits was made to the svn repository since the last release.

New features:

02 May 2005: Nemerle 0.3.1 released!

This version is a quick fix for issues reported with the 0.3.0 release.

Bugfixes:

There is a single nice feature that sneaked into this release, it is now possible to match inside the foreach loop directly (#436), that is:

foreach (op in ops) { | Op.A => ... | Op.B => ... } will now work as: foreach (op in ops) { match (op) { | Op.A => ... | Op.B => ... } }

29 Apr 2005: Nemerle 0.3.0 released!

This is the long awaited second milestone release (after 0.2.0 -- CLS consumer) with a new typing engine and a new parser.

About 300 svn commits has been made since the last release.

Additions:

Incompatible changes:

Library:

Fixed bugs from our bugtracker:

24 Apr 2005: Generics progress

I have been experimenting with adding generics support in Nemerle code emission engine. One could think that this should be an easy task in compiler already having parametric polymorphism. This is partially true, I don't have to touch typing engine too much, most of the information is already there. But on the other hand, there are still some blocking issues inside the compiler (informations which are not stored in program tree, because they were not used till now), and what is much more pain, in .NET frameworks we are using.

People are quite excited with lately released VS 2005 Beta2, but from the compiler perspective it doesn't bring too much. Literally all the bugs we have reported all over the last half year are still not fixed. That's ok, frameworks have bugs, in many they cases can be workarounded. But when we talk about generics... well, would you expect ToString or Equals methods to throw NotSupportedException? This is why I finally decided to move my experiments to Mono.

Of course live isn't easy here too. First I had to write methods recently added in MS.NET Beta2 in Mono BCL (unfortunately they are just quick hacks for my experiments, I didn't dig into mono S.R.E internals to create fully functional implementation). They are used to obtain members from instantiated generic type basing on member builders coming from noninstantiated type. For example consider

class G ['a] { public this (x : 'a) { } public foo (x : 'a) : list ['a] { } } ... def x = G (1); x.foo (2);

During compilation we have TypeBuilder representing G and MethodBuilder representing foo, we then want to obtain MethodInfo for G [int].foo, so we use TypeBuilder.GetMethod (instanciated, foo_method_builder). And with this thing everything seemed to go right with Mono, until I came into this. Well, I hope this won't be hard to fix, now I must deal with other issues inside Nemerle compiler anyways.

Summarizing, the smell of "fresh stuff" is still near generics in .NET, especially when we come into S.R.Emit API. But even with these issues I was able to make some progress. My current play-field already contains interesting stuff. Especially we can observe immediate advantage of using Nemerle type inference in working generic code like

def x = System.Collections.Generic.List (); x.Add ("Bla"); assert (x[0] == "Bla");

compared to C#'s

System.Collections.Generic.List<string> x = System.Collections.Generic.List <string> (); x.Add ("Bla"); assert (x[0] == "Bla");

Time will show how my experiment will progress into fully working solution. There is much chance that Nemerle will be the first bootstrapping compiler extensively using S.R.Emit to generate and utilize generics in the world (AFAIK gmcs isn't using generics for its own bootstrap).

BTW: We have a little Language Poll. Please stop by and give us some feedback about the tricky design decisions.

--Kamil

06 Apr 2005: New web site

We've been quite busy lately migrating content from our old XML-generated webpage to a wiki solution. It is using Media Wiki (yeah, the same as in wikipedia and mono-project sites). It seems to work quite well.

The silent hope behind this idea is to make it easier to publish some learning materials about Nemerle and fix/extend the existing, so external contributors can do it.

One of my personal fears was that it is going to be inefficient. Fortunately with eAccelarator it seems to work quite well (it can now handle about 15 hits per second which is well above the typical /. attack level :-).

Another thing was the inability to make a hard copy of several related pages, like some bigger tutorials. However after little perl voodoo we rip the content from the wiki to HTML, later to TeX and finally to PDF.

One thing remains to be moved -- it's the language reference manual. It contains grammar description which I have no idea how to put into the wiki... (other than simply putting it in <pre>).

--Michal

31 Mar 2005: Nemerle 0.2.10 released!

This is another preview before 0.3.0. We have fixed a handful of bugs and decided to give it another shot.

About 100 svn commits has been made since the last release.

Additions:

Bugfixes:

Fixed bugs from our bugtracker:

22 Mar 2005: Nemerle 0.2.9 released!

This is preview release before 0.3.0, which is real soon now. There are lots of changes in this version -- the parser and the typer (which constitute more then half of the compiler) have been replaced by entirely new implementations.

There is a number of backward incompatible changes in this release. Most of them were previously discussed on the mailing list, and received rather good feedback other only generate warnings in this release. We apologize for both. We hope they make Nemerle a better language. On the plus side -- we should be now far closer to certain language stabilization point.

0.3.0 should bring implicit conversions, List iterators as list[T] methods and maybe some more goodies.

Incompatible language changes:

Language additions:

Library changes:

Macro library changes:

Other stuff:

13 Mar 2005: Intersection types

This is RFC. There is a problem with new type inference engine and intersection types. Or maybe intersection types in general. But, from the beginning.

The type inference algorithm works by assigning type variables, that are yet-uninitialized types, to various parts of the program and trying to deduce something about the type of the program. For example:

class C { static foo (x : int) : string { ... } static bar[T] (x : T) : list[T] { ... } static Main () : void { def x = 3; // easy, x has type int def y = foo (x); // again easy, y has type string def z = bar (y); // harder -- z has type list[type_of y], // so it has type list[string] ... } }

Now consider lists:

class list {...} class Cons : list {...} // head and tail class Nil : list {...} // empty list

This is mostly how the lists look in Nemerle -- two subclasses of a list class. Now, let's have a look at how the list is constructed (the new keyword is skipped in Nemerle):

def empty = Nil (); def my_list = Cons (some_elem, empty);

There is no problem -- empty gets type Nil and Cons takes list as a second argument. But Nil is subtype of list, so the call to the Cons constructor is happily typed, and my_list gets type Cons.

But lets tell, we want to reuse variable first for empty list, and then for a non-empty one:

mutable the_list = Nil (); the_list = Cons (some_elem, the_list);

In the first line the_list gets type Nil. Now in the second line, the call to Cons is typed without a problem, like in the previous example. However it results in Cons type. And type Cons cannot be assigned to a cell of type Nil. The code like the above is quite common in Nemerle -- the syntax a little bit different:

mutable the_list = []; the_list = some_elem :: the_list;

But the meaning is the same. What can the user do is to explicitly upcast the_list to type list and everything is OK. But upcasts are supposed to be automatic. The previous inference algorithm had an hack here -- the constructors Cons and Nil had type list, so the upcast wasn't needed. But we sometimes want to use full Cons type, not the poorer list version. Therefore the new algorithm was supposed to lift this restriction, and do something about it.

The actual solution is to assign a special type at-most-Nil to the_list. Then, upon assignment, the type is changed to biggest common supertype of Nil and Cons, that is list. And everyone is happy -- we can use Nil() with its full Nil type and assignments work OK.

However there is a problem. Consider:

mutable x = 3; x = "string";

First off, this is most likely a bug. If the smallest common supertype of string and int were object it wouldn't be that big of a problem -- even now we just forbid two types, different than the object itself, to sum up to object -- because it is a bug in more than 50% of cases. However beside object they share three interfaces (IComparable, IFormattable, IConvertible). So the smallest common supertype of int and string is intersection of types IComparable, IFormattable and IConvertible. This is how NTE currently handles it. But we clearly (?) don't want this intersection type here. We want an error message.

So now the RFC -- my idea was to drop this beautiful intersection types -- that is if we want to sum up two types that have more than one most specific common supertype (like in the case above) -- to bomb out with an error. This is what constraint solver in Generic Java does. Generic C# doesn't have any constraint solver I'm aware of. The example with list would still work of course.

-- Michal

06 Mar 2005: Dots, vlists and matching

If you want my advice -- never write a metric truck load of code to debug it later. I'm still trying to debug the new type inference engine I wrote. 18 out of 60 dots of Nemerle.Compiler.dll are eaten :-) (ncc displays progress bar consisting of dots, that are replaced by underscores). This is getting annoying.

But, from the good news, some time ago we were discussing idea of adding new stuff to existing library classes. For example you have the System.String class and want the Nemerle.Utility.NString.Split static method, that works on lists and not on arrays, to be one of the regular String.Split overloads. We're not interested in really extending the class -- some syntactic sugar should be enough.

This seems doable, it is still in the design stage. The idea come back when I read this discussion on the ocaml-lib-devel mailing list. It was about the implementation of the VList data structure, that is another storage architecture for immutable ML-like lists. The benchmarks are quite promising -- it is much faster than the regular list for most operations. And taking into account that OCaml has a very good, precise garbage collector -- it gets even more interesting.

So the idea was to replace regular lists in Nemerle with vlists, and see what happens. Converting lists constructors isn't very hard, just a small compiler hack. The hard part is matching. Matching on lists in Nemerle looks like this (you can use the :: syntactic sugar that is really used in the sources in comments):

match (some_list) { | list.Cons (x, xs) => // x :: xs do_something_for_head (x); do_something_for_the_rest_of_the_list (xs); | list.Nil => // [] do_something_else () }

while the list data type definition looks like this:

variant list [T] { | Cons { head : T; tail : list [T]; } | Nil }

So the idea Kamil and I come up to was to annotate VList like this:

class VList [T] { public enum State { | Cons | Nil } [MatchOnThis] public TheState : State { get { ... } } [FieldInOption ("Cons")] public Head : T { get { ... } } [FieldInOption ("Cons")] public Tail : VList [T] { get { ... } } // the real implementation of vlists... }

The [MatchOnThis] macro would produce VList.Cons and VList.Nil bogus members, while the [FieldInOption] macros would add specific members to them. So the macthing like:

match (some_list) { | VList.Cons (x, xs) => do_something_for_head (x); do_something_for_the_rest_of_the_list (xs); | VList.Nil => do_something_else () }

would be possible. What would remain then would be changing the builtin [...] and :: literals to reference VList instead of list.

The idea is somehow broader. For example we could add matching to XML elements this way. Now we need the extending-existing-class part.

extend class XmlNode { [MatchOnThis] public NodeType : XmlNodeType {} [FieldInOption ("Attribute")] [FieldInOption ("Element")] // ... public Name : string {} [FieldInOption ("Attribute")] [FieldInOption ("CDATA")] // ... public Value : string {} // ... } match (some_xml) { | XmlNode.Attribute (name, value) => ... | _ => ... }

Anyway, just an idea ;-)

-- Michal

17 Feb 2005: Porting tests from C# to Nemerle

I have been playing a little bit with our C# to Nemerle code converter. The tool created by Bartek is still under development. It sometimes produces code, which needs some more tweaking to compile silently with ncc, but it is already quite powerful.

As a life test I tried converting positive test-cases from Mono's C# compiler test-suite. I took quite behavioral strategy: do not touch neither original C# code nor produced Nemerle programs, just test. I was wondering how much I can get with this and today I reached the number of 200 working cases from over 600 original ones. Note that they are probably not the best examples on HOW to program in Nemerle. ;-)

Quite nice I would say, taking into account that there are some features, which we do not support (unsafe code, iterators utilizing yield, etc.), unstable state of the converter and the fact that test-cases are mostly malicious programs, which used to crash mcs. The most problematic currently are the tests with implicit conversions, which will be supported in Nemerle only with the new typing engine, which Michal is currently working on.

The problem even with working cases was that our testing program does some more checks when executing regression tests. I had to provide output for many of them and mark / fix warnings issued by Nemerle compiler (mostly about unused variables). But at the end it was fun to see a few more hundreds of 'Testing blabla.n...verify...run...passed' lines during our nightly builds. One nice thing about compilers is that they are text to text tools and are quite straightforward to test (given that you have good test-cases).

The tool will be included in our next release. Of course it's better to write Nemerle code from the very beginning using all its unique features. But in case you are not familiar with the new language and have some medium sized application in C#, it can help you dive into functional programming more easily.

Cleaning meta-data information

One of the issues I have found annoying during this work was that we emitted much unnecessary meta-data information to the assembly (it posed some problems in custom attributes tests). So what and why was that?

You must remember that Nemerle has features not always fitting into .NET model and that it supports parametric polymorphism (aka generics) on 1.1 .NET framework (like Java 1.5 on its generics-lacking runtime). So, they can live safely within compiler, by to be able to create and load Nemerle assemblies without loosing information about special types and generics we have to encode them somehow in assemblies.

.NET custom attributes come very handy to do this. For example if we have

variant Tree { | Leaf | Node { left : Tree; value : int; right : Tree } } foo (l : list [Tree], x : int -> int) : int * Tree {...}

we encode them to something like:

[VariantType] class Tree { [VariantOptionType] class Leaf : Tree { } ... } [MemberType ("list(Tree) * (int -> int) -> int * Tree") foo (x : list, x : Func1) : Tuple2 { ... }

This way we can easily recreate what Tree is and what is the type of foo function after saving and loading of code to assembly.

Now, what was the original problem with polluting meta-data? We were emitting those special attributes for every member of every class. So even if some field had type int, which is simply expressible by System.Int32 we produced some extra info to be consistent. Optimizing those cases resulted in some performance boost and a very nice thing - Nemerle produced assemblies are now basically not distinguishable from csc or mcs generated ones, if you do not use any Nemerle specific types.

After we implement the usage of runtime generics, this will be the very common case. For example functional type 'a -> 'b will be translated to Func1 <'a, 'b>. This is what I understand by multilingual platform (ignoring many of its tweaks, about which we might tell later) ;-)

-- Kamil

27 Jan 2005: Comments ;-)

I took some time and set up a simple comment system. You just need to go to a specific blog entry (in archive/), and there will be a list of comments along with a form to post. It was a nice exercise in Nemerle programming.

From the other news, lupus did some mambo jumbo in mono to get our testcase 4x as fast as it used to be. Impressive! If only the other implementation was as fast... I guess I can now report this as a performance issue with a nice argument in hand.

My girlfriend has a birthday today. I also will in 3 days. Well, yet another year and I'll be closer to 50 than to day of my birth (it sounds so sad and serious ;-)

And this Garden State soundtrack is really great!

-- Michal

26 Jan 2005: What I think about exceptions performance

One of the rules the .NET was designed with in mind was that exceptions are only for exceptional situations. Therefore performance doesn't matter. The same thing can be noticed with delegates.

But the above assumption is a piece of crap. It is quite common to break the loop using exceptions in functional languages. Exceptions are treated there as a form of structuring program. Nobody would do it in .NET.

However there are cases when using exceptions would simplify program structure a lot, and I still cannot do it, because exceptions are soooooo slow.

I've been writing my MSc thesis right (Let S be a partial preorder on type variables...), I mentioned it some time ago here. I was clearly wrong assuming I've got everything done ;)

Anyway in addition to the theoretical background, I want to implement it. Until now I've created about 150k of code without even compiling it, which doesn't seem a good thing, but I cannot figure out a way to test pieces separately. Now what does it all have in common with exceptions?

Well, in the new typing engine it will be possible to save entire state on the stack, type some expression, inspect it, discard and pop state from the stack. All side-effect free. It can be usable for macros. It also simplifies things in many places -- for example to see what 'x' is we need to check for 'x' (a local), 'this.x', 'System.x', 'System.Foobar.x' and so on. This is easily done using the above mechanism (save, try typing, see if there were an error, restore and if there were no error run typing again).

But the most important point where it is used is overload resolution -- we need to try several possible resolution of the symbol being called.

Now, handling errors (user errors in programs) as exceptions in compilers is quite reasonable -- you can catch them for example in method compilation procedure and get just-one-error-in-a-method behavior, you can catch them somewhere else. And in general there is no performance issue here -- if any single throw will result in an error message, then there is no performance problem, exceptions aren't that slow.

But now, when we have this stacked typing, it is common to have ,,user errors'' in normal execution paths. In overload resolution it is common to just one of several possibilities to be error-free. So we cannot use exceptions in overload resolution. And if we cannot use exceptions in overload resolution, this is as bad as we couldn't have used them at all. We're back in the good ol' C model of checking return values for errors. That's why I don't think that exceptions should be that slow.

What does it mean ,,that'' slow? Well under mono/amd64 throwing and catching an exceptions takes about 18k cycles. It's quite similar on mono/x86. The other implementation I tasted wasn't better. All this means, that using my super hiper turbo fast machine I'm able to throw 100k exceptions per second. So using it for any other purposes then error reporting and cannot happen kind of situation is a performance suicide.

And here is the piece of (C#, hehe) code that I used for the tests.

using System; class M { int counter; Exception e; void throwing_foo (int x) { if (x % 3 == 0) throw e; } void testfoo (int x) { try { throwing_foo(x); } catch (Exception e) { counter++; } } public static void Main() { M m = new M(); m.e = new Exception(); for (int i = 0; i &lt; 1000000; i++) m.testfoo(i); Console.WriteLine(m.counter); } }

16 Dec 2004: Debugging support for Nemerle and other news

We were quite silent lately, but this is because we have many Nemerle parts under much rework. To name a few, we are currently developing a C# to Nemerle converter (already transforms and successully runs over 100 testcases from Mono's mcs testsuite), interactive interpreter, plugin for MS Visual Studio, debugging support for Nemerle compiled assemblies and fixing many bugs. Michal is working on the New Typing Engine, which will bring powerful type inference improvements to the compiler.

I (Kamil) was focusing mostly on bugs and reworking of compiler's API. In a meantime I tried adding debugging support in code generation. It was much simpler than I first thought about - just a few calls of ISymbolWriter during System.Reflection.Emiting the assembly. And here it goes:

debugging

With the svn version of compiler you are able to generate debuggable assemblies, step through code and inspect memory using MS CLR Debugger. Of course this work is in early stage and there are many problems. Locations are not yet stored correctly through entire compilation trees, so from time to time the "yellow" selection will go to some crazy places, but it will be hopefully easy to fix everywhere.

Debug support not yet works on Mono, but we are having the discussion with Mono developers and probably it will work here also soon.

I was also developing MS Visual Studio plugin for Nemerle using VSIP. It will be nice to connect all the debugging stuff with

visual studio

Visual Studio, but I was not able to find out how to do this (any help about developing in managed VSIP would be nice). The potential is great, code completion and class view are the features we really looking forward to. We will probably create some code completion engine, so our MonoDevelop bindings could get it also.

Keep up to date with our latest source code snapshots. We will probably wait a little bit more before the next release, because many of the new features are not yet complete. We are already in "post-beta" phase and we want to release high quality stuff.

18 Sep 2004: Nemerle 0.2.1 released!

This is a bugfix release to quickly fix issues found in 0.2.0.

12 Sep 2004: Nemerle 0.2.0 released!

Last minute note: if you want to rebuild the compiler from sources (and not only install it), in addition to nemerle-0.2.0.tar.gz you will also need nemerle-keys-0.2.0.tar.gz. This is due an error in the release process.

This version makes the Nemerle language full CLS consumer and producer. A number of non-CLS features are also in place, for C# compatibility, but we do not yet support CLSCompliant attribute checking. Any lack in CLS compliance is a bug now.

The language:

The library:

The compiler:

Other stuff:

07 Sep 2004: Type inference reloaded

I (Michal) moved to a new flat, and I had no Internet connection for a week. Which forced me to work on my Master's thesis about type system in Nemerle. Eh... I guess a week more, and it would have been finished :-)

I guess I solved all the problems (in theory, the implementation will be another matter). There are a few new cases it is going to handle. The main advantage is that it will postpone overload resolution for later, if it cannot be solved in the current point. Overloads come from two sources -- overloaded methods and field access. For example the new engine is going to grasp:

class B { foo : int; }; class C { foo : int; }; def _ = List.Map ([B()], fun (x) { x.foo });

as well as:

List.Sort (["foo", "bar"], String.Compare);

The current engine chokes on the first example, as when it types fun (x) { x.foo } it doesn't know the type of x (this field can be in several classes, but the current engine doesn't even check that, this has been reported as a bug several times). New engine will leave fun (x) { x.foo } partially typed, and return to it, when the type of x is known.

Second example causes problems, because there are two String.Compare, the second version taking culture argument (which we don't care about). The problem is similar as with the first example -- it does not know which version to use, until after it can see it's used for the List.Sort function, which is too late. The solution is exactly the same.

New algorithm will proceed by collecting typing constraints and trying to solve them. The last part seemed complicated to me, but it turned out to be quite simple (due to a simplistic nature of subtyping in .NET generics).

Now I've got a problem. There is nothing scientifically interesting in this approach. It's too simple. The most important advantage is that a/ it will be implemented, and b/ it will work. But this is no science. What remains is to cite Paul Graham: But for the hackers this label [computer science] is a problem. If what they're doing is called science, it makes them feel they ought to be acting scientific. So instead of doing what they really want to do, which is to design beautiful software, hackers in universities and research labs feel they ought to be writing research papers. I feel there is lots of truth in this essay, but I don't agree with some things there.

30 Aug 2004: Exceptional syntax and the need of a better parser

Suppose you want to collect the contents of some text files, but you want to tollerate any exceptions thrown in the File.OpenText method. Piece of cake, isn't it, you start XEmacs and type:

using System; class CollectFiles { public static Collect (file_names : list <string>) : string { | [] => "" | file_name :: rest => def file = try { File.OpenText (file_name) } catch { _ : Exception => null } if (file != null) { def s = f.ReadToEnd (); f.Close (); s + Collect (rest) } else Collect (rest) } }

But wait. Is this code ellegant? I don't think so, sir! Fortunately for us, some people came up with a better language construct to handle such situations:

| file_name :: rest => def f = File.OpenText (file_name) unless { _ : Exception => Collect (rest) } def s = f.ReadToEnd (); f.Close (); s + Collect (rest)

Much more readable, in my (Pawel's) opinion. Doesn't it look a little bit like Perl's `expr || die'? :)

It comes at a price though. The Nemerle parser can't handle such a construct on its own, we would have to clutter the parse tree with an additional node for the `unless' statement and expand it during typing -- and that's something I would like to avoid for a number of reasons. I guess this feature will have to wait until we develop the new parser supporting rewriting-based syntax extensions. I hope this will be the first thing we jump on after the 0.2 release.

Your comments are welcome!. Some of us (Michal :-), are not perfectly happy with the syntax above, your opinion does matter!

28 Aug 2004: 0.2.0 and new type inference engine approaching

Summer voyages has ended and we're working hard before 0.2.0 release. A few high priority bugs remain open. We're going to fix them before 0.2.0. The aim of this release is to get better CLS integration (hopefully Nemerle should get status of a full CLS producer and extender). Release notes are almost complete.

I (Michal) have been working on a new type inference engine. There will be probably lots of work with pen and paper, but I seem to know how to solve the subtyping equations now. Finally got the details an hour ago...

The new engine should type the following code just fine:

class B {} class X : B {} class Y : B {} [X (), Y ()] // now: [(X () : B), (Y () : B)]

This is the most important problem with the current engine -- it cannot use more general type, once the type variable is substituted. The other problem is code quality -- it really needs rewrite.

11 Aug 2004: Contributors, assembly attributes and Nemerle marries GAC

New people contributing to Nemerle project

Lately the interest in Nemerle is increasing. Not only in using the language (and reporting bugs), but also in development of compiler and tools.

Ricardo Fernández Pascual had contributed a nice work on multidimensional arrays and some other patches, now he is working on implementing Polyphonic C#'s features for asynchronous concurrency abstractions (I've seen some preliminary code and I was impressed by the usage of all this macro stuff I developed).

Just a few days ago Atsushi Eno contributed a CodeDom provider for Nemerle - it revealed some bugs in the compiler and crashed on lack of a few features. Now it is in one SVN repository and it will be rewritten in Nemerle itself.

In the present time, during summer vacations the development from core team was a little bit slowed down, now finally I've put some chunks of code to the repository.

Signing assemblies with assembly attributes

Compiler is now able to understand and compile

[assembly: SomeAssemblyAttribute ("foo")]

This allows giving version, title, keypair, etc. to compiled assemblies. Well, actually saving this metadata required some additional work (they are not loaded from saved assembly attributes, but there is a special API for this in System.Reflection).

Nemerle + GAC

After signing our assemblies with strong key pairs, it is possible now to put them into the GAC. Bootstrapping compiler with these keys was a little painful, but probably mostly because of my mistake... I have put one of the stages' assembly into the gac and forgotten about it... So in the middle of bootstrapping, everything crashed with a message that assembly cannot be found. It took me some time before I realized that runtime is loading assembly from the gac, not from the proper directory.

Finally I have successfully fixed all the problems, fixed command line option for referencing assembly by its strong name and then it was done - we can place and use Nemerle libraries into Mono and .Net Framework GACs!

30 Jun 2004: Nemerle 0.1.4 released

The 0.1.4 source tarball of Nemerle compiler has hit our server. This is yet another incremental release before 0.2.0.

There's a source tarball as well as MSI, DEB and RPM packages.

The language

The library

The compiler

Other stuff

Have fun testing it :-)

Copyright © 2004, 2005 University of Wroclaw
RSS feed :: Top