[nem-en] Integration again

[Michal Moskal]

On 11/16/06, Igor Tkachev <it at rsdn.ru> wrote:
> Michal, Kamil,
>
> We got other problems that delay our integration project. And it looks
> like we can't solve them without involving you.
>
> 1. First of all we do not completely understand how method's body
>    compilation process and typing work. The integration has to
>    postpone parsing and typing of methods for the performance purpose.
>    We build types' tree, but instead of method compilation, we just
>    store its location. Actual method body compilation is taken place
>    at the moment someone needs access to the method's parse/typed tree.
>
>    The problem is, sometime we get exceptions in the DtKind property
>    (we guess during the method by method typing). It's not clear what
>    is the local_context in the Resolve method:
>
>    public Resolve () : void
>    {
>      Util.locate (loc, {
>        def tmp = typer.local_context;
>        try {
>          typer.local_context = local_context;
>          DoResolve ();
>        } finally {
>          typer.local_context = tmp;
>        }
>      })
>    }
>
>    Do we have to store it somewhere? If so, where can we get it?
>
>    Also, what is the typer.solver.dt_store field? This is just a field
>    that has a wrong value and causes typing problems of some methods.

local_context is a field in the DelayedTyping class. It represents the
set of local variables at the point of program from which the given
delayed typing came from. I don't understand what kind of problems it
could pose.

To understand dt_store, you first need to realize that in the Nemerle
type engine one can save the state, do some typing, possibliy leading
to an error state and than discard the results. This is not very easy
to do, because that discarded part of typing could have changed
something in types describing different parts of the program (through
unification). This is where the complicated machinery in the Solver
and TyVar classes come in. They save a stack of undo logs for type
variables.

Now, if you save a state and do some typing on X obtaining Y you
*cannot* use Y after you restore the state, because the type bindings
resulting from typing of X are gone. Now because the delayed typings
generated during typing of X are logically part of Y, they should also
go away. This is where the dt_store comes in play. It is saved when
you save Solver state and later restored. So if you're getting
exceptions in DtKind, it means you're using typing results, that were
done in an unclean state, that was later rolled back.

I can only guess why that happens. It might be because you call typing
from two different threads and they mess up each other's Solver state.
The other option is that you run some typings and save state when the
Solver stack is not empty.

>    Another problem is the delayed typing. Sometimes we got
>    TExpr.Delayed in the typed tree and we do not know what to do to
>    resolve it in the correct way.

There should be no unresolved TExpr.Delayed after RunDelayedTypings
has finished (I've recently fixed one bug there though). However, if
you're getting exceptions in DtKind, you're also likely to get
unresolved TExpr.Delayed, the root cause is probably the same.

>    So, we need some explanation how method typing process works. We
>    need your comments and suggestions what we do wrong. And it would
>    be just great if you could find some time to help us debug the
>    method typing.

My suggestion would be to put assert(solver.IsTopLevel) in some
places. I'm not quite sure where though -- it should be pleaces where
you expect some typed code to be generated and stored.

>
>    If you need some software for this like VS 2005 or other Microsoft
>    stuff for development, we can ask our MVP guys for an MVPInvite
>    MSDN subscription with all needed software and licenses.
>
>
> 2. Locations again. We need somehow to distinguish generated by macro
>    or compiler locations and real locations from source code. So, we
>    will be able to filter them and to implement some features more
>    precisely.
>
>
> P.S. We are very close to release first version of the integration.
>      Right now, we are fixing bugs and we need your help to get the
>      compiler stable in completion mode.

Great!

-- 
   Michał