[nem-en] Metaphor, staging, typed reflection

[Sandro Magi]

Kamil Skalski wrote:
>> I think the
>> safety of compile-time macro and multistaging is actually equivalent.
> 
> In the same sense as giving a mathematical formula for calculating
> measurements to the engineer building bridge a day before your
> holidays instead of giving him the ready made computations. ;)  The
> difference isn't that big, but in one case you at least made sure that
> formula produced some results given the real life data.

*If you can* do the computation at compile-time, why wouldn't you? Of
course that makes sense. Our discussion focuses on the fact that a
single stage language (or two stage like Nemerle) must "simulate"
reflection and runtime code generation by unsafe techniques for the
class of problems that can't be handled purely statically.

For instance, loading a plugin depends on the .NET VM's  assembly
loading via unsafe native code - this code is in the (fairly large) TCB,
but it's still unsafe.

> Well.. if you actually have ANY code at client side, then you need to
> make sure that you have the update mechanism for copying new dlls to
> the client anyways. So I just ask for a reasonable response, which is
> safer:
> - giving client a ready code, which will replace the previously stored
> assembly
> - or giving client a ready code generator, which will be executed on
> his machine to generate the ready code?

Right, so instead of a multistaged algorithm, you would have to resort
to a compile-time macro, plus an (automated?) update distribution
system. Not a win in overall complexity I think. :-)

> We have the additional flexibility, but you will never convince me
> that JIT engine is more reliable than compiler generating target code
> in advance.

Either way you're relying on some component to generate the executable
code, compiler or JIT. Why do you see the former as much more reliable
than the latter? A compiler is just stage-0, and the runtime code
generator could be the same one used as at compile-time.

If you want assurance right down to the bare machine, either with
compilers or JITs, then you need to resort to proof-carrying code.

>> But "compile-time analysis and code generation" is exactly what
>> intensional type analysis and multistaging is, except all the analysis
>> can now be expressed in the host language you're compiling, and also
>> guided by the developer!
> 
> I don't get what you mean. So you agree that those example could be
> pushed to compile-time instead of run-time?

What I meant by the above, is that intensional type analysis is
"compile-time analysis" of runtime behaviour, and that multistaging is
"code generation" in your statement.

> We could push our debate one level of abstraction higher:
> 
> I state that for n-staged program it is safer, more efficient (as seen
> by the user's experience) and more convenient during development if
> for all stages k < n they are executed in advance at the programmer's
> development environment.

Metaphor, MetaOCaml, etc. are counter-examples. Specialized/partially
evaluated code blocks may provide a more efficient algorithm than can be
expressed at compile-time, just like a JIT can create fast path cases of
an algorithm.

As long as the properties ensured by the type system are never violated,
it is just as safe to do this at runtime as at compile-time. Would you
disagree?

>> 3. then we must conclude that adding intensional type analysis and
>> multistaging yields a safer, more efficient, more expressive language
>> overall.
> 
> I'm not quite sure what do you mean by this sentence...

My point is that with the above two features, you can safely express
more algorithms than you can without them. This makes it more expressive.

Instead of simulating some dynamic features when you need them, you
already have them and they're properly type checked. This makes it safer.

Efficiency is an inference from the fact that partial evaluation can
always lead to more optimized code, so it's more efficient.

> Multi-staging
> is a good tool - it gives more abstraction to the programmer and
> allows producing more efficient code to be executed by the user. But
> it is always safer to have system with less stages -

So this is perhaps our disagreement. You agreed above that multistaging
and macros are equivalently safe, but here you contradict yourself by
saying fewer stages are safer.

I contend that multistaging does not introduce less safety as long as
the type system is no weaker than it was prior to adding the staging
extensions. How could it be?

> another example
> here is comparison of compiler and interpreter, they do actually the
> same job for the user, allow running some code and see the results.
> But one is MUCH more complex than the other.
> 

Indeed, and yet compilers are seen as safer since they are typically
fitted with static type systems.

Sandro