[nem-en] Re: [nem-feedback] Visual Studio 2005 VSIP package
NoiseEHC
NoiseEHC at freemail.hu
Tue Feb 14 17:10:43 CET 2006
Ok, here is a patch (I hope it works, using TortoiseSVN)
Currently brace matching works but it is not perfect.
The problem is I think that LexerColorizer does not create the same tokens
as the PreParser. It should have to hande for example <[ ]> and
whitespace (it precedes tokens with the whitespace before them).
I tried to modify the Nemerle source but with no success. Because it takes
minutes to create a new nemerle.compiler.dll, I decided to create an
MSBuild task which can build it in one step. However it encounters an
internal compiler error and the msbuild task throws an exception (will
fix that). Kamil, can the compiled compiler (9.2) compile the nemerle
source in one step? I am really not interested in bootstrapping it.
Kamil Skalski wrote:
> 06-02-12, NoiseEHC <NoiseEHC at freemail.hu> napisał(a):
>
>> So here it is.
>>
>
> Ok, so plugin is now publicly available in
> http://nemerle.org/svn/vs-plugin/
>
>
>> I removed the brace matching since it requires lexer support.
>> Currently I do not know how to do it (either the LexerColorizer should
>> differentiate
>> between open/close braces or I should use the real lexer but how?)
>>
>
> You should be able to use
> def lex = LexerString (buffer);
> and
> def preparser = PreParser (lex);
> def tokens = preparser.PreParse ();
>
> (remember to first initialize environment, so syntax extensions are
> loaded from all used macros)
>
> This will give you a Token tree, which you can analyse for keywords
> and matchin braces (actually it contains all this information, only
> searching through it for specified location must be done). I'm not
> sure what will be the performance of this operation, but it would need
> experimentation.
>
>
>> Another problem is that the syntax highlighting is context dependent
>> (depends on
>> the using statements) and VS provides it with only an int state. So somehow
>> it would create a context table. For now it simply uses
>> Nemerle.Macros.dll only.
>>
>> You have to unzip the file to
>> C:\Program Files\Visual Studio 2005
>> SDK\2005.12\VisualStudioIntegration\Samples\NemerleIntegration\NemerleProject
>> because it references files from
>> C:\Program Files\Visual Studio 2005
>> SDK\2005.12\VisualStudioIntegration\Common\Source\CSharp\Project
>> as ..\..\..\Common\etc...
>>
>> It also loads the macro dll from c:\program
>> files\nemerle\Nemerle.Macros.dll so you have to have your Nemerle
>> installed there.
>>
>
> I think that our installer should create some env variable, like
> NEMERLE_DIR, so tools can easily access the place where Nemerle is
> installed.
>
>
>> Now the big question:
>> Can somebody help me finish this? I would need not only testers with VS
>> but somebody with knowledge with the Nemerle
>> compiler since the show stopper is that it requires modifying the source...
>>
>
> I think the way to go is to create some small testcases, like in
> ncc/testsuite/completion-playground
>
> This way we can focus on making compiler API work in well defined
> scenarios. The cruel fact is that at the moment I don't have time for
> much more than small fixes and answering questions...
>
>
>> Will it help if I manage to get some free VS licenses for the
>> university? (I think it can be possible as a MS sponsored project.)
>>
>>
>>> I will send you the latest version tomorrow and you could put it to
>>> the repository
>>>
>>>
>> _______________________________________________
>> https://nemerle.org/mailman/listinfo/devel-en
>>
>>
>>
>>
>>
>
>
> --
> Kamil Skalski
> http://nazgul.omega.pl
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> https://nemerle.org/mailman/listinfo/devel-en
>
-------------- next part --------------
Index: C:/Program Files/Visual Studio 2005 SDK/2005.12/VisualStudioIntegration/Samples/NemerleIntegration/NemerleProject/NemerleAuthoringScope.cs
===================================================================
--- C:/Program Files/Visual Studio 2005 SDK/2005.12/VisualStudioIntegration/Samples/NemerleIntegration/NemerleProject/NemerleAuthoringScope.cs (revision 0)
+++ C:/Program Files/Visual Studio 2005 SDK/2005.12/VisualStudioIntegration/Samples/NemerleIntegration/NemerleProject/NemerleAuthoringScope.cs (revision 0)
@@ -0,0 +1,31 @@
+using System;
+using Microsoft.VisualStudio.Package;
+using Microsoft.VisualStudio.TextManager.Interop;
+
+namespace NemerleProject
+{
+ public class NemerleAuthoringScope : AuthoringScope
+ {
+ public NemerleAuthoringScope()
+ : base() {
+ }
+
+ public override string Goto(Microsoft.VisualStudio.VSConstants.VSStd97CmdID cmd, IVsTextView textView, int line, int col, out TextSpan span) {
+ span = new TextSpan();
+ return null;
+ }
+
+ public override string GetDataTipText(int line, int col, out Microsoft.VisualStudio.TextManager.Interop.TextSpan span) {
+ span = new TextSpan();
+ return null;
+ }
+
+ public override Declarations GetDeclarations(Microsoft.VisualStudio.TextManager.Interop.IVsTextView view, int line, int col, TokenInfo info, ParseReason reason) {
+ return null;
+ }
+
+ public override Methods GetMethods(int line, int col, string name) {
+ return null;
+ }
+ }
+}
Index: C:/Program Files/Visual Studio 2005 SDK/2005.12/VisualStudioIntegration/Samples/NemerleIntegration/NemerleProject/NemerleColorizer.cs
===================================================================
--- C:/Program Files/Visual Studio 2005 SDK/2005.12/VisualStudioIntegration/Samples/NemerleIntegration/NemerleProject/NemerleColorizer.cs (revision 0)
+++ C:/Program Files/Visual Studio 2005 SDK/2005.12/VisualStudioIntegration/Samples/NemerleIntegration/NemerleProject/NemerleColorizer.cs (revision 0)
@@ -0,0 +1,104 @@
+using System;
+using Microsoft.VisualStudio.Package;
+using NC = Nemerle.Compiler;
+
+namespace NemerleProject
+{
+ class NemerleColorizer : IScanner
+ {
+ static NemerleColorizer() {
+ NC.LibraryReferenceManager.LoadMacrosFrom(@"c:\program files\nemerle\Nemerle.Macros.dll");
+ NC.GlobalEnv.Init();
+ }
+
+ public NemerleColorizer() {
+ lexer = new NC.LexerColorizer("");
+ }
+
+ NC.LexerColorizer lexer;
+
+ // Its calling depends on IVsColorizer.GetStateMaintenanceFlag, which defaults to true.
+ public bool ScanTokenAndProvideInfoAboutIt(TokenInfo tokenInfo, ref int state) {
+ NC.LexerColorizer.SyntaxToken tok = lexer.GetSyntaxToken(ref state);
+
+ // TokenInfo is only read in *.LanguageService.dll
+ tokenInfo.Trigger = TokenTriggers.None;
+ switch(tok.Token) {
+ case NC.SyntaxType.EndOfFile:
+ return false;
+
+ case NC.SyntaxType.Keyword:
+ tokenInfo.Color = TokenColor.Keyword;
+ tokenInfo.Type = TokenType.Keyword;
+ break;
+
+ case NC.SyntaxType.Identifier:
+ tokenInfo.Color = TokenColor.Identifier;
+ tokenInfo.Type = TokenType.Identifier;
+ break;
+
+ case NC.SyntaxType.OperatorDot:
+ case NC.SyntaxType.OperatorBrace:
+ case NC.SyntaxType.Operator:
+ tokenInfo.Color = TokenColor.Keyword;
+ tokenInfo.Type = TokenType.Operator;
+ switch(tok.Token) {
+ case NC.SyntaxType.OperatorBrace:
+ tokenInfo.Trigger = TokenTriggers.MatchBraces;
+ break;
+ case NC.SyntaxType.OperatorDot:
+ tokenInfo.Trigger = TokenTriggers.MemberSelect;
+ break;
+ default:
+ break;
+ }
+ break;
+
+ case NC.SyntaxType.Comment:
+ tokenInfo.Color = TokenColor.Comment;
+ if(state == 0)
+ tokenInfo.Type = TokenType.LineComment;
+ else
+ tokenInfo.Type = TokenType.Comment;
+ break;
+
+ case NC.SyntaxType.CharLiteral:
+ tokenInfo.Color = TokenColor.String;
+ tokenInfo.Type = TokenType.Literal;
+ break;
+
+ case NC.SyntaxType.IntLiteral:
+ tokenInfo.Color = TokenColor.Number;
+ tokenInfo.Type = TokenType.Literal;
+ break;
+
+ case NC.SyntaxType.StringLiteral:
+ tokenInfo.Color = TokenColor.String;
+ tokenInfo.Type = TokenType.String;
+ break;
+
+ default:
+ tokenInfo.Color = TokenColor.Text;
+ tokenInfo.Type = TokenType.Text;
+ break;
+ }
+
+ tokenInfo.StartIndex = tok.StartPos;
+ tokenInfo.EndIndex = tok.EndPos;
+
+ // Nemerle lexer count columns from 0, and the EndPos is inclusive. That is VS expects.
+ //System.Diagnostics.Trace.WriteLine(string.Format("{0},{1} : {2}", tok.StartPos, tok.EndPos, tok.Token.ToString()));
+
+ //TODO: it has to handle whitespace and # (pragma, region, etc)
+ // currently it handles } as a token starting from the beginning of line...
+
+ return true;
+ }
+
+ // The current implementation in *.LanguageService.dll always uses offset 0.
+ public void SetSource(string source, int offset) {
+ lexer.SetString(source, offset);
+ //System.Diagnostics.Trace.WriteLine(source);
+ }
+ }
+}
Index: C:/Program Files/Visual Studio 2005 SDK/2005.12/VisualStudioIntegration/Samples/NemerleIntegration/NemerleProject/NemerleLanguage.cs
===================================================================
--- C:/Program Files/Visual Studio 2005 SDK/2005.12/VisualStudioIntegration/Samples/NemerleIntegration/NemerleProject/NemerleLanguage.cs (revision 6122)
+++ C:/Program Files/Visual Studio 2005 SDK/2005.12/VisualStudioIntegration/Samples/NemerleIntegration/NemerleProject/NemerleLanguage.cs (working copy)
@@ -1,89 +1,64 @@
-
using System;
-using System.Collections.Generic;
-using System.Text;
-using System.Runtime.InteropServices;
-
using Microsoft.VisualStudio.Package;
using Microsoft.VisualStudio.TextManager.Interop;
-
using NC = Nemerle.Compiler;
//TODO: if you open a single .n file (not a project) then there is no syntax highlight
-//TODO: match_brace finds the pair of braces inside comments (needs real lexer support)
namespace NemerleProject
{
- [Guid("0F15B13F-4C95-405f-97A5-7E0E341C4FD3")]
+ [System.Runtime.InteropServices.Guid("0F15B13F-4C95-405f-97A5-7E0E341C4FD3")]
class NemerleLanguage : LanguageService
{
private NemerleColorizer _scanner = null;
private LanguagePreferences _preferences = null;
- private IVsTextLines _buffer = null;
- private TextSpan[] match_braces(ParseRequest req) {
- int pos;
- _buffer.GetPositionOfLineIndex(req.Line, req.Col, out pos);
- --pos;
- char brace = req.Text[pos];
- char complement;
- switch(brace) {
- case ')':
- complement = '(';
- break;
- case ']':
- complement = '[';
- break;
- case '}':
- complement = '{';
- break;
- default:
- return null;
+ /// <summary>
+ /// Recursively add brace pairs to Sink.
+ /// </summary>
+ /// <param name="Group">Parent token which has Child or Body so we can enumerate</param>
+ /// <param name="Sink">The object holding the braces</param>
+ /// <param name="Level">Recursing depth so the Sink can decide the correct brace to choose when they are nested</param>
+ private void AddBraces(NC.Token Group, AuthoringSink Sink, int Level) {
+ foreach(NC.Token token in Group) {
+ // We cannot optimize by ignoring groups with wrong location early since
+ // for example loosegroup contains only the first token's location.
+ if(token is NC.Token.RoundGroup || token is NC.Token.BracesGroup || token is NC.Token.SquareGroup || token is NC.Token.QuoteGroup) {
+ // Since PreParse seems to destroy BeginBrace/EndBrace and alike
+ // we have to calculate the assumed TextSpans.
+ int bracelen = (token is NC.Token.QuoteGroup) ? 2 : 1;
+ //FUCK: seems VS counts lines and columns from 0,0 while Nemerle from 1,1
+ TextSpan start = new TextSpan();
+ start.iStartLine = start.iEndLine = token.Location.Line-1;
+ start.iStartIndex = token.Location.Column-1;
+ start.iEndIndex = start.iStartIndex + bracelen;
+ TextSpan end = new TextSpan();
+ end.iStartLine = end.iEndLine = token.Location.EndLine - 1;
+ end.iStartIndex = token.Location.EndColumn - bracelen - 1;
+ end.iEndIndex = end.iStartIndex + bracelen;
+ Sink.MatchPair(start, end, Level);
+ // Nemerle parser counts lines/columns from 1/1, and the EndPos is exclusive.
+ //System.Diagnostics.Trace.WriteLine(string.Format("{0},{1}-{2},{3} : {4}", token.Location.Line, token.Location.Column, token.Location.EndLine, token.Location.EndColumn, token.ToString()));
+ AddBraces(token, Sink, Level+1);
+ } else if(token is NC.Token.LooseGroup || token is NC.Token.Namespace) {
+ AddBraces(token, Sink, Level+1);
+ }
}
-
- int nesting = 0;
-
- do {
- char curr = req.Text[pos];
- if(curr == brace)
- nesting++;
- else if(curr == complement)
- nesting--;
- --pos;
- }
- while(pos >= 0 && nesting > 0);
-
- if(nesting > 0)
- return null;
- else {
- TextSpan[] result = new TextSpan[2];
- _buffer.GetLineIndexOfPosition(pos + 1, out result[0].iStartLine,
- out result[0].iStartIndex);
- result[0].iEndLine = result[0].iStartLine;
- result[0].iEndIndex = result[0].iStartIndex + 1;
- result[1].iStartLine = req.Line;
- result[1].iStartIndex = req.Col - 1;
- result[1].iEndLine = req.Line;
- result[1].iEndIndex = req.Col;
-
- return result;
- }
}
public override AuthoringScope ParseSource(ParseRequest req) {
//System.Diagnostics.Trace.WriteLine(string.Format("ParseSource{0}", req.Reason));
- if(req.Reason != ParseReason.MatchBraces)
+ // HighlightBraces -> Show brace pairs when moving cursor
+ // MatchBraces -> Jump to maching brace pair when pressed Ctrl+]
+ if(req.Reason != ParseReason.HighlightBraces && req.Reason != ParseReason.MatchBraces)
return new NemerleAuthoringScope();
- NC.LexerColorizer lexer = new NC.LexerColorizer(req.Text);
- int state = 0;
- while(true) {
- NC.LexerColorizer.SyntaxToken tok = lexer.GetSyntaxToken(ref state);
- if(tok.Token == NC.SyntaxType.EndOfFile)
- break;
- //if(tok.Token == Nemerle.Compiler.SyntaxType.OperatorBrace && tok.
- }
+ NC.LexerString lexer = new Nemerle.Compiler.LexerString(req.Text, new NC.Location(0, 1, 1));
+ NC.PreParser preparser = new Nemerle.Compiler.PreParser(lexer);
+ NC.Token.BracesGroup braces = preparser.PreParse();
+ AddBraces(braces, req.Sink, 0);
+
return new NemerleAuthoringScope();
}
@@ -95,8 +70,6 @@
public override IScanner GetScanner(IVsTextLines buffer) {
if(_scanner == null)
_scanner = new NemerleColorizer();
- this._buffer = buffer;
-
return _scanner;
}
@@ -109,129 +82,7 @@
_preferences.EnableMatchBracesAtCaret = true; // hilites braces when cursor moved over them
//_preferences.IndentStyle = IndentingStyle.Smart;
}
-
return _preferences;
}
}
-
- class NemerleColorizer : IScanner
- {
- static NemerleColorizer() {
- NC.LibraryReferenceManager.LoadMacrosFrom(@"c:\program files\nemerle\Nemerle.Macros.dll");
- NC.GlobalEnv.Init();
- }
-
- public NemerleColorizer() {
- lexer = new NC.LexerColorizer("");
- }
-
- #region IScanner Members
-
- NC.LexerColorizer lexer;
-
- // Its calling depends on IVsColorizer.GetStateMaintenanceFlag, which defaults to true.
- public bool ScanTokenAndProvideInfoAboutIt(TokenInfo tokenInfo, ref int state) {
- NC.LexerColorizer.SyntaxToken tok = lexer.GetSyntaxToken(ref state);
-
- // TokenInfo is only read in *.LanguageService.dll
- tokenInfo.Trigger = TokenTriggers.None;
- switch(tok.Token) {
- case NC.SyntaxType.EndOfFile:
- return false;
-
- case NC.SyntaxType.Keyword:
- tokenInfo.Color = TokenColor.Keyword;
- tokenInfo.Type = TokenType.Keyword;
- break;
-
- case NC.SyntaxType.Identifier:
- tokenInfo.Color = TokenColor.Identifier;
- tokenInfo.Type = TokenType.Identifier;
- break;
-
- case NC.SyntaxType.OperatorDot:
- case NC.SyntaxType.OperatorBrace:
- case NC.SyntaxType.Operator:
- tokenInfo.Color = TokenColor.Keyword;
- tokenInfo.Type = TokenType.Operator;
- switch(tok.Token) {
- case NC.SyntaxType.OperatorBrace:
- tokenInfo.Trigger = TokenTriggers.MatchBraces;
- break;
- case NC.SyntaxType.OperatorDot:
- tokenInfo.Trigger = TokenTriggers.MemberSelect;
- break;
- default:
- break;
- }
- break;
-
- case NC.SyntaxType.Comment:
- tokenInfo.Color = TokenColor.Comment;
- if(state == 0)
- tokenInfo.Type = TokenType.LineComment;
- else
- tokenInfo.Type = TokenType.Comment;
- break;
-
- case NC.SyntaxType.CharLiteral:
- tokenInfo.Color = TokenColor.String;
- tokenInfo.Type = TokenType.Literal;
- break;
-
- case NC.SyntaxType.IntLiteral:
- tokenInfo.Color = TokenColor.Number;
- tokenInfo.Type = TokenType.Literal;
- break;
-
- case NC.SyntaxType.StringLiteral:
- tokenInfo.Color = TokenColor.String;
- tokenInfo.Type = TokenType.String;
- break;
-
- default:
- tokenInfo.Color = TokenColor.Text;
- tokenInfo.Type = TokenType.Text;
- break;
- }
-
- tokenInfo.StartIndex = tok.StartPos;
- tokenInfo.EndIndex = tok.EndPos;
-
- return true;
- }
-
- // The current implementation in *.LanguageService.dll always uses offset 0.
- public void SetSource(string source, int offset) {
- lexer.SetString(source, offset);
- System.Diagnostics.Trace.WriteLine(source);
- }
-
- #endregion
- }
-
- public class NemerleAuthoringScope : AuthoringScope
- {
- public NemerleAuthoringScope()
- : base() {
- }
-
- public override string Goto(Microsoft.VisualStudio.VSConstants.VSStd97CmdID cmd, IVsTextView textView, int line, int col, out TextSpan span) {
- span = new TextSpan();
- return null;
- }
-
- public override string GetDataTipText(int line, int col, out Microsoft.VisualStudio.TextManager.Interop.TextSpan span) {
- span = new TextSpan();
- return null;
- }
-
- public override Declarations GetDeclarations(Microsoft.VisualStudio.TextManager.Interop.IVsTextView view, int line, int col, TokenInfo info, ParseReason reason) {
- return null;
- }
-
- public override Methods GetMethods(int line, int col, string name) {
- return null;
- }
- }
}
Index: C:/Program Files/Visual Studio 2005 SDK/2005.12/VisualStudioIntegration/Samples/NemerleIntegration/NemerleProject/NemerleProject.csproj
===================================================================
--- C:/Program Files/Visual Studio 2005 SDK/2005.12/VisualStudioIntegration/Samples/NemerleIntegration/NemerleProject/NemerleProject.csproj (revision 6122)
+++ C:/Program Files/Visual Studio 2005 SDK/2005.12/VisualStudioIntegration/Samples/NemerleIntegration/NemerleProject/NemerleProject.csproj (working copy)
@@ -58,6 +58,10 @@
<Name>Microsoft.VisualStudio.TextManager.Interop</Name>
</Reference>
<Reference Include="Microsoft.VisualStudio.TextManager.Interop.8.0, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
+ <Reference Include="Nemerle, Version=0.9.1.0, Culture=neutral, PublicKeyToken=e080a9c724e2bfcd, processorArchitecture=MSIL">
+ <SpecificVersion>False</SpecificVersion>
+ <HintPath>..\..\..\..\..\..\Nemerle\Nemerle.dll</HintPath>
+ </Reference>
<Reference Include="Nemerle.Compiler, Version=0.9.1.0, Culture=neutral, PublicKeyToken=5291d186334f6101, processorArchitecture=MSIL">
<SpecificVersion>False</SpecificVersion>
<HintPath>..\..\..\..\..\..\Nemerle\Nemerle.Compiler.dll</HintPath>
@@ -82,6 +86,8 @@
<Compile Include="AssemblyInfo.cs">
<SubType>Code</SubType>
</Compile>
+ <Compile Include="NemerleAuthoringScope.cs" />
+ <Compile Include="NemerleColorizer.cs" />
<Compile Include="NemerleLanguage.cs" />
<Compile Include="NemerleFileNode.cs" />
<Compile Include="NemerleMenus.cs" />
More information about the devel-en
mailing list