1 ![Vim Logo](https://github.com/vim/vim/blob/master/runtime/vimlogo.gif)
5 This is a new syntax for Vim script that was introduced with Vim 9.0.
6 It intends making Vim script faster and better.
11 ## 1. FASTER VIM SCRIPT
13 The third item on the poll results of 2018, after popup windows and text
14 properties, both of which have been implemented, is faster Vim script.
17 I have been throwing some ideas around, and soon came to the conclusion
18 that the current way functions are called and executed, with
19 dictionaries for the arguments and local variables, is never going to be
20 very fast. We're lucky if we can make it twice as fast. The overhead
21 of a function call and executing every line is just too high.
23 So what then? We can only make something fast by having a new way of
24 defining a function, with similar but different properties of the old
26 * Arguments are only available by name, not through the a: dictionary or
28 * Local variables are not available in an l: dictionary.
29 * A few more things that slow us down, such as exception handling details.
31 I Implemented a "proof of concept" and measured the time to run a simple
32 for loop with an addition (Justin used this example in his presentation,
37 for i in range(1, 2999999)
43 | --------| -------- |
44 | Vim old | 5.018541 |
48 | Vim new | 0.073595 |
50 That looks very promising! It's just one example, but it shows how much
51 we can gain, and also that Vim script can be faster than builtin
54 LuaJit is much faster at Lua-only instructions. In practice the script would
55 not do something useless counting, but change the text. For example,
56 reindent all the lines:
60 for i in range(1, 100000)
61 call setline(i, ' ' .. getline(i))
62 let totallen += len(getline(i))
67 | --------| -------- |
68 | Vim old | 0.578598 |
72 | Vim new | 0.079692 |
74 [These times were measured on a different system by Dominique Pelle]
76 The differences are smaller, but Vim 9 script is clearly the fastest.
77 Using LuaJIT is only a little bit faster than plain Lua here, clearly the call
78 back to the Vim code is costly.
80 How does Vim9 script work? The function is first compiled into a sequence of
81 instructions. Each instruction has one or two parameters and a stack is
82 used to store intermediate results. Local variables are also on the
83 stack, space is reserved during compilation. This is a fairly normal
84 way of compilation into an intermediate format, specialized for Vim,
85 e.g. each stack item is a typeval_T. And one of the instructions is
86 "execute Ex command", for commands that are not compiled.
89 ## 2. DEPRIORITIZE INTERFACES
91 Attempts have been made to implement functionality with built-in script
92 languages such as Python, Perl, Lua, Tcl and Ruby. This never gained much
93 foothold, for various reasons.
95 Instead of using script language support in Vim:
96 * Encourage implementing external tools in any language and communicate
97 with them. The job and channel support already makes this possible.
98 Really any language can be used, also Java and Go, which are not
100 * No priority for the built-in language interfaces. They will have to be kept
101 for backwards compatibility, but many users won't need a Vim build with these
103 * Improve the Vim script language, it is used to communicate with the external
104 tool and implements the Vim side of the interface. Also, it can be used when
105 an external tool is undesired.
107 Altogether this creates a clear situation: Vim with the +eval feature
108 will be sufficient for most plugins, while some plugins require
109 installing a tool that can be written in any language. No confusion
110 about having Vim but the plugin not working because some specific
111 language is missing. This is a good long term goal.
113 Rationale: Why is it better to run a tool separately from Vim than using a
114 built-in interface and interpreter? Take for example something that is
116 * The built-in interface uses the embedded python interpreter. This is less
117 well maintained than the python command. Building Vim with it requires
118 installing developer packages. If loaded dynamically there can be a version
120 * When running the tool externally the standard python command can be used,
121 which is quite often available by default or can be easily installed.
122 * The built-in interface has an API that is unique for Vim with Python. This is
123 an extra API to learn.
124 * A .py file can be compiled into a .pyc file and execute much faster.
125 * Inside Vim multi-threading can cause problems, since the Vim core is single
126 threaded. In an external tool there are no such problems.
127 * The Vim part is written in .vim files, the Python part is in .py files, this
129 * Disadvantage: An interface needs to be made between Vim and Python.
130 JSON is available for this, and it's fairly easy to use. But it still
131 requires implementing asynchronous communication.
134 ## 3. BETTER VIM SCRIPT
136 To make Vim faster a new way of defining a function needs to be added.
137 While we are doing that, since the lines in this function won't be fully
138 backwards compatible anyway, we can also make Vim script easier to use.
139 In other words: "less weird". Making it work more like modern
140 programming languages will help. No surprises.
142 A good example is how in a function the arguments are prefixed with
143 "a:". No other language I know does that, so let's drop it.
145 Taking this one step further is also dropping "s:" for script-local variables;
146 everything at the script level is script-local by default. Since this is not
147 backwards compatible it requires a new script style: Vim9 script!
149 To avoid having more variations, the syntax inside a compiled function is the
150 same as in Vim9 script. Thus you have legacy syntax and Vim9 syntax.
152 It should be possible to convert code from other languages to Vim
153 script. We can add functionality to make this easier. This still needs
154 to be discussed, but we can consider adding type checking and a simple
155 form of classes. If you look at JavaScript for example, it has gone
156 through these stages over time, adding real class support and now
157 TypeScript adds type checking. But we'll have to see how much of that
158 we actually want to include in Vim script. Ideally a conversion tool
159 can take Python, JavaScript or TypeScript code and convert it to Vim
160 script, with only some things that cannot be converted.
162 Vim script won't work the same as any specific language, but we can use
163 mechanisms that are commonly known, ideally with the same syntax. One
164 thing I have been thinking of is assignments without ":let". I often
165 make that mistake (after writing JavaScript especially). I think it is
166 possible, if we make local variables shadow commands. That should be OK,
167 if you shadow a command you want to use, just rename the variable.
168 Using "var" and "const" to declare a variable, like in JavaScript and
169 TypeScript, can work:
173 def MyFunction(arg: number): number
185 The similarity with JavaScript/TypeScript can also be used for dependencies
186 between files. Vim currently uses the `:source` command, which has several
188 * In the sourced script, is not clear what it provides. By default all
189 functions are global and can be used elsewhere.
190 * In a script that sources other scripts, it is not clear what function comes
191 from what sourced script. Finding the implementation is a hassle.
192 * Prevention of loading the whole script twice must be manually implemented.
194 We can use the `:import` and `:export` commands from the JavaScript standard to
195 make this much better. For example, in script "myfunction.vim" define a
196 function and export it:
199 vim9script " Vim9 script syntax used here
201 var local = 'local variable is not exported, script-local'
203 export def MyFunction() " exported function
206 def LocalFunction() " not exported, script-local
210 And in another script import the function:
213 vim9script " Vim9 script syntax used here
215 import MyFunction from 'myfunction.vim'
218 This looks like JavaScript/TypeScript, thus many users will understand the
221 These are ideas, this will take time to design, discuss and implement.
222 Eventually this will lead to Vim 9!
225 ## Code for sum time measurements
227 Vim was build with -O2.
232 for i in range(1, 2999999)
241 for i in range(1, 3000000):
244 return py3eval('sum')
250 for i = 1, 2999999 do
254 return luaeval('sum')
259 for i in range(1, 2999999)
265 let start = reltime()
267 echo 'Vim old: ' .. reltimestr(reltime(start))
269 let start = reltime()
271 echo 'Python: ' .. reltimestr(reltime(start))
273 let start = reltime()
275 echo 'Lua: ' .. reltimestr(reltime(start))
277 let start = reltime()
279 echo 'Vim new: ' .. reltimestr(reltime(start))
282 ## Code for indent time measurements
287 for i in range(1, 100000)
288 setline(i, ' ' .. getline(i))
289 totallen += len(getline(i))
296 for i in range(1, 100000)
297 call setline(i, ' ' .. getline(i))
298 let totallen += len(getline(i))
309 totallen = totallen + string.len(b[i])
312 return luaeval('totallen')
317 cb = vim.current.buffer
319 for i in range(0, 100000):
321 totallen += len(cb[i])
323 return py3eval('totallen')
327 call setline(1, range(100000))
328 let start = reltime()
330 echo 'Vim old: ' .. reltimestr(reltime(start))
334 call setline(1, range(100000))
335 let start = reltime()
337 echo 'Python: ' .. reltimestr(reltime(start))
341 call setline(1, range(100000))
342 let start = reltime()
344 echo 'Lua: ' .. reltimestr(reltime(start))
348 call setline(1, range(100000))
349 let start = reltime()
351 echo 'Vim new: ' .. reltimestr(reltime(start))