Tips for using NeoVim as devtool - 2

This is the second article about NeoVim tips, if you are interested in the first one, please check here

Motivation: the need for a scratch pad within NeoVim

Quite often in my daily work, I need to check some response that are in JSON or XML format. I could open VSCode and paste in and format whatever I got to readable format, or I could also find an online formatter to format these things. But, as a VIM user, I want to do it in a VIM way. The answer comes down to scratch buffer.

Solution: Scratch buffer in VIM

A scratch buffer, well, is a scratch buffer.
In VIM, this means:

  1. It should only stay within memory, no actual file backing. So setlocal buftype=nofile
  2. It should not have swap file backing. So setlocal noswapfile
  3. It should be hidden from the list of buffers. So set bufhidden=hide

Based on these characteristics, we can create a VIM function that creates a scratch buffer:

1
2
3
4
5
6
7
command! -nargs=0 Ns call Newscratch()
fun! Newscratch()
execute 'tabnew '
setlocal buftype=nofile
setlocal bufhidden=hide
setlocal noswapfile
endfun

The above code declares a function called Newscratch in vim and map that function as Ns for short. Newscratch creates a new tab, and then set that tab as a scratch as defined above.

Since I'm in the process of migrating my vim configuration into Lua for NeoVim, here is a Lua version of the same function

1
2
3
4
5
6
7
8
9
10
11
12
13
function new_scratch()
vim.api.nvim_command('tabnew')
--[[
[Parameters:
[{listed} Sets 'buflisted'
[{scratch} Creates a "throwaway" scratch-buffer for temporary work (always
['nomodified'). Also sets 'nomodeline' on the buffer.
]]
vim.api.nvim_create_buf(false, true)
vim.opt_local.buftype = 'nofile'
vim.opt_local.bufhidden = 'hide'
vim.opt_local.swapfile = false
end

Bonus: Enhanced scratch buffer

I have found several issues with my initial implementation of scratch buffer, this is not an exhausted list, but the issues are:

  1. I sometimes want to save the content of the buffer to an actual file, I found myself doing that quite often, to a point I want to have it builtin to the scratch buffer.
  2. Quite often, I want to format the content of the buffer, if it's xml, I want to use XML formatter, if JSON, use JSON formatter. I also need to decode base64 encoded strings to normal text, and it would be a good idea to just include this as well.
  3. When JSON and XML files are large, they are quite difficult to navigate, I want to be able to use the vim builtin folding functionality to fold such files.

These things listed above are what I use most often, therefore, I ended up enhancing my scratch buffer with some keymaps.
First up is the saving part, in VIM script, you can achieve it like this:

1
nnoremap <buffer> <silent> ,s :execute(':file scratch_') .. localtime() .. (&ft != '' ? '.' .. &ft : '')<cr><bar>:set buftype= swapfile<bar>:w<cr>

The above code maps ,s to saving the scratch file with localtime() (which returns unix timestamp) and appending the appropriate file format based on the &ft variable, it also reset the buffer type so subsequent changes can be saved to the file.

Next, I added the ability to format, this is done through external tools: for JSON, I used jq; for XML, I used xmllint. Both of these software are open source.

1
2
3
nnoremap <silent> <buffer> ,xm :%!xmllint --format - <cr>
nnoremap <silent> <buffer> ,ff :%!jq . <cr>
vmap <silent> <buffer> ,dc :!base64 -d <cr>

As you can see, I mapped ,ff to jq formatting and ,xm to XML formatting, ,dc is mapped to base64 decoding.

Last but not least is the folding, I eventually combined the formatting with the folding, so I will present the updated version of the above code that included folding:

1
2
nnoremap <silent> <buffer> ,ff :%!jq . <cr><bar>:setf json<bar> set foldmethod=expr<bar>set foldexpr=nvim_treesitter#foldexpr()<bar>:redraw!<bar>:foldopen<cr>
nnoremap <silent> <buffer> ,xm :%!xmllint --format - <cr><bar>:setf html<bar>:set foldmethod=expr<bar>set foldexpr=nvim_treesitter#foldexpr()<bar>:redraw!<cr>

This folding uses nvim_treesitter to fold, so a prerequisite is to have treesitter installed.

Once I solved the three issues I faced, I combined the functionality into the VIM function I created, the final version is as below:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
command! -nargs=0 Ns call Newscratch()
fun! Newscratch()
execute 'tabnew '
nnoremap <silent> <buffer> <Leader>ff :%!jq . <cr><bar>:setf json<bar> set foldmethod=expr<bar>set foldexpr=nvim_treesitter#foldexpr()<bar>:redraw!<bar>:foldopen<cr>
nnoremap <silent> <buffer> ,ff :%!jq . <cr><bar>:setf json<bar> set foldmethod=expr<bar>set foldexpr=nvim_treesitter#foldexpr()<bar>:redraw!<bar>:foldopen<cr>
nnoremap <silent> <buffer> ,xm :%!xmllint --format - <cr><bar>:setf html<bar>:set foldmethod=expr<bar>set foldexpr=nvim_treesitter#foldexpr()<bar>:redraw!<cr>
nnoremap <buffer> <silent> ,s :execute(':file scratch_') .. localtime() .. (&ft != '' ? '.' .. &ft : '')<cr><bar>:set buftype= swapfile<bar>:w<cr>
nnoremap <buffer> <silent> ,n :execute(':file ~/notes/note_') .. localtime() .. (&ft != '' ? '.' .. &ft : '')<cr><bar>:set buftype= swapfile<bar>:w<cr>
vmap <silent> <buffer> ,dc :!base64 -d <cr>
setlocal buftype=nofile
setlocal bufhidden=hide
setlocal noswapfile
setlocal wrap
endfun

Future Improvements

I'm actually quite happy with what I have achieved with the script I wrote above. I do acknowledge there are several things that can be improved, those are on my wishlist, and I do not have a way to achieve them at the moment:

  1. Format file upon pasting based on the file type detection. -- This is my most wanted feature, but question is how will you be able to know the file type without a file extension, probably will never be able to achieve.
  2. Scratch buffer that can be opened within a float window within VIM. -- This is achievable, but may not have too much value, maybe can do that later.
  3. Split things into function. -- This is probably really necessary, but for now, I will just leave as is. Maybe will convert to function when I have time.