Thursday, September 15, 2016

Setting up ctags - automatically for multiple independent projects

Howdy everyone.

For those of you using vim, you'll know that <C-]> will jump to a function definition.
The problem with this is that to jump between files searching for a definition, ctags files must be hanging around somewhere for vim to find them. There are plenty of guides out there to set it up, including the great guide at http://ctags.sourceforge.net/faq.html.

Unfortunately, this generates a huge tags file at the root of your projects directory saving references to all of your projects. If the projects are independent this is less than desirable.

This post describes how to set up separate tag trees for all of your projects independently.
Note that this assumes that each project is controlled by its own git repository. This method should be easily extendable to any project structure which has a predictable file/directory at its root and nowhere else in the project (if it's git controlled the .git directory fits this nicely).

We will accomplish this automation through 3 steps

  1. Build a script to generate hidden .tag files
  2. Add instructions to our .vimrc to update tags upon file save
  3. Automatically regenerate the full tag tree via cronjobs
1. Scripting the tag generation
-------------------------------------

The script can be found in the attached file updateTags.sh.

The important bits are the following:

TOP_DIR=$1
for rootdir in $(find "$TOP_DIR" -type d -name '.git'); do
  rootdir_nongit=$(echo $rootdir | sed -e 's|/[^/]*$||');
  for subdir in $(find $rootdir_nongit -type d \( -name '.*'                \
                        -o -name 'ext' -o -name 'doc' -o -name 'docs'       \
                        -o -name 'bin' \) -prune -o -type d -print); do
    cd "$subdir"
    ctags -f .tags --format=2 --excmd=mixed --extra=+q+f --fields=nKsaSmtl *;
  done
  cd $rootdir_nongit
  ctags -f .tags --format=2 --excmd=mixed --extra=+q+f --fields=nKsaSmtl    \
        --file-scope=no -R *;
done

The TOP_DIR variable merely is a convenience variable to represent the input, which should
be a directory. We then have the outermost loop, which cycles through all directories within $TOP_DIR which have a .git directory, and saves extracts the names of those variables to the rootdir_nongit variable.
The inner loop cycles through all subdirectories of $rootdir_nongit, and drops ctags files into them with the name .tags (because making them hidden prevents them from cluttering the file list for ls).
After all the subdirectories (except for hidden, ext, doc, docs and bin subdirectories) have ctags files created, a giant ctags file with references for the entire project is dropped in at the $rootdir_nongit level (this is the -R option).
You may want to modify the ctags options, see the man page for a complete description.

The attached file also contains a bunch of logging statements, and clears out any empty ctags files.

2. Telling VIM where to find the tag files
--------------------------------------------------
The following code should be added to your .vimrc file.

i) Set tags to the .tags file in the directory of the file you're editing
let &tags=expand('%:p:h')."/.tags"

ii) Find the .tags file in the root directory when entering a file
function! Setup_tags ()
  for rootDirs in finddir(".git", expand('%:p:h').";~/Projects",-1)
    if strlen(findfile('.tags',rootDirs[0:-5]))
      let &tags .= rootDirs[0:-5] . ".tags,"
    endif
  endfor
endfunction
autocmd VimEnter *.* :call Setup_tags()
This function finds the root directory (containing the .git directory, terminating the search for the root directory at the Projects directory), and adds it's .tags file to the list of tags locations to search. The final command outside of the function actually sets up the tags when vim is entered.

iii) Automatically update the tag file upon save
Since ctags is pretty quick, regenerating the tag file is quick and imperceptible, so I just like to regenerate the local .tags file upon file save.
To ensure that tag files are only updated (not generated if they are missing), add the following:
function! Update_tags ()
  if strlen(findfile('.tags', expand('%:p:h')))
    :silent !(cd %:p:h;ctags -f .tags --format=2 --excmd=mixed --extra=+q+f --fields=nKsaSmtl *)&
  endif
endfunction
autocmd BufWritePost * :call Update_tags()
the silent causes this to happen without prompting the user to acknowledge. The cd %:p:h ensures that the .tags file in the directory of the file you are editing gets regenerated - rather than the directory you entered vim from.

NOTE: If the files you work on are extra large, it may be better to just append the changes to the .tags file (which does not remove tags to functions that were deleted - hence the default of just regenerating the whole thing). To append, add the -a flag to the ctags call.

3. Automatically regenerate the .tags files
--------------------------------------------------
Automatically running scripts is trivial with the magic of cron. Since we have already created an updateTags.sh script, add the following line to your crontab (via crontab -e):
*/30 * * * * ~/updateTags.sh ~/Projects > ~/.local/logs/updateTags.log 2>&1
This line causes the tags to regenerate every half hour (change the 30 to 15 for quarter hour updates, etc). Furthermore, if you are using the linked file with all the logging, all the logged output will be dumped into your ~/.local/logs/updateTags.log file (make sure that the ~/.local/logs file exists before the cron job executes).

Setting up ctags - automatically for multiple independent projects

Howdy everyone.

For those of you using vim, you'll know that <C-]> will jump to a function definition.
The problem with this is that to jump between files searching for a definition, ctags files must be hanging around somewhere for vim to find them. There are plenty of guides out there to set it up, including the great guide at http://ctags.sourceforge.net/faq.html.

Unfortunately, this generates a huge tags file at the root of your projects directory saving references to all of your projects. If the projects are independent this is less than desirable.

This post describes how to set up separate tag trees for all of your projects independently.
Note that this assumes that each project is controlled by its own git repository. This method should be easily extendable to any project structure which has a predictable file/directory at its root and nowhere else in the project (if it's git controlled the .git directory fits this nicely).

We will accomplish this automation through 3 steps

  1. Build a script to generate hidden .tag files
  2. Add instructions to our .vimrc to update tags upon file save
  3. Automatically regenerate the full tag tree via cronjobs
1. Scripting the tag generation
-------------------------------------

The script can be found in the attached file updateTags.sh.

The important bits are the following:

TOP_DIR=$1
for rootdir in $(find "$TOP_DIR" -type d -name '.git'); do
  rootdir_nongit=$(echo $rootdir | sed -e 's|/[^/]*$||');
  for subdir in $(find $rootdir_nongit -type d \( -name '.*'                \
                        -o -name 'ext' -o -name 'doc' -o -name 'docs'       \
                        -o -name 'bin' \) -prune -o -type d -print); do
    cd "$subdir"
    ctags -f .tags --format=2 --excmd=mixed --extra=+q+f --fields=nKsaSmtl *;
  done
  cd $rootdir_nongit
  ctags -f .tags --format=2 --excmd=mixed --extra=+q+f --fields=nKsaSmtl    \
        --file-scope=no -R *;
done

The TOP_DIR variable merely is a convenience variable to represent the input, which should
be a directory. We then have the outermost loop, which cycles through all directories within $TOP_DIR which have a .git directory, and saves extracts the names of those variables to the rootdir_nongit variable.
The inner loop cycles through all subdirectories of $rootdir_nongit, and drops ctags files into them with the name .tags (because making them hidden prevents them from cluttering the file list for ls).
After all the subdirectories (except for hidden, ext, doc, docs and bin subdirectories) have ctags files created, a giant ctags file with references for the entire project is dropped in at the $rootdir_nongit level (this is the -R option).
You may want to modify the ctags options, see the man page for a complete description.

The attached file also contains a bunch of logging statements, and clears out any empty ctags files.

2. Telling VIM where to find the tag files
--------------------------------------------------
The following code should be added to your .vimrc file.

i) Set tags to the .tags file in the directory of the file you're editing
let &tags=expand('%:p:h')."/.tags"

ii) Find the .tags file in the root directory when entering a file
function! Setup_tags ()
  for rootDirs in finddir(".git", expand('%:p:h').";~/Projects",-1)
    if strlen(findfile('.tags',rootDirs[0:-5]))
      let &tags .= rootDirs[0:-5] . ".tags,"
    endif
  endfor
endfunction
autocmd VimEnter *.* :call Setup_tags()
This function finds the root directory (containing the .git directory, terminating the search for the root directory at the Projects directory), and adds it's .tags file to the list of tags locations to search. The final command outside of the function actually sets up the tags when vim is entered.

iii) Automatically update the tag file upon save
Since ctags is pretty quick, regenerating the tag file is quick and imperceptible, so I just like to regenerate the local .tags file upon file save, via the following line:
autocmd BufWritePost * :silent !(cd %:p:h;ctags -f .tags 
                                \--format=2 --excmd=mixed --extra=+q+f 
                                \--fields=nKsaSmtl *)&
the silent causes this to happen without prompting the user to acknowledge. The cd %:p:h ensures that the .tags file in the directory of the file you are editing gets regenerated - rather than the directory you entered vim from.

NOTE: If the files you work on are extra large, it may be better to just append the changes to the .tags file (which does not remove tags to functions that were deleted - hence the default of just regenerating the whole thing). To append, add the -a flag to the ctags call.

3. Automatically regenerate the .tags files
--------------------------------------------------
Automatically running scripts is trivial with the magic of cron. Since we have already created an updateTags.sh script, add the following line to your crontab (via crontab -e):
*/30 * * * * ~/updateTags.sh ~/Projects > ~/.local/logs/updateTags.log 2>&1
This line causes the tags to regenerate every half hour (change the 30 to 15 for quarter hour updates, etc). Furthermore, if you are using the linked file with all the logging, all the logged output will be dumped into your ~/.local/logs/updateTags.log file (make sure that the ~/.local/logs file exists before the cron job executes).