Directory-local Org Agendas

I recently moved to Germany, which has been a lovely experience for the most part. However, there are a lot of things to keep track of in an international move, and Org Mode has proven indispensable for this. My partner is also making the leap across the pond soon, and we have a lot of shared responsibilities. Coordinating these was getting difficult. So, last weekend, I spent some time coming up with a solution.

On a related note, my partner, Jay, has been getting very good [src]​ at using Org Mode along with git for making websites. The amount of Emacs skills he's picked up in just a few months working on it has been incredible to watch. I have yet to really get him into Emacs-lisp, but he's already fallen into the trap of Emacs; he will learn to love lisp eventually.

Naturally, then, Org Mode should be the best way to manage our shared tasks and responsibilities. I may write another article in the future about how we set it up on our phones using Syncthing and Orgzly Revived, but for this article, I want to focus on a problem that has bugged me since starting to use Org Agenda so many years ago, and a potential solution I found recently that, while not the most elegant, is a nice trade-off in my opinion.

The idea of sharing an agenda with other people is very natural to me. I have two separate calendars for work and personal events, one of which is semi-public. I would like to be able to, for example, share my screen in a virtual meeting and show my work agenda without mixing in any personal tasks.

Ideally, Org Agenda should know that when it is launched from a file in, let's say a ~/work directory, only files within that directory should be considered for building the agenda.

Also, the configuration of the agenda files and other Org settings should be independent of the location of the folder. In other words, moving the ~/work folder to a different location shouldn't affect how the agenda looks when launched from within it. This is even more necessary for a shared agenda with multiple people because hardly any two file-systems are alike. It would be an extra burden to manually coordinate and maintain a consistent file structure outside of the synced folder.

It seems like a job very well suited to directory-local variables. However, my previous attempts to modify the behavior of Org Agenda through file- and directory-local variables had proven fruitless, and I gave up on the problem more than a year ago.

The potential for a solution here feels like it could unlock Org Agenda being used as a shared project management tool in many more places. My mind immediately goes to long term planning of open source projects, given my background and given that most forges can render Org files as html, usually by default.

The idea that the management of a project is something that should exist outside of a project has always bugged me on a spiritual level. I mean, what actually is a Github milestone? From a practical point of view it is a layer in the hierarchy of project tasks that exists somewhere in between low-level issues and the mission statement. But what is it? From an informatics point of view, a milestone is data that is tethered to, and only exists on, a specific host. Milestones may be accidentally stored in various distributed caches, but that is not the goal nor is it reliable. If your git forge were to vanish tomorrow, most of the code, and the history of that code, will be recovered from peoples' local copies, but project management tools, like milestones, would likely be irrecoverable because of the reliance on a single node. In some cases, it is difficult to transfer milestones and such between forges even when both forges are still accessible.

On the other hand, using a plain text file to hold a project README seems like a no-brainer. If somebody showed me a cool project that had a README.ai or a README.docx, I would probably back away slowly. But we generally put up with having little to no control over our project management tooling, it might as well be a proprietary binary format.

To keep this article a practical length, let's quickly move on from my spiritually-adopted beliefs about plaintext and project management to the solution at hand.

Unknowingly, the issue that I ran into before with directory-local variables is that Org Agenda buffers, being entirely in-memory and lacking a corresponding file, do not automatically trigger loading dir-local variables. However, these buffers do still have a default-directory, which is enough for the built-in function hack-dir-local-variables-non-file-buffer to work with. That function name seemed almost comically well-suited for my problem, and I was suspicious that was all I needed. In a way it is a bit more complicated, and that is where the trade-off I mentioned is.

;; .dir-locals.el

;; Does not work!
((nil
  (org-agenda-mode-hook . hack-dir-local-variables-non-file-buffer)))

Changing the value of the org-agenda-mode-hook locally is not sufficient because this customization to the hook is, itself, not loaded by default. It's a chicken or the egg problem. What we need is to globally modify org-agenda-mode-hook:

;; .dir-locals.el

;; Kinda working...
((nil
  (eval . (add-hook 'org-agenda-mode-hook
                  'hack-dir-local-variables-non-file-buffer))))

This global change is compatible with my requirements above because it is not specific to any particular folder. But it is still a shame that globals seem necessary.

This approach works most of the time, but there are a few edge cases. Issues arise because the agenda buffer is not always created from scratch, especially when using org-agenda-sticky. If an existing buffer of the same name already exists, it may be reused in some circumstances. In that case, the reused buffer will carry over the original default-directory, which may not match your current directory.

So, I modified the org-agenda-mode-hook to first cd (change directory) to the that of the most recent buffer. Then, when hack-dir-local-variables-non-file-buffer is evaluated, it will be from within the same directory as the buffer from which you launched the agenda.

I just love that name. "Oh, would you like to hack a buffer that doesn't have a file associated with it to use directory local variables anyways?" When was this very specific function introduced? This program is full of mysteries.

;; .dir-locals.el

;; Good (enough)!
((nil
  (eval . (add-hook 'org-agenda-mode-hook
                  (lambda ()
                    (cd (with-current-buffer
                            (other-buffer (current-buffer) t)
                          default-directory))
                    (hack-dir-local-variables-non-file-buffer))))))

I recognize there are probably some edge cases here, I can't realistically test all possible window configurations, but generally, other-buffer seems to be a reliable way to get the buffer that launched the agenda.

So, that is basically all you need to tailor Org Agenda views depending on which folder you are in. Now, what modifications should you actually make? The obvious one is to update the variable org-agenda-files to only use files local to the directory.

In order to keep my promise to myself that these settings will be independent of the location of the directory, the org-agenda-files must be specified as relative paths. In order to accomplish that, all files need a common understanding of what the root directory is, or where they are in relation to it. Otherwise, a relative path might mean two different things to two files that are in different sub-directories.

One way to do it is to simply not deal with that level of complexity. You can limit the scope of directory-local variables to only include a single directory non-recursively. That way, each file will have a common root for relative paths, because they are all guaranteed to be in that root folder.

In a dir-locals file, the directory is represented by a string, and the current directory is represented by an empty string "". So, for the simplest case of just using all Org files in the current directory as agenda files, you can add this file to the folder:

;; .dir-locals.el

((""
   ;; Settings for files in current directory, not recursive
  (nil
   (eval . (add-hook 'org-agenda-mode-hook
                   (lambda ()
                     (cd (with-current-buffer
                             (other-buffer (current-buffer) t)
                           default-directory))
                     (hack-dir-local-variables-non-file-buffer))))
   ;; Add all Org files in current directory to org-agenda-files
   (org-agenda-files . (".")))))

Of course, any number of other settings related to Org Mode and agendas can be put here, such as specifying custom todo keywords or the behavior of blocked tasks, and they will apply to both Org Mode buffers and Org Agenda buffers launched from within that folder.

Emacs is a pretty lightweight dependency, all things considered, and some clever Makefile targets or package.json scripts could launch Emacs with pre-configured agenda views. Running this command from a folder with a .dir-locals.el like the one above will show all outstanding todo items in the folder:

emacs -f hack-dir-local-variables-non-file-buffer -f org-todo-list

Yes, that's right, it's our friend hack-dir-local-variables-non-file-buffer, one last time! Because Emacs was launched without a file name specified, the directory-local variables won't be loaded by default. So the agenda function, either org-agenda-list or org-todo-list, will have to be specified after this beautifully named function.

And that's pretty much all I have to say about it! There are a lot of other ways to customize the agenda to your projects' needs. While I'm not a huge fan of adding a global hook, the return value is pretty great. Maybe someday I could try to extend Org Agenda to have better built-in support for this kind of usage, but that is a little more involved than a 10-line config file.

Anyways, I hope this is useful, and I hope more people share agendas with each other. It really is a great way to keep track of everything in a collaborative way.