Declarative Magic
A colleague suggested that we should add to our compile.txt flow an ability for the tool
to automatically identify any folder path in the compile.txt file that contains the text /sim_lib/ as a simulation library path and do some under-the-hood handling (e.g. copying modelsim.ini) in order to make it work.
This immediately set off my "hmm that seems to magic for me" sensor.
However, I couldn't express why it seems to grate on me. So I decided to compose a post to think about it.
So, do I dislike this, or not?
The first thing I did was google "declarative vs magic" and read an AI Slop definition of declarative programming:
Declarative programming is a programming paradigm that uses "magic" code to achieve a desired
state, rather than defining the exact steps to get there.
I guess I like that idea okay. I don't like this definition one bit though. The AI Slop generator went on to add another definition:
In declarative programming, the programmer describes what they want to achieve, rather than how
to achieve it. The system then figures out how to get there.
The funny thing is, my question is not about programming really. We're doing the programming imperatively under the hood. So this is maybe about "declarative configuration?"
P.S. the extreme of declarative programming is to invent new domain specific languages (DSL)
https://stackoverflow.com/questions/1784664/what-is-the-difference-between-declarative-and-imperative-paradigm-in-programmin
But I actually might be okay with creating a domain specific language. We're not quite doing that
either.
I keep having the word "discoverability" jump into my head. It seems to me that having our code
look for /sim_lib/ in the compile.txt file is not something anyone is going to be able to discover.
That's the magic that rubs me the wrong way, I think. But why is that?
I think a more concrete example might be in order:
Let's look at what this compile.txt can do right now, and what has changed?
One thing right off the bat is that the compile.txt file used to only contain files. Now it contains paths as well. It used to be that having a path that is not a file would raise an error. Now it won't. That's a big change.
The original idea of the compile.txt was to explicitly specify the files to be compiled and their order. This was such that we didn't have to depend on any under the hood magic of modelsim, ISE, Vivado, etc. in determining dependencies.
But then we added a bunch of pragma style commands started with a backtick ("`"). E.g. "if sim","end_default_library","end","xil_sim_lib_for_msim", "default_library"
Clearly these are not discoverable, because I didn't even know we had default_library and
end_default_library in there.
Also, these backtick commands are essentially the definition of a DSL, aren't they? Is that cool? Is that the best solution?
I think that's a separate problem, and touches on the need for good documentation if you're trying
to implement a new DSL. Not only do you need good documentation, you also need good discoverability
of your documentation.
Next, thought: the overview of this PR says what we are changing in the PR, not why we are doing it.
One thing we are doing is removing the old `xil_sim_lib_for_msim backtick command. What exactly was wrong with this command? What were it's benefits? Why is it a problem?
The old behavior hid some magic under the hood. The 'xil_sim_lib_for_msim accepted a relative path
for the library folder. Under the hood it would know to check both /opt/sim_lib and /misc/sim_lib
as the base for these relative paths.
I think the idea there was that we could add additional magic if the location changed or if new
tools required other special handling. But in the end it maybe just made it harder to debug when
the files didn't exist and things failed.
But why not just change it to support (or require) an absolute path for `xil_sim_lib_for_msim? I mean, isn't it modelsim-library-only magic anyway? Other types of paths aren't supported anyway. Right?
Now, the DSL no longer has an explicit command `xil_sim_lib_for_msim but instead infers it from an element in the path. But it treats it effectively the same way it used to.
Is that what bothers me about this? I prefer explicit magic? I don't like implicit magic?
I'm totally okay with this process doing magic things behind the hood. The magic doing these things is imperatively programmed, and the source is there to inspect. So it's not really magic at all. You see the `backtick command you grep the code to find this backtick, you can see what it does. (Documentation and discoverability notwithstanding.)
But if the code magically calls this functionality based on string matching in the compile.txt file, then it's no longer clear that magic is going on. Anywhere in the code I could accidentally use a path that matches the magic string and suddenly it will change the behavior. It's not explicit, it's implicit. e.g.
The old way:
`xil_sim_lib_for_msim Xilinx/sim_library_1234/
relative/adder.vhd
relative/comparator.vhd
relative/mux.v
other.v
top.vhd
The new way
/misc/sim_lib/Xilinx/sim_library_1234/
relative/adder.vhd
relative/comparator.vhd
relative/mux.v
other.v
top.vhd
Ah, wait! This makes clear another issue that might rub me the wrong way: relative paths. Essentially the `xil_sim_lib_for_msim is standing in for an environment variable that sets the correct base path, e.g. in a bash style:
XIL_SIM_LIB_FOR_MSIM=/misc/sim_lib
echo $XIL_SIM_LIB_FOR_MSIM/Xilinx/sim_library_1234/
The old explicit `xil_sim_lib_for_msim allowed the library to handle the possibility of different install locations on different OS's, different machines, or different tools. And it made it evident that some magic was happening for this special case, and why (the string msim is pretty clearly calling out a modelsim specific line of magic).
But the new way is now locked down to a specific path. That prevents flexibility in the future. But maybe changing paths is just a hypothetical concern?
So why is it bad to have an absolute path in a file that is by-default relative? Why does this rub me the wrong way? Is it simply not elegant?
How should it be done?
Maybe split the difference? Maybe:
`xil_sim_lib_for_msim /misc/some_lib/Xilinx/sim_library_1234/
relative/adder.vhd
relative/comparator.vhd
relative/mux.v
other.v
top.vhd
This indicates yet another two oddities:
First, I put in some_lib instead of sim_lib, but it should handle this fine. It's explicit now, which is good, as there is no magic matching on the phrase sim_lib, and thus no artificial requirement for the location of the path. That's an improvement, I think, as it is more universal.
Second, it concerns me that if we have to move the installed library in the future, e.g. to support multiple incompatible versions, that this specific line will become problematic, and building an old project will fail and will be difficult to debug.
Third, it makes me question what the behavior of a relative path. What should it do in this case?
`xil_sim_lib_for_msim Xilinx/sim_library_1234/
relative/adder.vhd
relative/comparator.vhd
relative/mux.v
other.v
top.vhd
It should then look in the current directory, right? That's probably sane, and could handle nicely the case of a legacy library that has been archived with the project.
Shouldn't the framework, the environment, and the tools handle the location of the simulation library for the user? Why should the user need to know the exact path of the simulation libraries? Why is this in this file at all? Most synthesis/simulation tools configure this separately.