Matlab Makefile

http://programmers.stackexchange.com/questions/188528/how-do-i-transparently-cache-data-in-intermediate-stages-of-processing

What you are describing in your ideal solution is very similar to what is provided by the make programand makefiles. A makefile essentially expresses a dependency graph from a set of output files, through a set of intermediate files, to a set of input files, along with commands to transform a file at one step to the next.

Inferring names for the various functions you mention above, you might get something like this:

ckt.mat : ckt_properties.mat
    matlab -r generate_ckt.m ckt_properties.mat

freq.mat : fconfig.mat
    matlab -r generate_fpoints.m fconfig.mat

result.mat : ckt.mat freq.mat
    matlab -r freq_dom_sim.m ckt.mat freq.mat

red_ckt.mat : ckt.mat red_order.mat
    matlab -r run_PRIMA.m ckt.mat red_order.mat

This says that ckt.mat depends on ckt_properties.mat, and you can generate ckt.mat when you need to by running matlab generate_ckt.m ckt_properties.mat on the command line. "When you need to" means when the modification time of the source (ckt_properties.mat) is newer than that of the target (ckt.mat).

Now maybe you can do everything with files and makefiles, but this keeps you largely outside of Matlab's IDE. You could also do something purely within Matlab by creating a structure that mimics the aspects of the filesystem that make relies upon, namely file names, modification times, and contents. In other words, create structures that bind a matrix and a modification time (perhaps held as a simple scalar) under a name. Then you would need another structure that encodes the dependency relationships, which is essentially a list of tuples containing a target structure, a list of source structures, and a transformation function. All this is doable (and might even have been done, I don't know), but it might be easier to just use makefiles.

shareimprove this answer

answered Feb 27 at 2:11 Randall Cook 1,171514

	Awesome answer, thanks! I knew about makefiles, but my friend told me its not really the same thing, so I didn't mention it. Just one more small issue. Would it be easier/better to compare size of matrices rather than date of modification of the .mat file? – Milind R Feb 27 at 3:10
	Also, would it be possible to invoke functions using makefiles? – Milind R Feb 27 at 3:11
	Ok I found out what I needed to. But I don't understand how you pass fconfig.mat as a parameter togenerate_fpoints.m. Unable to find a reference for that usage. – Milind R Feb 27 at 5:32
	You basically need to recompute derived objects when their dependent source objects change. Looking for changes in size is one approach, but can miss things. For example, changing the value of a scalar does not change its size. Looking at the modification time is more reliable, provided it is updated on every change. Computing the checksum of an object is probably the best of all, but can be difficult to compute for structured data like matrices. It is best done with binary blobs of data. – Randall Cook Feb 27 at 18:41
	Passing command-line arguments to matlab scripts is a little tricky. I delved into pseudocode when writing my answer. I did some digging and found a couple pages with some suggestions: quora.com/… andstackoverflow.com/questions/7958320/… – Randall Cook Feb 27 at 18:44