Packing and Functors

Date: 2011-08-10
Category: OCaml
Tags: ocaml, tooling



We have recently worked on modifying the OCaml system to be able to pack a set of modules within a functor, parameterized on some signatures. This page presents this work, funded by Jane Street.

All the patches on this page are provided for OCaml version 3.12.1.

Packing Functors

Installation of the modified OCaml system

The patch for OCaml 3.12.1 is available here:

ocaml+libfunctor-3.12.1.patch.gz (26 kB)

To use it, you can use the following recipe, that will compile and install the patched version in ~/ocaml+libfunctor-3.12.1/bin/.

~% wget http://caml.inria.fr/pub/distrib/ocaml-3.12/ocaml-3.12.1.tar.gz
~% tar zxf ~/ocaml-3.12.1.tar.gz
~% cd ocaml-3.12.1
~/ocaml-3.12.1% wget ocamlpro.com/code/ocaml+libfunctor-3.12.1.patch.gz
~/ocaml-3.12.1% gzip -d ocaml+libfunctor-3.12.1.patch.gz
~/ocaml-3.12.1% patch -p1 < ocaml+libfunctor-3.12.1.patch
~/ocaml-3.12.1% ./configure –prefix ~/ocaml+libfunctor-3.12.1
~/ocaml-3.12.1% make coldstart
~/ocaml-3.12.1% make ocamlc ocamllex ocamltools
~/ocaml-3.12.1% make library-cross
~/ocaml-3.12.1% make bootstrap
~/ocaml-3.12.1% make all opt opt.opt
~/ocaml-3.12.1% make install
~/ocaml-3.12.1% cd ~
~% export PATH=$HOME/ocaml+libfunctor-3.12.1/bin:$PATH

Note that it needs to bootstrap the compiler, as the format of object files is not compatible with the one of ocaml-3.12.1.

Usage of the lib-functor patch.

Now that you are equiped with the new system, you can start using it. The lib-functor patch adds two new options to the compilers ocamlc and ocamlopt:

If the interface x.mli contains :

type t
val compare : t -> t -> int

and the files xset.ml and xmap.ml contain respectively :

module T = Set.Make(X)
module T = Map.Make(X)

Then :

~/test% ocamlopt -c -for-pack Xx -functor x.cmi xset.ml
~/test% ocamlopt -c -for-pack Xx -functor x.cmi xmap.ml
~/test% ocamlopt -pack-functor MakeSetAndMap -o xx.cmx xset.cmx xmap.cmx

will construct a compiled unit whose signature is (that you can get with ocamlopt -i xx.cmi, see below) :

module MakeSetAndMap :
functor (X : sig type t val compare : t -> t -> int end) -> sig
  module Xset : sig
    module T : sig
      type elt = X.t
      type t = Set.Make(X).t
      val empty : t
      val is_empty : t -> bool
      …
    end
  end
  module Xmap : sig
    module T : sig
      type key = X.t
      type ‘a t = ‘a Map.Make(X).t
      val empty : ‘a t
      val is_empty : ‘a t -> bool
      …
    end
  end
end

Other extension: printing interfaces

OCaml only allows you to print the interface of a module or interface by compiling its source with the -i option. However, you don’t always have the source of an object interface (in particular, if it was generated by packing), and you might still want to do it.

In such a case, the lib-functor patch allows you to do that, by using the -i option on an interface object file:

~/test% cat > a.mli
val x : int
~/test% ocamlc -c -i a.mli
val x : int
~/test% ocamlc -c -i a.cmi
val x : int

Other extension: packing interfaces

OCaml only allows you to pack object files inside another object file (.cmo or .cmx). When doing so, you can either provide an source interface (.mli) that you need to compile to provide the corresponding object interface (.cmi), or the object interface will be automatically generated by exporting all the sub-modules within the packed module.

However, sometimes, you would want to be able to specify the interfaces of each module separately, so that:

In such a case, the lib-functor patch allows you to do that, by using the -pack option on interface object files:

test% cat > a.mli
val x : int
test% cat > b.mli
val y : string
test% ocamlc -c a.mli b.mli
test% ocamlc -pack -o c.cmi a.cmi b.cmi
test% ocamlc -i c.cmi
module A : sig val x : int end
module B : sig val y : string end

Using ocp-pack to pack source files

Installation of ocp-pack

Download the source file from:

ocp-pack-1.0.1.tar.gz (20 kB, GPL Licence, Copyright OCamlPro SAS)

Then, you just need to compile it with:

~% tar zxf ocp-pack-1.0.1.tar.gz
~% cd ocp-pack-1.0.1
~/ocp-pack-1.0.1% make
~/ocp-pack-1.0.1% make install

Usage of ocp-pack

ocp-pack can be used to pack source files of modules within just one source file. It allows you to avoid the use of the -pack option, that is not always supported by all ocaml tools (for example, ocamldoc). Moreover, ocp-pack tries to provide the correct locations to the compiler, so errors are not reported within the generated source file, but within the original source files.

It supports the following options:

% ocp-pack -help
Usage:
ocp-pack -o target.ml [options] files.ml*

Options:
-o <filename.ml> generate filename filename.ml
-rec use recursive modules
all .ml files must have a corresponding .mli file
-pack-functor <modname> create functor with name <modname>
-functor <filename.mli> use filename as an argument for functor
-mli output the .mli file too
.ml files without .mli file will not export any value
-no-ml do not output the .ml file
-with-ns use directory structure to create a hierarchy of modules
-v increment verbosity
–version display version information

ocp-pack automatically detects interface sources and implementation sources. When only the interface source is available, it is assumed that it is a type-only module, i.e. no val items are present inside.

Here is an example of using ocp-pack to build the ocamlgraph package:

test% ocp-pack -o graph.ml \
lib/bitv.ml lib/heap.ml lib/unionfind.ml \
src/sig.mli src/dot_ast.mli src/sig_pack.mli \
src/version.ml src/util.ml src/blocks.ml \
src/persistent.ml src/imperative.ml src/delaunay.ml \
src/builder.ml src/classic.ml src/rand.ml src/oper.ml \
src/path.ml src/traverse.ml src/coloring.ml src/topological.ml \
src/components.ml src/kruskal.ml src/flow.ml src/graphviz.ml \
src/gml.ml src/dot_parser.ml src/dot_lexer.ml src/dot.ml \
src/pack.ml src/gmap.ml src/minsep.ml src/cliquetree.ml \
src/mcs_m.ml src/md.ml src/strat.ml
test% ocamlc -c graph.ml
test% ocamlopt -c graph.ml

The -with-ns option can be used to automatically build a hierarchy of modules. With that option, sub-directories are seen as sub-modules. For example, packing a/x.ml, a/y.ml and b/z.ml will give a result like:

[code language=”fsharp”] module A = struct module X = struct … end module Y = struct … end end module B = struct module Z = struct … end end [/code]

Packing modules as functors

The -pack-functor and -functor options provide the same behavior as the same options with the lib-functor patch. The only difference is that -functor takes the interface source as argument, not the interface object.

Packing recursive modules

When trying to pack modules with ocp-pack, you might discover that your toplevel modules have recursive dependencies. This is usually achieved by types declared abstract in the interfaces, but depending on each other in the implementations. Such modules cannot simply packed by ocp-pack.

To handle them, ocp-pack provides a -rec option. With that option, modules are put within a module rec construct, and are all required to be accompagnied by an interface source file.

Moreover, in many cases, OCaml is not able to compile such recursive modules:

To solve these two issues in most cases, you can use the following patch (you can apply it using the same recipe as for lib-functor, and even apply both patches on the same sources):

With this patch, recursive modules are typed in an environment that is enriched progressively with the final types of the modules as soon as they become available. Also, during code generation, a topological order is computed on the recursive modules, and the subset of modules that can be initialized using in that topological order are immediatly generated, leaving only the other modules to be reordered.



About OCamlPro:

OCamlPro is a R&D lab founded in 2011, with the mission to help industrial users benefit from state-of-the art programming languages like OCaml and Rust.

We design, create and implement custom ad-hoc software for our clients. We also have a long experience in developing and maintaining open-source tooling for OCaml, such as Opam, TryOCaml, ocp-indent, ocp-index and ocp-browser, and we contribute to the core-development of OCaml, notably with our work on the Flambda optimizer branch.

Another area of expertise is that of Formal Methods, with tools such as our SMT Solver Alt-Ergo (check our Alt-Ergo Users'). We also provide vocational trainings in OCaml and Rust, and we can build courses on formal methods on-demand. Do not hesitate to reach out by email: contact@ocamlpro.com.