OpenVDB  12.0.0
VolumeExecutable.h
Go to the documentation of this file.
1 // Copyright Contributors to the OpenVDB Project
2 // SPDX-License-Identifier: Apache-2.0
3 
4 /// @file compiler/VolumeExecutable.h
5 ///
6 /// @authors Nick Avramoussis, Francisco Gochez, Richard Jones
7 ///
8 /// @brief The VolumeExecutable, produced by the OpenVDB AX Compiler for
9 /// execution over Numerical OpenVDB Grids.
10 ///
11 
12 #ifndef OPENVDB_AX_COMPILER_VOLUME_EXECUTABLE_HAS_BEEN_INCLUDED
13 #define OPENVDB_AX_COMPILER_VOLUME_EXECUTABLE_HAS_BEEN_INCLUDED
14 
15 #include "CustomData.h"
16 #include "AttributeRegistry.h"
17 #include "AttributeBindings.h"
18 
19 #include <openvdb/version.h>
20 #include <openvdb/Grid.h>
21 
22 #include <unordered_map>
23 
24 class TestVolumeExecutable;
25 
26 namespace llvm {
27 class ExecutionEngine;
28 class LLVMContext;
29 }
30 
31 namespace openvdb {
33 namespace OPENVDB_VERSION_NAME {
34 namespace ax {
35 
36 class Compiler;
37 
38 /// @brief Object that encapsulates compiled AX code which can be executed on a
39 /// collection of VDB volume grids. Executables are created by the compiler
40 /// and hold the final immutable JIT compiled function and context.
41 /// @details The VolumeExecutable is returned from the ax::Compiler when
42 /// compiling AX code for volume execution. The class represents a typical AX
43 /// executable object; immutable except for execution settings and implements
44 /// 'execute' functions which can be called multiple times for arbitrary sets
45 /// of inputs. The intended usage of these executables is to configure their
46 /// runtime arguments and then call VolumeExecutable::execute with your VDBs.
47 /// For example:
48 /// @code
49 /// VolumeExecutable::Ptr exe = compiler.compile<VolumeExecutable>("@a += 1");
50 /// exe->setTreeExecutionLevel(0); // only process leaf nodes
51 /// exe->setValueIterator(VolumeExecutable::IterType::ALL); // process all values
52 /// exe->execute(vdbs); // run on a set of vdbs
53 /// exe->execute(grid); // run on a single vdb
54 /// @endcode
55 ///
56 /// The Volume executable is initialised with specific configurable settings:
57 /// - Iteration Level: min=0, max=RootNode::Level.
58 /// By default, processes the entire VDB tree hierarchy.
59 /// @sa setTreeExecutionLevel
60 /// - Iteration Type: ON
61 /// By default, processes ACTIVE values.
62 /// @sa setValueIterator
63 /// - Active Tile Streaming: ON, OFF or AUTO depending on AX code.
64 /// By default, if AX detects that the AX program may produce unique
65 /// values for leaf level voxels that would otherwise comprise a
66 /// given active tile, this setting is set to ON or AUTO. Otherwise it is
67 /// set to OFF.
68 /// @sa setActiveTileStreaming
69 /// - Grain sizes: 1:32
70 /// The default grain sizes passed to the tbb partitioner for leaf level
71 /// processing and active tile processing.
72 /// @sa setGrainSize
73 /// @sa setActiveTileStreamingGrainSize
74 /// - AttributeBindings: None
75 /// Whether to indriect any AX accesses to different grid names.
76 /// @sa setAttributeBindings
77 ///
78 /// For more in depth information, see the @ref vdbaxcompilerexe documentation.
80 {
81 public:
82  using Ptr = std::shared_ptr<VolumeExecutable>;
84 
85  /// @brief Copy constructor. Shares the LLVM constructs but deep copies the
86  /// settings. Multiple copies of an executor can be used at the same time
87  /// safely.
88  VolumeExecutable(const VolumeExecutable& other);
89 
90  ////////////////////////////////////////////////////////
91 
92  ///@{
93  /// @brief Run this volume executable binary on target volumes.
94  /// @details This method reads from the stored settings on the executable
95  /// to determine certain behaviour and runs the JIT compiled function
96  /// across every valid VDB value. Topology may be changed, deleted or
97  /// created.
98  ///
99  /// This method is thread safe; it can be run concurrently from the same
100  /// VolumeExecutable instance on different inputs.
101  ///
102  /// @param grids The VDB Volumes to process
103  void execute(openvdb::GridPtrVec& grids) const;
104  void execute(openvdb::GridBase& grids) const;
105  ///@}
106 
107  ////////////////////////////////////////////////////////
108 
109  /// @brief Set the behaviour when missing grids are accessed. Default
110  /// behaviour is true, which creates them with default transforms and
111  /// background values
112  /// @param flag Enables or disables the creation of missing attributes
113  void setCreateMissing(const bool flag);
114  /// @return Whether this executable will generate new grids.
115  bool getCreateMissing() const;
116 
117  /// @brief Set the execution level for this executable. This controls what
118  /// nodes are processed when execute is called. Possible values depend on
119  /// the OpenVDB configuration in use, however a value of 0 will always
120  /// correspond to the lowest level (leaf-level). By default, the min
121  /// level is zero (LeafNodeType::LEVEL) and the max level is the root
122  /// node's level (RootNodeType::LEVEL). In other words, the default
123  /// execution level settings process the whole of the tree.
124  /// @note A value larger that the number of levels in the tree (i.e. larger
125  /// than the root node's level) will cause this method to throw a runtime
126  /// error.
127  /// @param min The minimum tree execution level to set
128  /// @param max The maximum tree execution level to set
129  void setTreeExecutionLevel(const Index min, const Index max);
130  /// @param level The tree execution level to set. Calls setTreeExecutionLevel
131  /// with min and max arguments as level.
132  void setTreeExecutionLevel(const Index level);
133  /// @brief Get the tree execution levels.
134  /// @param min The minimum tree execution level
135  /// @param max The maximum tree execution level
136  void getTreeExecutionLevel(Index& min, Index& max) const;
137 
138  /// @brief The streaming type of active tiles during execution.
139  /// @param ON active tiles are temporarily densified (converted to leaf
140  /// level voxels) on an "as needed" basis and the subsequent voxel
141  /// values are processed. The temporarily densified node is added to the
142  /// tree only if a non constant voxel buffer is produced. Otherwise a
143  /// child tile may be created or the original tile's value may simply be
144  /// modified.
145  /// @param OFF tile topologies are left unchanged and their single value is
146  /// processed.
147  /// @param AUTO the volume executable analyzes the compiled kernel and
148  /// attempts to determine if expansion of active tiles would lead to
149  /// different, non-constant values in the respective voxels. This is
150  /// done on a per grid basis; ultimately each execution will be set to
151  /// ON or OFF. This option will always fall back to ON if there is any
152  /// chance the kernel may produce child nodes
153  ///
154  /// @note The volume executable always runs an AUTO check on creation and
155  /// will set itself to ON (if all grids always need child nodes), OFF (if
156  /// grids never need child nodes) or remains as AUTO (if this depends on
157  /// which grid is being processed).
158  ///
159  /// @details When an AX kernel is run over coarser levels of the tree (i.e.
160  /// not leaf voxels), it is often desirable to densify these areas into
161  /// unique voxels such that they can each receive a unique value. For
162  /// example, consider the following AX code which assigns a vector volume
163  /// to the world space position of each voxel:
164  /// @code
165  /// v@v = getvoxelpws();
166  /// @endcode
167  /// Active tiles hold a single value but comprise an area greater than
168  /// that of a single voxel. As the above kernel varies with respect to
169  /// a nodes position, we'd need to replace these tiles with leaf voxels
170  /// to get unique per node values. The stream flag is initialised to ON
171  /// in this case.
172  ///
173  /// This behaviour, however, is not always desirable .i.e:
174  /// @code
175  /// v@v = {1,2,3};
176  /// @endcode
177  /// In this instance, all values within a volume receive the same value
178  /// and are not dependent on any spatially or iteratively varying
179  /// metrics. The stream flag is set to OFF.
180  ///
181  /// The AUTO flag is set in cases where the runtime access pattern of the
182  /// inputs determines streaming:
183  /// @code
184  /// f@density = f@mask;
185  /// f@mask = 0;
186  /// @endcode
187  /// In this instance, the runtime topology and values of \@mask determines
188  /// whether child topology needs to be created in \@density, but \@mask
189  /// itself does not need streaming. Streaming will be set to ON for
190  /// density but OFF for mask.
191  ///
192  /// @note This behaviour is only applied to active tiles. If the value
193  /// iterator is set to OFF, this option is ignored.
194  /// @warning This option can generate large amounts of leaf level voxels.
195  /// It is recommended to use a good concurrent memory allocator (such as
196  /// jemalloc) for the best performance.
197  enum class Streaming { ON, OFF, AUTO };
198  /// @brief Controls the behaviour of expansion of active tiles.
199  /// @param s The behaviour to set
200  void setActiveTileStreaming(const Streaming& s);
201  /// @return The current stream behaviour.
202  Streaming getActiveTileStreaming() const;
203  /// @return The current stream behaviour for a particular grid. This is
204  /// either ON or OFF.
205  /// @param name The name of the grid to query
206  /// @param type The grids type
207  Streaming getActiveTileStreaming(const std::string& name,
208  const ast::tokens::CoreType& type) const;
209 
210  enum class IterType { ON, OFF, ALL };
211  /// @brief Set the value iterator type to use with this executable. Options
212  /// are ON, OFF, ALL. Default is ON.
213  /// @param iter The value iterator type to set
214  void setValueIterator(const IterType& iter);
215  /// @return The current value iterator type
216  IterType getValueIterator() const;
217 
218  ///@{
219  /// @brief Set the threading grain sizes used when iterating over nodes
220  /// in a VDB.
221  /// @details Two grain sizes are provided, the first of which (g1) is used
222  /// to determine the chunk size of nodes which are not being streamed (see
223  /// setActiveTileStream). Leaf node execution always uses this grain size.
224  /// The default value for g1 is 1 which is typically appropriate for most
225  /// AX kernels.
226  /// The second grain size is used when streaming execution over active
227  /// tiles in a VDB. This execution model differs significantly from
228  /// typical leaf node execution due to the potential for substantially
229  /// more memory to be allocated. The default value is 32, which works well
230  /// for the default configuration of OpenVDB. If streaming is disabled,
231  /// this value has no effect.
232  /// @note Setting g1 or g2 to zero has the effect of disabling
233  /// multi-threading for the respective node executions. Setting both to
234  /// zero will disable all multi-threading performed by the execute method.
235  void setGrainSize(const size_t g1);
236  void setActiveTileStreamingGrainSize(const size_t g2);
237  /// @return The current g1 grain size
238  /// @sa setGrainSize
239  size_t getGrainSize() const;
240  /// @return The current g2 grain size
241  /// @sa setActiveTileStreamingGrainSize
242  size_t getActiveTileStreamingGrainSize() const;
243  ///@}
244 
245  /// @brief Set attribute bindings.
246  /// @param bindings A map of attribute bindings to expected names on
247  /// the geometry to be executed over. By default the AX attributes will be
248  /// bound to volumes of the same name. Supplying bindings
249  /// for a subset of the attributes will leave the others unchanged.
250  /// AX attributes can only bind to a single volume and vice versa.
251  /// However, in a single set call these can be swapped e.g. a -> b and b -> a.
252  /// When bindings are overriden through subsequent calls to this function,
253  /// any dangling volumes will be automatically bound by name.
254  /// To reset these bindings call get function and create a target set of bindings
255  /// for each attribute of name -> name.
256  void setAttributeBindings(const AttributeBindings& bindings);
257  /// @return The current attribute bindings map
258  const AttributeBindings& getAttributeBindings() const;
259 
260  ////////////////////////////////////////////////////////
261 
262  // foward declaration of settings for this executable
263  template <bool> struct Settings;
264 
265  /// @brief Command Line Interface handling for the VolumeExecutable.
266  /// @details This class wraps the logic for converting commands specific
267  /// to the VolumeExecutable to the internal Settings. Subsequent
268  /// executables can be initialized from the CLI object that gets created.
270  {
271  ~CLI();
272  CLI(CLI&&);
273  CLI& operator=(CLI&&);
274  static CLI create(size_t argc, const char* argv[], bool* used=nullptr);
275  static void usage(std::ostream& os, const bool verbose);
276  private:
277  friend class VolumeExecutable;
278  CLI();
279  std::unique_ptr<Settings<true>> mSettings;
280  };
281 
282  /// @brief Intialize the Settings of this executables from the CLI object
283  /// @param cli The CLI object
284  /// @{
285  void setSettingsFromCLI(const CLI& cli);
286  /// @}
287 
288  ////////////////////////////////////////////////////////
289 
290  /// @return The tree execution level.
291  OPENVDB_DEPRECATED_MESSAGE("Use getTreeExecutionLevel(Index&, Index&)")
292  Index getTreeExecutionLevel() const;
293 
294 private:
295  friend class Compiler;
296  friend class ::TestVolumeExecutable;
297 
298  /// @brief Constructor, expected to be invoked by the compiler. Should not
299  /// be invoked directly.
300  /// @param context Shared pointer to an llvm:LLVMContext associated with the
301  /// execution engine
302  /// @param engine Shared pointer to an llvm::ExecutionEngine used to build
303  /// functions. Context should be the associated LLVMContext
304  /// @param accessRegistry Registry of volumes accessed by AX code
305  /// @param customData Custom data which will be shared by this executable.
306  /// It can be used to retrieve external data from within the AX code
307  /// @param functions A map of function names to physical memory addresses
308  /// which were built by llvm using engine
309  /// @param tree The AST linked to this executable. The AST is not stored
310  /// after compilation but can be used during construction of the exe to
311  /// infer some pre/post processing optimisations.
312  VolumeExecutable(const std::shared_ptr<const llvm::LLVMContext>& context,
313  const std::shared_ptr<const llvm::ExecutionEngine>& engine,
314  const AttributeRegistry::ConstPtr& accessRegistry,
315  const CustomData::ConstPtr& customData,
316  const std::unordered_map<std::string, uint64_t>& functions,
317  const ast::Tree& tree);
318 
319 private:
320  // The Context and ExecutionEngine must exist _only_ for object lifetime
321  // management. The ExecutionEngine must be destroyed before the Context
322  const std::shared_ptr<const llvm::LLVMContext> mContext;
323  const std::shared_ptr<const llvm::ExecutionEngine> mExecutionEngine;
324  const AttributeRegistry::ConstPtr mAttributeRegistry;
325  const CustomData::ConstPtr mCustomData;
326  const std::unordered_map<std::string, uint64_t> mFunctionAddresses;
327  std::unique_ptr<Settings<false>> mSettings;
328 };
329 
330 } // namespace ax
331 } // namespace OPENVDB_VERSION_NAME
332 } // namespace openvdb
333 
334 #endif // OPENVDB_AX_COMPILER_VOLUME_EXECUTABLE_HAS_BEEN_INCLUDED
335 
Definition: Compiler.h:32
IterType
Definition: VolumeExecutable.h:210
std::shared_ptr< VolumeExecutable > Ptr
Definition: VolumeExecutable.h:82
#define OPENVDB_AX_API
Definition: Platform.h:289
Definition: VolumeExecutable.h:263
Definition: Coord.h:590
Index32 Index
Definition: Types.h:54
const std::enable_if<!VecTraits< T >::IsVec, T >::type & max(const T &a, const T &b)
Definition: Composite.h:110
Object that encapsulates compiled AX code which can be executed on a collection of VDB volume grids...
Definition: VolumeExecutable.h:79
The custom data class is a simple container for named openvdb metadata. Its primary use case is passi...
Definition: CustomData.h:33
std::vector< GridBase::Ptr > GridPtrVec
Definition: Grid.h:508
Streaming
The streaming type of active tiles during execution.
Definition: VolumeExecutable.h:197
Definition: IndexIterator.h:44
The Attribute Bindings class is used by the compiled Executables to handle the mapping of AX Attribut...
CoreType
Definition: Tokens.h:31
Definition: Exceptions.h:13
The compiler class. This holds an llvm context and set of compiler options, and constructs executable...
Definition: Compiler.h:50
These classes contain lists of expected attributes and volumes which are populated by compiler during...
Command Line Interface handling for the VolumeExecutable.
Definition: VolumeExecutable.h:269
Access to the CustomData class which can provide custom user user data to the OpenVDB AX Compiler...
This class stores a list of access names, types and their dependency connections. ...
Definition: AttributeRegistry.h:39
const std::enable_if<!VecTraits< T >::IsVec, T >::type & min(const T &a, const T &b)
Definition: Composite.h:106
#define OPENVDB_VERSION_NAME
The version namespace name for this library version.
Definition: version.h.in:121
#define OPENVDB_DEPRECATED_MESSAGE(msg)
Definition: Platform.h:148
#define OPENVDB_USE_VERSION_NAMESPACE
Definition: version.h.in:218
Abstract base class for typed grids.
Definition: Grid.h:77
This class wraps an interface for a map of attribute bindings. These map attributes in AX code to con...
Definition: AttributeBindings.h:36