o i e0[@sdZddlmmZddlmZddlmZddlm Z ddlm Z ddl m Z edGd d d ej jjZd d ZGd ddejZddZddZddZdS)z9Library for exporting inference-only Keras models/layers.N) keras_export) base_layer) functional) sequential)io_utilszkeras.export.ExportArchivec@s^eZdZdZddZejjjddZ dddZ d d Z dd d Z d dZ ddZddZdS) ExportArchivea ExportArchive is used to write SavedModel artifacts (e.g. for inference). If you have a Keras model or layer that you want to export as SavedModel for serving (e.g. via TensorFlow-Serving), you can use `ExportArchive` to configure the different serving endpoints you need to make available, as well as their signatures. Simply instantiate an `ExportArchive`, use `track()` to register the layer(s) or model(s) to be used, then use the `add_endpoint()` method to register a new serving endpoint. When done, use the `write_out()` method to save the artifact. The resulting artifact is a SavedModel and can be reloaded via `tf.saved_model.load`. Examples: Here's how to export a model for inference. ```python export_archive = ExportArchive() export_archive.track(model) export_archive.add_endpoint( name="serve", fn=model.call, input_signature=[tf.TensorSpec(shape=(None, 3), dtype=tf.float32)], ) export_archive.write_out("path/to/location") # Elsewhere, we can reload the artifact and serve it. # The endpoint we added is available as a method: serving_model = tf.saved_model.load("path/to/location") outputs = serving_model.serve(inputs) ``` Here's how to export a model with one endpoint for inference and one endpoint for a training-mode forward pass (e.g. with dropout on). ```python export_archive = ExportArchive() export_archive.track(model) export_archive.add_endpoint( name="call_inference", fn=lambda x: model.call(x, training=False), input_signature=[tf.TensorSpec(shape=(None, 3), dtype=tf.float32)], ) export_archive.add_endpoint( name="call_training", fn=lambda x: model.call(x, training=True), input_signature=[tf.TensorSpec(shape=(None, 3), dtype=tf.float32)], ) export_archive.write_out("path/to/location") ``` **Note on resource tracking:** `ExportArchive` is able to automatically track all `tf.Variables` used by its endpoints, so most of the time calling `.track(model)` is not strictly required. However, if your model uses lookup layers such as `IntegerLookup`, `StringLookup`, or `TextVectorization`, it will need to be tracked explicitly via `.track(model)`. Explicit tracking is also required if you need to be able to access the properties `variables`, `trainable_variables`, or `non_trainable_variables` on the revived archive. cCs*g|_i|_tj|_g|_g|_g|_dSN)_endpoint_names_endpoint_signaturestf __version__tensorflow_version variablestrainable_variablesnon_trainable_variablesselfrJ/var/www/myenv/lib/python3.10/site-packages/keras/src/export/export_lib.py__init__]s  zExportArchive.__init__cCst|tjjjstdt|d|t|tjr!|j s!tdt |ds)g|_ |j |t|tjrO|j |j 7_ |j|j7_|j|j7_dSdS)z;Track the variables (and other assets) of a layer or model.zInvalid resource type. Expected an instance of a TensorFlow `Trackable` (such as a Keras `Layer` or `Model`). Received instead an object of type 'z'. Object received: zJThe layer provided has not yet been built. It must be built before export._trackedN) isinstancer __internal__tracking Trackable ValueErrortyperLayerbuilthasattrrappendrrr)rresourcerrrtrackes*    zExportArchive.trackNcCs||jvr td|d|rtj||d}||j|<nt|tjjjr3| s0td|d|}ntdt ||||j |dS)aRegister a new serving endpoint. Arguments: name: Str, name of the endpoint. fn: A function. It should only leverage resources (e.g. `tf.Variable` objects or `tf.lookup.StaticHashTable` objects) that are available on the models/layers tracked by the `ExportArchive` (you can call `.track(model)` to track a new model). The shape and dtype of the inputs to the function must be known. For that purpose, you can either 1) make sure that `fn` is a `tf.function` that has been called at least once, or 2) provide an `input_signature` argument that specifies the shape and dtype of the inputs (see below). input_signature: Used to specify the shape and dtype of the inputs to `fn`. List of `tf.TensorSpec` objects (one per positional input argument of `fn`). Nested arguments are allowed (see below for an example showing a Functional model with 2 input arguments). Example: Adding an endpoint using the `input_signature` argument when the model has a single input argument: ```python export_archive = ExportArchive() export_archive.track(model) export_archive.add_endpoint( name="serve", fn=model.call, input_signature=[tf.TensorSpec(shape=(None, 3), dtype=tf.float32)], ) ``` Adding an endpoint using the `input_signature` argument when the model has two positional input arguments: ```python export_archive = ExportArchive() export_archive.track(model) export_archive.add_endpoint( name="serve", fn=model.call, input_signature=[ tf.TensorSpec(shape=(None, 3), dtype=tf.float32), tf.TensorSpec(shape=(None, 4), dtype=tf.float32), ], ) ``` Adding an endpoint using the `input_signature` argument when the model has one input argument that is a list of 2 tensors (e.g. a Functional model with 2 inputs): ```python model = keras.Model(inputs=[x1, x2], outputs=outputs) export_archive = ExportArchive() export_archive.track(model) export_archive.add_endpoint( name="serve", fn=model.call, input_signature=[ [ tf.TensorSpec(shape=(None, 3), dtype=tf.float32), tf.TensorSpec(shape=(None, 4), dtype=tf.float32), ], ], ) ``` This also works with dictionary inputs: ```python model = keras.Model(inputs={"x1": x1, "x2": x2}, outputs=outputs) export_archive = ExportArchive() export_archive.track(model) export_archive.add_endpoint( name="serve", fn=model.call, input_signature=[ { "x1": tf.TensorSpec(shape=(None, 3), dtype=tf.float32), "x2": tf.TensorSpec(shape=(None, 4), dtype=tf.float32), }, ], ) ``` Adding an endpoint that is a `tf.function`: ```python @tf.function() def serving_fn(x): return model(x) # The function must be traced, i.e. it must be called at least once. serving_fn(tf.random.normal(shape=(2, 3))) export_archive = ExportArchive() export_archive.track(model) export_archive.add_endpoint(name="serve", fn=serving_fn) ``` zEndpoint name 'z' is already taken.)input_signaturezThe provided tf.function 'z' has never been called. To specify the expected shape and dtype of the function's arguments, you must either provide a function that has been called at least once, or alternatively pass an `input_signature` argument in `add_endpoint()`.anIf the `fn` argument provided is not a `tf.function`, you must provide an `input_signature` argument to specify the shape and dtype of the function arguments. Example: export_archive.add_endpoint( name='call', fn=model.call, input_signature=[ tf.TensorSpec( shape=(None, 224, 224, 3), dtype=tf.float32, ) ], )N) r rr functionr rtypes experimentalGenericFunction_list_all_concrete_functionssetattrr )rnamefnr# decorated_fnrrr add_endpoints k    zExportArchive.add_endpointcCsjt|tttfstdt|dtdd|Ds+tdttdd|Dt||t|dS)aRegister a set of variables to be retrieved after reloading. Arguments: name: The string name for the collection. variables: A tuple/list/set of `tf.Variable` instances. Example: ```python export_archive = ExportArchive() export_archive.track(model) # Register an endpoint export_archive.add_endpoint( name="serve", fn=model.call, input_signature=[tf.TensorSpec(shape=(None, 3), dtype=tf.float32)], ) # Save a variable collection export_archive.add_variable_collection( name="optimizer_variables", variables=model.optimizer.variables) export_archive.write_out("path/to/location") # Reload the object revived_object = tf.saved_model.load("path/to/location") # Retrieve the variables optimizer_variables = revived_object.optimizer_variables ``` zNExpected `variables` to be a list/tuple/set. Received instead object of type 'z'.css|] }t|tjVqdSr)rr Variable.0vrrr 8sz8ExportArchive.add_variable_collection..zgExpected all elements in `variables` to be `tf.Variable` instances. Found instead the following types: css|]}t|VqdSr)rr/rrrr2<sN)rlisttuplesetrrallr))rr*rrrradd_variable_collectionsz%ExportArchive.add_variable_collectioncsjstdi}jD] }|||<qdjvr)jd|d<tjj|||ddfddjD}t d|d |d S) aWrite the corresponding SavedModel to disk. Arguments: filepath: `str` or `pathlib.Path` object. Path where to save the artifact. options: `tf.saved_model.SaveOptions` object that specifies SavedModel saving options. **Note on TF-Serving**: all endpoints registered via `add_endpoint()` are made visible for TF-Serving in the SavedModel artifact. In addition, the first endpoint registered is made visible under the alias `"serving_default"` (unless an endpoint with the name `"serving_default"` was already registered manually), since TF-Serving requires this endpoint to be set. z4No endpoints have been set yet. Call add_endpoint().serving_defaultr)options signaturesz c3s |] }tt||VqdSr)_print_signaturegetattrr0r*rrrr2bs  z*ExportArchive.write_out..zSaved artifact at 'z+'. The following endpoints are available: N) r r_filter_and_track_resources_get_concrete_fnr saved_modelsavejoinr print_msg)rfilepathr9r:r* endpointsrrr write_out@s.  zExportArchive.write_outcCs4||jvr t||St||d}t|dS)z&Workaround for some SavedModel quirks.r@r)r r<_trackable_childrenr3values)rendpointtracesrrrr?ls  zExportArchive._get_concrete_fncsfddjD}t|S)Ncg|]}|qSrr?r=rrr uzBExportArchive._get_variables_used_by_endpoints..)r _list_variables_used_by_fns)rfnsrrr _get_variables_used_by_endpointstsz.ExportArchive._get_variables_used_by_endpointscsfddjD}t|\}}t||_g_ddlm}tdrCjD]}t j | }|D] }t ||rAj|q4q(dSdS)zBTrack resources used by endpoints / referenced in `track()` calls.crKrrLr=rrrrM{rNz=ExportArchive._filter_and_track_resources..r) IndexLookuprN)r rOr3_all_variables _misc_assets+keras.src.layers.preprocessing.index_lookuprRrrr train TrackableView descendantsrr )rrPtvsntvsrRrootrX trackablerrrr>xs      z)ExportArchive._filter_and_track_resourcesr)__name__ __module__ __qualname____doc__rr rr no_automatic_dependency_trackingr"r-r7rFr?rQr>rrrrrsA   *, rcCst}||t|tjtjfr0tj t |j }t|t r't |dkr'|g}|d|j|n|}|s:td|g}|d|j|||dS)NservezSThe model provided has never called. It must be called at least once before export.)rr"rr Functionalr Sequentialr nest map_structure_make_tensor_specinputsr3lenr-__call___get_save_specrrF)modelrDexport_archiver# save_specrrr export_models rpcsJeZdZdZ     dfdd ZddZdd d Zfd d ZZS) ReloadedLayerasReload a Keras model/layer that was saved via SavedModel / ExportArchive. Arguments: filepath: `str` or `pathlib.Path` object. The path to the SavedModel. call_endpoint: Name of the endpoint to use as the `call()` method of the reloaded layer. If the SavedModel was created via `model.export()`, then the default endpoint name is `'serve'`. In other cases it may be named `'serving_default'`. Example: ```python model.export("path/to/artifact") reloaded_layer = ReloadedLayer("path/to/artifact") outputs = reloaded_layer(inputs) ``` The reloaded object can be used like a regular Keras layer, and supports training/fine-tuning of its trainable weights. Note that the reloaded object retains none of the internal structure or custom methods of the original object -- it's a brand new layer created around the saved function. **Limitations:** * Only call endpoints with a single `inputs` tensor argument (which may optionally be a dict/tuple/list of tensors) are supported. For endpoints with multiple separate input tensor arguments, consider subclassing `ReloadedLayer` and implementing a `call()` method with a custom signature. * If you need training-time behavior to differ from inference-time behavior (i.e. if you need the reloaded object to support a `training=True` argument in `__call__()`), make sure that the training-time call function is saved as a standalone endpoint in the artifact, and provide its name to the `ReloadedLayer` via the `call_training_endpoint` argument. rcNTc s$tj|||dtj||_||_||_||_t |j|r't |j||_ n||jj vr5|jj ||_ nt d|d|rct |j|rMt |j||_n||jj vr[|jj ||_nt d|d|j g}|ro||jt|\}} |D] } |j| ddqw| D] } |j| ddqd|_dS)N) trainabler*dtypezThe endpoint 'zy' is neither an attribute of the reloaded SavedModel, nor an entry in the `signatures` field of the reloaded SavedModel. T)rrF)superrr r@load _reloaded_objrD call_endpointcall_training_endpointrr<call_endpoint_fnr:rcall_training_endpoint_fnr rO_add_existing_weightr) rrDrwrxrrr*rsall_fnsrYrZr1 __class__rrrsB          zReloadedLayer.__init__cs&|jjjj|fddddS)zACalls add_weight() to register but not create an existing weight.csSrr)___weightrr sz4ReloadedLayer._add_existing_weight..)r*shapersrrgetterN) add_weightr*rrs)rrrrrrrr{s  z"ReloadedLayer._add_existing_weightFcKs.|r|jr|j|fi|S|j|fi|Sr)rxrzry)rritrainingkwargsrrrcallszReloadedLayer.callcs(t}|j|j|jd}i||S)N)rDrwrx)rt get_configrDrwrx)r base_configconfigr}rrrs  zReloadedLayer.get_config)rcNTNN)F) r]r^r_r`rr{rr __classcell__rrr}rrqs)= rqcCstj|j|j|jdS)N)rsr*)r TensorSpecrrsr*)xrrrrhsrhcCsJ|d}|jdd}|d}d|dg|dd}d|}|S)NrT)verbose z * Endpoint ''rb)r(pretty_printed_signaturesplitrB)r+r* concrete_fnpprinted_signaturelinesrIrrrr;#s    r;c Csg}g}t}t}|D]T}t|dr|j}nt|dr"|g}n|g}|D]8}|jD]}t||vr@|||t|q,|jD]}t||vr^t||vr^|||t|qDq'q ||fS)Nconcrete_functionsget_concrete_function) r5rrrridr addr) rPrrtrainable_variables_idsnon_trainable_variables_idsr+rrr1rrrrO,s2           rO)r`tensorflow.compat.v2compatv2r tensorflow.python.util.tf_exportrkeras.src.enginerrrkeras.src.utilsrrr AutoTrackablerrprrqrhr;rOrrrrs      s