o i ec@s:dZddlmZddlmZddlmZddlZddlZddlmm Z ddl m Z ddl m Z ddl mZdd lmZdd lmZdd lmZd Zd dZddZGdddeZedgdedgdGdddeZedgdedgdGdddeZedgdedgdGdddeZd d!ZdS)"z8Module implementing the V1 version of RNN cell wrappers.)absolute_import)division)print_functionN)_enumerated_map_structure_up_to)_parse_config_to_function)_serialize_function_to_config)RNNCell) keras_export) tf_exportzis not an RNNCellcCs&zt||WdStyYdSw)NFT)getattrAttributeError)obj attr_namerX/var/www/myenv/lib/python3.10/site-packages/keras/src/layers/rnn/legacy_cell_wrappers.py_hasattr(s   rcCslt|dt|dt|dpt|dt|g}gd}t|s4ddt||D}td||d |d S) aRaises a TypeError if cell is not like an RNNCell. NOTE: Do not rely on the error message (in particular in tests) which can be subject to change to increase readability. Use ASSERT_LIKE_RNNCELL_ERROR_REGEXP. Args: cell_name: A string to give a meaningful error referencing to the name of the functionargument. cell: The object which should behave like an RNNCell. Raises: TypeError: A human-friendly exception. output_size state_sizeget_initial_state zero_state)z!'output_size' property is missingz 'state_size' property is missingz=either 'zero_state' or 'get_initial_state' method is requiredzis not callablecSsg|]\}}|s|qSrr).0errorcondrrr Osz'assert_like_rnncell..z-The argument {!r} ({}) is not an RNNCell: {}.z, N)rcallableallzip TypeErrorformatjoin) cell_namecell conditionserrorsrrrassert_like_rnncell1s r$csleZdZdZfddZddZdddZed d Zed d Z d dZ fddZ e dddZ ZS)_RNNCellWrapperV1zBase class for cells wrappers V1 compatibility. This class along with `_RNNCellWrapperV2` allows to define cells wrappers that are compatible with V1 and V2, and defines helper methods for this purpose. csJtj|i|td|||_t|tjjjr#|j |jdddSdS)Nr!)name) super__init__r$r! isinstancetf __internal__tracking Trackable_track_trackable)selfr!argskwargs __class__rrr(_s  z_RNNCellWrapperV1.__init__cKst)aCalls the wrapped cell and performs the wrapping logic. This method is called from the wrapper's `call` or `__call__` methods. Args: inputs: A tensor with wrapped cell's input. state: A tensor or tuple of tensors with wrapped cell's state. cell_call_fn: Wrapped cell's method to use for step computation (cell's `__call__` or 'call' method). **kwargs: Additional arguments. Returns: A pair containing: - Output: A tensor with cell's output. - New state: A tensor or tuple of tensors with new wrapped cell's state. )NotImplementedErrorr/inputsstate cell_call_fnr1rrr_call_wrapped_cellfsz$_RNNCellWrapperV1._call_wrapped_cellNcCs|j|||jj|dS)aRuns the RNN cell step computation. We assume that the wrapped RNNCell is being built within its `__call__` method. We directly use the wrapped cell's `__call__` in the overridden wrapper `__call__` method. This allows to use the wrapped cell and the non-wrapped cell equivalently when using `__call__`. Args: inputs: A tensor with wrapped cell's input. state: A tensor or tuple of tensors with wrapped cell's state. scope: VariableScope for the subgraph created in the wrapped cells' `__call__`. Returns: A pair containing: - Output: A tensor with cell's output. - New state: A tensor or tuple of tensors with new wrapped cell's state. )r8scope)r9r!__call__)r/r6r7r:rrrr;zs z_RNNCellWrapperV1.__call__cC|jjSN)r!rr/rrrrz_RNNCellWrapperV1.state_sizecCr<r=)r!rr>rrrrr?z_RNNCellWrapperV1.output_sizecCsFtt|jd|j||WdS1swYdSN ZeroState)r* name_scopetype__name__r!rr/ batch_sizedtyperrrrs $z_RNNCellWrapperV1.zero_statecs@d|jjj|jdi}t}tt|t|S)Nr!) class_nameconfig)r!r3rD get_configr'dictlistitemsr/rI base_configr2rrrJs z_RNNCellWrapperV1.get_configcCsF|}|d}ztd|||fi|WSty"tdw)Nr!ztRNNCellWrapper cannot reconstruct the wrapped cell. Please overwrite the cell in the config with a RNNCell instance.)copypopr$r ValueError)clsrIcustom_objectsr!rrr from_configs   z_RNNCellWrapperV1.from_configr=)rD __module__ __qualname____doc__r(r9r;propertyrrrrJ classmethodrU __classcell__rrr2rr%Ws      r%z1keras.__internal__.legacy.rnn_cell.DropoutWrapper)v1znn.rnn_cell.DropoutWrappercseZdZdZ        dfdd ZddZed d Zd d Zd dZ dddZ ddZ fddZ e dfdd ZZS)DropoutWrapperz@Operator adding dropout to inputs and outputs of the given cell.?FNc  stj|fdi| | durt| std| | pt_tdMdd} |df|df|d ffD]5\} } | | \}}|dura|d ksK|d krUtd | d |t d| t |q5t d| |q5Wdn1suwY|_ |_ |_ d_d_d_|r߈durtdddfddtjtjrjdkr|durtdt|fdd|_t|jfdd|j_t|jfdd|j_dSdS)a}Create a cell with added input, state, and/or output dropout. If `variational_recurrent` is set to `True` (**NOT** the default behavior), then the same dropout mask is applied at every step, as described in: [A Theoretically Grounded Application of Dropout in Recurrent Neural Networks. Y. Gal, Z. Ghahramani](https://arxiv.org/abs/1512.05287). Otherwise a different dropout mask is applied at every time step. Note, by default (unless a custom `dropout_state_filter` is provided), the memory state (`c` component of any `LSTMStateTuple`) passing through a `DropoutWrapper` is never modified. This behavior is described in the above article. Args: cell: an RNNCell, a projection to output_size is added to it. input_keep_prob: unit Tensor or float between 0 and 1, input keep probability; if it is constant and 1, no input dropout will be added. output_keep_prob: unit Tensor or float between 0 and 1, output keep probability; if it is constant and 1, no output dropout will be added. state_keep_prob: unit Tensor or float between 0 and 1, output keep probability; if it is constant and 1, no output dropout will be added. State dropout is performed on the outgoing states of the cell. **Note** the state components to which dropout is applied when `state_keep_prob` is in `(0, 1)` are also determined by the argument `dropout_state_filter_visitor` (e.g. by default dropout is never applied to the `c` component of an `LSTMStateTuple`). variational_recurrent: Python bool. If `True`, then the same dropout pattern is applied across all time steps per run call. If this parameter is set, `input_size` **must** be provided. input_size: (optional) (possibly nested tuple of) `TensorShape` objects containing the depth(s) of the input tensors expected to be passed in to the `DropoutWrapper`. Required and used **iff** `variational_recurrent = True` and `input_keep_prob < 1`. dtype: (optional) The `dtype` of the input, state, and output tensors. Required and used **iff** `variational_recurrent = True`. seed: (optional) integer, the randomness seed. dropout_state_filter_visitor: (optional), default: (see below). Function that takes any hierarchical level of the state and returns a scalar or depth=1 structure of Python booleans describing which terms in the state should be dropped out. In addition, if the function returns `True`, dropout is applied across this sublevel. If the function returns `False`, dropout is not applied across this entire sublevel. Default behavior: perform dropout on all terms except the memory (`c`) state of `LSTMCellState` objects, and don't try to apply dropout to `TensorArray` objects: ``` def dropout_state_filter_visitor(s): # Never perform dropout on the c state. if isinstance(s, LSTMCellState): return LSTMCellState(c=False, h=True) elif isinstance(s, TensorArray): return False return True ``` **kwargs: dict of keyword arguments for base layer. Raises: TypeError: if `cell` is not an `RNNCell`, or `keep_state_fn` is provided but not `callable`. ValueError: if any of the keep_probs are not between 0 and 1. rGNz9dropout_state_filter_visitor must be callable. Received: DropoutWrapperInitcSst|}t|}||fSr=)r*convert_to_tensorget_static_value)v tensor_value const_valuerrrtensor_and_const_values  z7DropoutWrapper.__init__..tensor_and_const_valueinput_keep_probstate_keep_proboutput_keep_probrz Parameter z# must be between 0 and 1. Received _z7When variational_recurrent=True, dtype must be providedcSstdgt|fdS)Nrir)r*concat TensorShapeas_list)srrrconvert_to_batch_shape@sz7DropoutWrapper.__init__..convert_to_batch_shapecs|}tjj||dS)N)seedrG)r*randomuniform)rn inner_seedshape)rorGrr batch_noiseFsz,DropoutWrapper.__init__..batch_noiser^zdWhen variational_recurrent=True and input_keep_prob < 1.0 or is unknown, input_size must be providedc|d|dS)Ninputrs _gen_seedirnrur/rrU z)DropoutWrapper.__init__..crv)Nr7rxryr{r}rrr~\rcrv)Noutputrxryr{r}rrr~cr)r'r(rr%_default_dropout_state_filter_visitor_dropout_state_filterr*rBrRsetattrfloat_variational_recurrent _input_size_seed_recurrent_input_noise_recurrent_state_noise_recurrent_output_noiser)_input_keep_probnumbersRealrrr)r/r!rfrhrgvariational_recurrent input_sizerGrpdropout_state_filter_visitorr1reprobattr tensor_prob const_probr2)rurorGr/rr(sN         zDropoutWrapper.__init__cCsN|jdurdSd||f}t|j|d}tt|dddd@S)Nz%s_%dzutf-8i)rstrencodeinthashlibmd5 hexdigest)r/ salt_prefixindexsaltstringrrrrzis   zDropoutWrapper._gen_seedcCs|jSr=)r!r>rrr wrapped_cellpszDropoutWrapper.wrapped_cellcCs|j|d|_dS)NT)r!buildbuilt)r/ inputs_shaperrrrts  zDropoutWrapper.buildcCs4||}t|}t|||}|||S)z7Performs dropout given the pre-calculated noise tensor.)r*floordivide set_shape get_shape)r/ unused_indexvaluenoise keep_prob random_tensor binary_tensorretrrr$_variational_recurrent_dropout_valuexs  z3DropoutWrapper._variational_recurrent_dropout_valuecs^|dur|}jsfdd}t||g||gRSfdd}t||g|||gRS)zADecides whether to perform standard dropout or recurrent dropout.Ncs0t|tr|rtjj|d|dS|S)Nr^)raterp)r)boolr*nndropoutrz)r| do_dropoutrbrrr/rrrs z(DropoutWrapper._dropout..dropoutcs"t|tr|r|||S|Sr=)r)rr)r|rrbn)rr/rrrs )rr)r/valuesrrecurrent_noisershallow_filtered_substructurerrrr_dropouts"  zDropoutWrapper._dropoutc Ksdd}||jr||d|j|j}|||fi|\}}||jr7tjj|j|}||d|j |j|}||j rF||d|j |j }||fS)a'Runs the wrapped cell and applies dropout. Args: inputs: A tensor with wrapped cell's input. state: A tensor or tuple of tensors with wrapped cell's state. cell_call_fn: Wrapped cell's method to use for step computation (cell's `__call__` or 'call' method). **kwargs: Additional arguments. Returns: A pair containing: - Output: A tensor with cell's output. - New state: A tensor or tuple of tensors with new wrapped cell's state. cSst|t p |dkS)Nri)r)r)prrr_should_dropoutsz:DropoutWrapper._call_wrapped_cell.._should_dropoutrwr7r) rrr_state_keep_probr*r+nestget_traverse_shallow_structurerr_output_keep_probr) r/r6r7r8r1rr new_staterrrrr9s:   z!DropoutWrapper._call_wrapped_cellcsp|j|j|j|j|j|jd}|jtkr%t|j\}}}| |||dt }t t |t |S)z*Returns the config of the dropout wrapper.)rfrhrgrrrp) dropout_fndropout_fn_typedropout_fn_module)rrrrrrrrrupdater'rJrKrLrM)r/rIfunction function_typefunction_modulerOr2rrrJs*  zDropoutWrapper.get_configcsFd|vr|}t||ddd}|d||d<tt|j||dS)NrrrrrT)rPrrQr'r]rU)rSrIrTdropout_state_filterr2rrrUs  zDropoutWrapper.from_config)r^r^r^FNNNNr=)rDrVrWrXr(rzrYrrrrr9rJrZrUr[rrr2rr]s.+  0 5r]z2keras.__internal__.legacy.rnn_cell.ResidualWrapperznn.rnn_cell.ResidualWrappercsHeZdZdZd fdd ZddZfddZed fd d ZZ S) ResidualWrapperzBRNNCell wrapper that ensures cell inputs are added to the outputs.Nc tj|fi|||_dS)aConstructs a `ResidualWrapper` for `cell`. Args: cell: An instance of `RNNCell`. residual_fn: (Optional) The function to map raw cell inputs and raw cell outputs to the actual cell outputs of the residual network. Defaults to calling nest.map_structure on (lambda i, o: i + o), inputs and outputs. **kwargs: dict of keyword arguments for base layer. N)r'r( _residual_fn)r/r! residual_fnr1r2rrr(s zResidualWrapper.__init__c  sB|||fi|\}}ddfdd}|jp|||}||fS)aGRun the cell and apply the residual_fn. Args: inputs: cell inputs. state: cell state. cell_call_fn: Wrapped cell's method to use for step computation (cell's `__call__` or 'call' method). **kwargs: Additional arguments passed to the wrapped cell's `call`. Returns: Tuple of cell outputs and new state. Raises: TypeError: If cell inputs and outputs have different structure (type). ValueError: If cell inputs and outputs have different structure (value). cSs||dSr=)rassert_is_compatible_withinpoutrrrassert_shape_match>sz>ResidualWrapper._call_wrapped_cell..assert_shape_matchcs2tj||tj||tjdd||S)NcSs||Sr=rrrrrr~EszQResidualWrapper._call_wrapped_cell..default_residual_fn..)r*rassert_same_structure map_structure)r6outputsrrrdefault_residual_fnAs  z?ResidualWrapper._call_wrapped_cell..default_residual_fn)r) r/r6r7r8r1rrr res_outputsrrrr9)s z"ResidualWrapper._call_wrapped_cellcsR|jdurt|j\}}}|||d}ni}t}tt|t|S)z+Returns the config of the residual wrapper.N)rresidual_fn_typeresidual_fn_module)rrr'rJrKrLrM)r/rrrrIrOr2rrrJMs  zResidualWrapper.get_configcs<d|vr|}t||ddd}||d<tt|j||dS)Nrrrr)rPrr'rrU)rSrIrTresidual_functionr2rrrU_s zResidualWrapper.from_configr=) rDrVrWrXr(r9rJrZrUr[rrr2rrs $rz0keras.__internal__.legacy.rnn_cell.DeviceWrapperznn.rnn_cell.DeviceWrappercs<eZdZdZfddZddZddZfdd ZZS) DeviceWrapperz=Operator that ensures an RNNCell runs on a particular device.c r)aOConstruct a `DeviceWrapper` for `cell` with device `device`. Ensures the wrapped `cell` is called with `tf.device(device)`. Args: cell: An instance of `RNNCell`. device: A device string or function, for passing to `tf.device`. **kwargs: dict of keyword arguments for base layer. N)r'r(_device)r/r!devicer1r2rrr(us zDeviceWrapper.__init__c Cstt|jd/tjj|j|j ||WdWdS1s,wYWddS1ss:          & c  \  X