o i e&@sdZddlZddlmmZddlmZdZdZ dZ dZ dZ dZ d Zd Zd Zd d ZGdddZd%ddZddZddZddZddZddZddZdd Zd!d"Zd#d$ZdS)&z0Utilities used by both the GRU and LSTM classes.N)get_device_nameapi_implementsapi_preferred_deviceCPUGPUz4Layer %s will use cuDNN kernels when running on GPU.zLayer %s will not use cuDNN kernels since it doesn't meet the criteria. It will use a generic GPU kernel as fallback when running on GPU.cCsdS)NFr r r R/var/www/myenv/lib/python3.10/site-packages/keras/src/layers/rnn/gru_lstm_utils.pyuse_new_gru_lstm_impl1sr c@s eZdZdZddZddZdS) DefunWrapperz;A wrapper with no deep copy of the Defun in LSTM/GRU layer.c Cs||_||_||_|jdvrtd|jd|jd|jt|jdtti}|jdkr8ddl m }|j }n dd l m }|j }tj|d |d |_dS) N)lstmgruz>Defun wrapper only applies to LSTM and GRU layer, but given {} time_major go_backwards_r r)r )rF autographexperimental_attributes)rr layer_name ValueErrorformat_FUNCTION_API_NAME_ATTRIBUTEstruuiduuid4keras.src.layers.rnnr lstm_with_backend_selectionrgru_with_backend_selectiontffunction defun_layer)selfrrrsupportive_attributesr layer_funcrr r r __init__:s4      zDefunWrapper.__init__cCs&t||j|j|j}||t|<|SN)typerrrid)r"memo new_wrapperr r r __deepcopy__^s   zDefunWrapper.__deepcopy__N)__name__ __module__ __qualname____doc__r%r+r r r r r 7s $r FcsDfddfdd|D}fdd|D}tj||ddS)aUtility function convert variable to cuDNN compatible parameter. Note that Keras weights for kernels are different from the cuDNN format. Eg.: ``` Keras cuDNN [[0, 1, 2], <---> [[0, 2, 4], [3, 4, 5]] [1, 3, 5]] ``` If the input weights need to be in a unified format, then set `transpose_weights=True` to convert the weights. Args: weights: list of weights for the individual kernels and recurrent kernels. biases: list of biases for individual gate. shape: the shape for the converted variables that will be feed to cuDNN. transpose_weights: boolean, whether to transpose the weights. Returns: The converted weights that can be feed to cuDNN ops as param. csrt|S|Sr&)r transpose)w)transpose_weightsr r convertsz$canonical_to_params..convertcsg|] }t|qSr rreshape.0x)r3shaper r sz'canonical_to_params..csg|]}t|qSr r4r6)r9r r r:sraxis)rconcat)weightsbiasesr9r2r )r3r9r2r canonical_to_paramsfs r@cCsFt|d}tjt|tjdd}tj||d}tt||S)a_Check the mask tensor and see if it right padded. For cuDNN kernel, it uses the sequence length param to skip the tailing timestep. If the data is left padded, or not a strict right padding (has masked value in the middle of the sequence), then cuDNN kernel won't be work properly in those cases. Left padded data: [[False, False, True, True, True]]. Right padded data: [[True, True, True, False, False]]. Mixture of mask/unmasked data: [[True, False, True, False, False]]. Note that for the mixed data example above, the actually data RNN should see are those 2 Trues (index 0 and 2), the index 1 False should be ignored and not pollute the internal states. Args: mask: the Boolean tensor with shape [batch, timestep] Returns: boolean scalar tensor, whether the mask is strictly right padded. rr;)maxlen)rr9 reduce_sumcastint32 sequence_mask reduce_allequal)maskmax_seq_length count_of_trueright_padded_maskr r r is_sequence_right_paddedsrLcCsttjt|ddS)Nrr;)r reduce_anyrF logical_not)rHr r r has_fully_masked_sequencesrOcCstjdr+|s|durdS|durt|S|dur)tjt|t|SdS|dur1dS|r8t|}t t |t t |S)N is_rocm_buildFT) r sysconfigget_build_inforFmathrG reduce_min reduce_maxr0 logical_andrLrNrO)rHrsequence_lengthsr r r is_cudnn_supported_inputss$    rXcCs$|rdnd}tjt|tj|dS)aOCalculate the sequence length tensor (1-D) based on the masking tensor. The masking tensor is a 2D boolean tensor with shape [batch, timestep]. For any timestep that should be masked, the corresponding field will be False. Consider the following example: a = [[True, True, False, False], [True, True, True, False]] It is a (2, 4) tensor, and the corresponding sequence length result should be 1D tensor with value [2, 3]. Note that the masking tensor must be right padded that could be checked by, e.g., `is_sequence_right_padded()`. Args: mask: Boolean tensor with shape [batch, timestep] or [timestep, batch] if time_major=True. time_major: Boolean, which indicates whether the mask is time major or batch major. Returns: sequence_length: 1D int32 tensor. rrr;)rrBrCrD)rHrtimestep_indexr r r calculate_sequence_by_masks rZcCs&t|t|i}||tj|d|dS)NFr)r_FUNCTION_DEVICE_ATTRIBUTEupdaterr )unique_api_namepreferred_devicefuncr#function_attributesr r r generate_defun_backends racCs$t}|dur dStjjj|jS)zAParse the current context and return the device type, eg CPU/GPU.N)rrcompatv1 DeviceSpec from_string device_type)current_devicer r r get_context_device_typesrhcCs@tdtj|tjddWdS1swYdS)Nz/cpu:0runtime)dtypename)rdeviceconstantfloat32) runtime_namer r r ris $ricCst|tjr |S|S)z/Read the value of a variable if it is variable.) isinstancerVariable read_value)vr r r read_variable_values rtcOs$|j|i|}|||S)aRegister a specialization of a `Function` into the graph. This won't actually call the function with the inputs, and only put the function definition into graph. Register function with different input param will result into multiple version of functions registered in graph. Args: func: the `Function` instance that generated by a @defun *args: input arguments for the Python function. **kwargs: input keyword arguments for the Python function. Returns: a `ConcreteFunction` object specialized to inputs and execution context. Raises: ValueError: When the input function is not a defun wrapped python function. )get_concrete_function add_to_graphadd_gradient_functions_to_graph)r_argskwargs concrete_funcr r r function_registersr{)F)r/rtensorflow.compat.v2rbv2rtensorflow.python.eager.contextrrr[CPU_DEVICE_NAMEGPU_DEVICE_NAMERUNTIME_UNKNOWN RUNTIME_CPU RUNTIME_GPUCUDNN_AVAILABLE_MSGCUDNN_NOT_AVAILABLE_MSGr r r@rLrOrXrZrarhrirtr{r r r r s4  /!