o i e1@sdZddlZddlmZddlmZddlmZddlmZddlm Z ddlm Z dd l m Z dd l mZdd lmZd d ZddZddZddZddZddZddZddZedgdd"ddZd d!ZdS)#z|Gradient checker for functions. The gradient checker verifies numerically that an function properly computes the gradients N)backprop)context)dtypes)indexed_slices)ops)tensor) array_ops) tf_logging) tf_exportcCs(t|tr|Sd}|D]}||9}q |S)N) isinstanceint)tyxrX/var/www/myenv/lib/python3.10/site-packages/tensorflow/python/ops/gradient_checker_v2.py_product!s  rcCsBt|tjrtrtjdd|jDdd|jD|jdS|S)aYConverts IndexedSlices to IndexedSlicesValue with numpy indices/values. When eager execution is enabled, converts IndexedSlices to IndexedSlicesValue with numpy indices/values. Args: a: any value. Returns: If a is IndexedSlices and eager execution is enabled, calls numpy() on a's fields. Otherwise returns a unchanged. cSg|]}|qSrnumpy.0rrrr ;z(_eval_indexed_slices..cSrrrrrrrr<r)indicesvalues dense_shape) r r IndexedSlicesrexecuting_eagerlyIndexedSlicesValuerrr)arrr_eval_indexed_slices+s r"cCst|tjr |St|tjrt}||St|tj ret |j }t |jt |jks:Jd|j|j|ft|j|jD]!\}}d|krQt |ksZnJd||f|||7<qA|S|S)a@Converts Tensors, EagerTensors, and IndexedSlicesValue to numpy arrays. Args: a: any value. Returns: If a is EagerTensor or Tensor, returns the evaluation of a by calling numpy() or run(). If a is IndexedSlicesValue, constructs the corresponding dense numpy array. Otherwise returns a unchanged. z8IndexedSlicesValue has %s value slices but %s indices %srz*IndexedSlicesValue has invalid index %s %s)r r EagerTensorrrTensorget_default_sessionrunrr npzerosrlenrrzip)r!sessarr values_sliceindexrrr _to_numpyAs&       r/csPtr fdd}|Sddt||Dtfdd}|S)a,Return a function that executes 'f'. In TF 2.x, this is the same as `f`. In TF 1.x, returns a Python function that executes the graph defined by `f` in a Session. Args: f: the function. xs_dtypes: dtypes of f's arguments. xs_shapes: shapes of f's arguments. Returns: csttj|SN)maprconvert_to_tensorxs_data)frrdecorated_eagernsz!_prepare..decorated_eagercSsg|] \}}tj||dqS)shape)r placeholder)rx_dtypex_shaperrrrrs z_prepare..cs&dd|D}jtt|dS)NcSsg|]}t|qSr)r/)rr!rrrrzrz5_prepare..decorated_graph..) feed_dict)r&dictr*r3)r+xsrrrdecorated_graphysz!_prepare..decorated_graph)rrr*rr%)r5 xs_dtypes xs_shapesr6r?r)r5r+r>rr_prepare^s rBcs||}|jjrt|jdn|j}|jrdnd}t|}t|dd} t||} tj| |f|jjjd} tj||jd} | |jj} t ||gt fdd|gdd |Ddgd d |D}t| D]T}d| |<t|| g|Rd }t|}t|tjrt|j|jD]\}}|| }|| }| |||f|j7<qn|dur| | j| |ddf<d | |<qm| d krt|| g|Rd }|j|jkrtd |j|jft|rtd tdd| | S)aYComputes the theoretical Jacobian for f regarding xs[param]. One can think of the relation among f, xs and y as y = f(xs). Args: f: the function. y_shape: the shape of the result. y_dtype: the dtype of the result. xs: a list of tensors. param: the index of the target parameter. Returns: A 2-d numpy array representing the Jacobian. It has "y_size" rows and "x_size" columns where "x_size" is the number of elements in xs[param] and "y_size" is the number of elements in the result. Raises: ValueError: If result is empty but the gradient is nonzero. )rCr Ndtypecs|d|iS)Ndyr)rFr>grad_fn_unpreprrsz/_compute_theoretical_jacobian..cSg|]}|jqSrrDrzrrrrz1_compute_theoretical_jacobian..cSrJrr7rKrrrrrMrz3Empty gradient has wrong shape: expected %s, got %sz#Empty tensor with nonzero gradientszTheoretical Jacobian = %s)rE is_complextupler8rr'r( real_dtypeas_numpy_dtyperavelviewrgradients_functionrBranger/r"r rr r*rrflat ValueErroranyloggingvlog)r5y_shapey_dtyper>paramrr;y_factorx_size x_val_sizey_sizejacobiandy_data dy_data_flatgrad_fnrowgradivc_beginc_endrrGr_compute_theoretical_jacobiansJ        rlcCsz||j}||j}t||jrdnd}||jrdnd}|jj}|jj}dd|D} dd|D} dd|D}||} tjd||dd} tj||f|d} t || | }t |D]V}| ||}| |||7<t ||}|| ||<| |||8<t ||}|| ||<||| }| || d d |f<q]tdd | | S) aRComputes the numeric Jacobian for f regarding xs[param]. One can think of the relation among f, xs and y as y = f(xs). Args: f: the function. y_size: the number of elements of the result. y_dtype: the dtype of the result. xs: a list of tensors. param: the index of the target parameter. delta: the amount of perturbation we give to the input. Returns: A 2-d numpy array representing the Jacobian. It has "y_size" rows and "x_size" columns where "x_size" is the number of elements in xs[param] and "y_size" is the number of elements in the result. rCr cSrJrrDrrrrrrMz-_compute_numeric_jacobian..cSrJrr7rrrrrrMcSsg|] }tt|qSr)r'asarrayr/rrrrrsrDrNzNumeric Jacobian = %s)r8rErrNrPrQr'rmr(rBrUrRrSr/rYrZ)r5rar\r>r]deltar;r:r_r@rArscalerbcoloriginaly_posy_negdiffrrr_compute_numeric_jacobians2       ruc Cs||}|j}tjtjtjtjtjtjg}|j|vs#Jd|j |f|} | j|vs1Jd| j t |} t |||||} t || ||||} | | fS)z0Computes the theoretical and numerical jacobian.z>Cannot compute gradient for unsupported type %s of argument %sz4Cannot compute gradient for unsupported type %s of y) rErfloat16bfloat16float32float64 complex64 complex128 base_dtypenamerrlru) r5r[r\r>r]rnrr allowed_typest2rajacob_tjacob_nrrr_compute_gradients rcsfddDddD}ddD}t||}|ttfddttDS)z)Compute gradients for a list of x values.cSsg|]}t|qSr)rr2rrrrrsz*_compute_gradient_list..cSrJrrDrrrrrrMcSrJrr7rrrrrrMc s(g|]}tjtj|qSr)rr8ras_dtyperE)rrhrnr5r>rrrr s)rBrOr*rUr))r5r>rnr@rAf_temprrr_compute_gradient_lists  rztest.compute_gradient)v1cCs6t|ttfstdt||durd}t|||S)aComputes the theoretical and numeric Jacobian of `f`. With y = f(x), computes the theoretical and numeric Jacobian dy/dx. Args: f: the function. x: the arguments for the function as a list or tuple of values convertible to a Tensor. delta: (optional) perturbation used to compute numeric Jacobian. Returns: A pair of lists, where the first is a list of 2-d numpy arrays representing the theoretical Jacobians for each argument, and the second list is the numerical ones. Each 2-d array has "y_size" rows and "x_size" columns where "x_size" is the number of elements in the corresponding argument and "y_size" is the number of elements in f(x). Raises: ValueError: If result is empty but the gradient is nonzero. ValueError: If x is not list, but any other type. Example: >>> @tf.function ... def test_func(x): ... return x*x ... >>> >>> class MyTest(tf.test.TestCase): ... ... def test_gradient_of_test_func(self): ... theoretical, numerical = tf.test.compute_gradient(test_func, [1.0]) ... # ((array([[2.]], dtype=float32),), ... # (array([[2.000004]], dtype=float32),)) ... self.assertAllClose(theoretical, numerical) zZ`x` must be a list or tuple of values convertible to a Tensor (arguments to `f`), not a %sNgP?)r listrOrWtyper)r5rrnrrrcompute_gradient&s' rcCsBd}t||D]\}}|js|jrt|t||}q|S)aComputes maximum elementwise gap. Computes the maximum elementwise gap between two lists of tensors of the same shape. Args: grad1: a lists of tensors. grad2: a lists of tensors with the same shape as grad1. Returns: The maximum elementwise gap between the two. r)r*sizer'maximumfabsmax)grad1grad2errorj_tj_nrrr max_errorZs   rr0)__doc__rr'tensorflow.python.eagerrrtensorflow.python.frameworkrrrrtensorflow.python.opsrtensorflow.python.platformr rY tensorflow.python.util.tf_exportr rr"r/rBrlrurrrrrrrrs,          "H:   3