o i e@sdZddlZddlZddlZddlZddlZddlZddlZddl m Z m Z ddl m Z ddlmZmZmZddlmZmZddlmZmZdd lmZdd lmZddlZddlmmZGd d d e ZGd ddeZGdddeZ GdddeZ!GdddZ"GdddZ#GdddeZ$GdddeZ%GdddeZ&ej'(de)dfej*dfej+dfe,dfej-dfej.dfej/dd d!d"fej/d#d d!d"fej/dd$d"fej/d#d$d"fe/d"fg d%d&Z0Gd'd(d(eZ1e 2d)ej3j4vd*Gd+d,d,eZ5e 2d)ej3j4vd*Gd-d.d.eZ6e 2d/ej3j4vd0Gd1d2d2eZ7e 2d3ej3j4vd4Gd5d6d6eZ8e 2d7ej3j4vd8Gd9d:d:eZ9e 2d;ej3j4vd<Gd=d>d>eZ:e 2d?ej3j4vd@GdAdBdBeZ;GdCdDdDeZGdIdJdJeZ?GdKdLdLeZ@GdMdNdNeZAGdOdPdPeZBGdQdRdReZCGdSdTdTeZDGdUdVdVeZEGdWdXdXeZFGdYdZdZeZGGd[d\d\eZHGd]d^d^eZIGd_d`d`eZJej'jKejjLdakdbdcZMGdddedeeZNGdfdgdgeZOGdhdidieZPGdjdkdkeZQGdldmdmeZRe SejjLdnkdodpdqZTe SejjLdrkp>ejjLdsko>ejjLdtkdudvdwZUdxdyZVdzd{ZWd|d}ZXd~dZYddZZddZ[ej'j\ejjLdkddcddZ]ddZ^GdddeZ_GdddeZ`dS)z Dataset testing operations. Tests all dataset operations, including creation, with the exception of: 1. Slicing operations for read and write, handled by module test_slicing 2. Type conversion for read and write (currently untested) N)utTestCase)get_data_file_path)FileGroupDataset)is_empty_dataspaceproduct)h5fh5t)H5pyDeprecationWarning)versionc@eZdZddZddZdS) BaseDatasetcCst|d|_dS)Nw)rmktempfselfrF/var/www/myenv/lib/python3.10/site-packages/h5py/tests/test_dataset.pysetUp'szBaseDataset.setUpcCs|jr |jdSdSN)rcloserrrrtearDown*szBaseDataset.tearDownN)__name__ __module__ __qualname__rrrrrrr&s rc@eZdZdZddZdS)TestReprz5 Feature: repr(Dataset) behaves sensibly cCs<|jdd}|t|t|j|t|tdS)z( repr() works on live and dead datasets fooN)rcreate_datasetassertIsInstancereprstrrrdsrrrtest_repr_open4s zTestRepr.test_repr_openN)rrr__doc__r*rrrrr /s r c@steZdZdZddZddZddZdd Zd d Zd d Z ddZ ddZ e eed dddZddZdS)TestCreateShapez@ Feature: Datasets can be created from a shape only cC |jdd}||jddS)z Create a scalar dataset r!rNrr$ assertEqualshaperdsetrrrtest_create_scalarBz"TestCreateShape.test_create_scalarcCr-)z Create a size-1 dataset r!rNr.r1rrrtest_create_simpleGr4z"TestCreateShape.test_create_simplecC |jdd}||jddS)z+ Create a size-1 dataset with integer shaper!rr5Nr.r1rrrtest_create_integerLr4z#TestCreateShape.test_create_integercCsX|jdd}||jd||jd|jdd}||jd||jddS) Create an extended dataset r!?r;bar <Nrr$r/r0sizer1rrrtest_create_extendedQ z$TestCreateShape.test_create_extendedcCsX|jdd}||jd||jd|jdd}||jd||jddS)r9r!r;r:r<r=r@NrAr1rrrtest_create_integer_extendedZrDz,TestCreateShape.test_create_integer_extendedcCs&|jdd}||jtddS)) Confirm that the default dtype is float r!r:z=f4Nrr$r/dtypenpr1rrrtest_default_dtypecsz"TestCreateShape.test_default_dtypecCs<|t|jdWddS1swYdS)z Missing shape raises TypeError r!N assertRaises TypeErrorrr$rrrrtest_missing_shapehs "z"TestCreateShape.test_missing_shapecCsF|jjddtjd}tdvrtdt||j tjdS)rFr!r:rH)ppc64lez(Storage of long double deactivated on %sN) rr$rI longdoubleplatformmachinepytestxfailr/rHr1rrrtest_long_doublems z TestCreateShape.test_long_double complex256zNo support for complex256cCs0|jjddtdd}||jtddS)rFr!r:rWrON)rr$rIrHr/r1rrrtest_complex256ts zTestCreateShape.test_complex256cCs<|jdd}||jd|jdd}||jddS)Nsfoor5sbar/baz)r.rr2dset2rrrtest_name_bytes{szTestCreateShape.test_name_bytesN)rrrr+r3r6r8rCrErJrNrVrskipIfhasattrrIrXr\rrrrr,<s    r,c@sxeZdZdZddZddZddZdd Zd d Zd d Z ddZ ddZ ddZ ddZ ddZddZddZdS)TestCreateDatazA Feature: Datasets can be created from existing data cC0tdd}|jjd|d}||j|jdS)z- Create a scalar dataset from existing array rrr!dataNrIonesrr$r/r0rrbr2rrrr3 z!TestCreateData.test_create_scalarcCr`)z/ Create an extended dataset from existing data r:rr!raNrcrerrrrCrfz#TestCreateData.test_create_extendedcCs4|jjdddd}||tj|d|jvdS)z1 Create dataset with missing intermediate groups z /foo/bar/bazr?r?z.check_vlen_asciiaabcrabdefcdrONrr$rIarrayobject)rrrrrtest_create_bytestrings  z%TestCreateData.test_create_bytestringcCs4|jjdtjddgddd}|j|tjdddS) NrrrS3rOrarnr)rr$rIrrr rr1rrrtest_create_np_sszTestCreateData.test_create_np_scszfdd}|jjddd|jjdddgd|jjddgdggd|jjd tjddgtd ddS) Ncrr)rr CSET_UTF8rrrrcheck_vlen_utf8rz;TestCreateData.test_create_strings..check_vlen_utf8rabcrardefrrrOr)rrrrrtest_create_stringss  z"TestCreateData.test_create_stringscCsN|t|jjdtjddgdddWddS1s wYdS)NrrrU3rOra)rLrMrr$rIrrrrrtest_create_np_us  "zTestCreateData.test_create_np_ucCs*|jjddd|t|jdjdS)Nr!rrO)rr$rir rvrrrr test_empty_create_via_None_shapesz/TestCreateData.test_empty_create_via_None_shapecCs2|jjdtjddd|t|jdjdS)Nr!rrOra)rr$rhEmptyrir rvrrrr!test_empty_create_via_Empty_classsz0TestCreateData.test_empty_create_via_Empty_classcCsH|t|jjddtddWddS1swYdS)Nr<r#rnro)rLrurr$rIrprrrrtest_create_incompatible_datas "z,TestCreateData.test_create_incompatible_dataN)rrrr+r3rCrjrsrwrrrrrrrrrrrrr_s    r_c @seZdZdZejdddejddejddfddejddejd d fd d ejd d d d fejd d d dffddejdd ddfejd d fgddZ ddZ ddZ ddZ ddZ d S)TestReadDirectlyzI Feature: Read data directly from Dataset into a Numpy array +source_shape,dest_shape,source_sel,dest_seldrr?2r@FZNrlr?rr r>rYr>rnc Csltjt|dd|}|jd||d}tj|ddd}|} ||| |<||||tj || dS)Nint64rOr2ra) rIrpr rrr$fullcopy read_directtestingassert_array_equal) r writable_file source_shape dest_shape source_seldest_sel source_valuesr2arrexpectedrrrtest_read_directs  z!TestReadDirectly.test_read_directcCsN|jddtjdddd}tjddd}||tj|tjddddS)Nr2r?r?rrOra)r$rIrprdrrrrrr2rrrr test_no_sels zTestReadDirectly.test_no_selcCsl|jddd}tdd}tt||tjddtjddWddS1s/wYdS NedsetrrOrrr?rr@)r$rIrdrTraisesrMrs_)rr empty_dsetrrrr test_emptys   $"zTestReadDirectly.test_emptycCT|jdddd}td}tt||WddS1s#wYdSNr2rrrOr$rIrdrTrrMrrrrrtest_wrong_shape    "z!TestReadDirectly.test_wrong_shapecCX|jdddd}tjddd}tt||WddS1s%wYdSNr2rgrrOF)orderrrrrrtest_not_c_contiguous   "z&TestReadDirectly.test_not_c_contiguous)rrrr+rTmark parametrizerIrrrrrrrrrrrs.$   rc @seZdZdZejdddejddejddfddejddejd d fd d ejd d d d fejd d d dffddejdd ddfejd d fgddZ ddZ ddZ ddZ d S)TestWriteDirectlyz> Feature: Write Numpy array directly into Dataset rrrr?rr@rrNrrrrrrYr>rnc Csj|jd|ddd}tt||}tj|ddd}||||<||||tj|dd|dS)Nr2int32rrH fillvaluerO) r$rIrpr rrr write_directrr) rrrrrrr2rrrrrtest_write_directs   z#TestWriteDirectly.test_write_directcCsf|jddd}tt|tdtjddtjddWddS1s,wYdSr)r$rTrrMrrIrdr)rrrrrrr!s *"zTestWriteDirectly.test_emptycCrrr$rIrdrTrrMrrrrrr&rz"TestWriteDirectly.test_wrong_shapecCrrrrrrrr,rz'TestWriteDirectly.test_not_c_contiguous) rrrr+rTrrrIrrrrrrrrrr s.$  rc@sPeZdZdZddZddZddZdd Zd d Zd d Z ddZ ddZ dS)TestCreateRequirezS Feature: Datasets can be created only if they don't exist in the file cCs.|jddd}||t||jddS)z& Create new dataset with no conflicts r!rmrN)rrequire_datasetr%rr/r0r1rrr test_create9s zTestCreateRequire.test_createcCs0|jddd}|jddd}|||dS)z) require_dataset yields existing dataset r!rmrNrrr/rZrrrtest_create_existing?sz&TestCreateRequire.test_create_existingcCs|jddd}|jddd}||||jddd}|jddd}||||jddd}|jddd}|||dS) z; require_dataset with integer shape yields existing datasetr!r?rr<rbazsbazNrrZrrrtest_create_1DEs  z TestCreateRequire.test_create_1DcCsP|jddd|t|jdddWddS1s!wYdS)z6 require_dataset with shape conflict yields TypeError r!rmr)r?r#Nrr$rLrMrrrrrtest_shape_conflictSs "z%TestCreateRequire.test_shape_conflictcCsL|jd|t|jdddWddS1swYdS)z< require_dataset with object type conflict yields TypeError r!rmrN)r create_grouprLrMrrrrrtest_type_conflictYs  "z$TestCreateRequire.test_type_conflictcCsP|jddd}|t|jdddWddS1s!wYdS)zL require_dataset with dtype conflict (strict mode) yields TypeError r!rmrS10Nrr1rrrtest_dtype_conflict_s "z%TestCreateRequire.test_dtype_conflictcCs4|jddd}|jjddddd}|||dS)z* require_dataset with exactly dtype match r!rmrTexactN)rr$rr/rZrrrtest_dtype_exactfsz"TestCreateRequire.test_dtype_exactcCsH|jddd}|jjddddd}|||||jtddS)zJ require_dataset with convertible type succeeds (non-strict mode) r!rmi4i2FrN)rr$rr/rHrIrZrrrtest_dtype_closems z"TestCreateRequire.test_dtype_closeN) rrrr+rrrrrrrrrrrrr3s rc@XeZdZdZddZddZddZdd Zd d Zd d Z ddZ ddZ ddZ dS)TestCreateChunkedzL Feature: Datasets can be created by manually specifying chunks cCs$|jjdddd}||jddS)z Create via chunks tuple r!rrr0chunksNrr$r/rr1rrrtest_create_chunks|z$TestCreateChunked.test_create_chunkscCs$|jjdddd}||jddS)z Create via chunks integer r!rr?rrNrr1rrrtest_create_chunks_integerrz,TestCreateChunked.test_create_chunks_integercCB|t|jjddddWddS1swYdS)z& Illegal chunk size raises ValueError r!rrrNrLrurr$rrrrtest_chunks_mismatch "z&TestCreateChunked.test_chunks_mismatchcCD|t|jjdddddWddS1swYdS)z3 Chunked format required for given storage options r!rrF)r0maxshaperNrrrrrtest_chunks_falses "z#TestCreateChunked.test_chunks_falsecCr)z> Attempting to create chunked scalar dataset raises TypeError r!r)rrNrKrrrrtest_chunks_scalarrz$TestCreateChunked.test_chunks_scalarcCs6|jjdddd}||jt|t|jddS)z Auto-chunking of datasets r!)rrTrrYN)rr$r%rtupler/lenr1rrrtest_auto_chunkssz"TestCreateChunked.test_auto_chunkscCs&|jjddddd}||jddS)z7 Auto-chunking with pathologically large element sizes r!rn S100000000Tr0rHrr5Nrr1rrrtest_auto_chunks_abusesz(TestCreateChunked.test_auto_chunks_abusecCs|jjddtjdd}d|ddddf<|t|ddddfdkd |d<|t|dd kd |d d <|t|d d d kdS) z+ Test scalar assignment of chunked dataset r!)rnrr)rrrrr?rN( rrY)rr$rIrriallr1rrrtest_scalar_assignments "  z(TestCreateChunked.test_scalar_assignmentcCstjtdd}|jjddddWdn1swYtjtdd}|jjdddd WddS1s Feature: Datasets can be created with fill value cCs6|jjdddd}||dd||dddS)z- Fill value is reflected in dataset contents r!rg@rrrN)rr$r/r1rrrtest_create_fillvalsz'TestCreateFillvalue.test_create_fillvalcCs4|jjdddd}||jd||jtjdS)z( Fill value is recoverable via property r!rg@rN)rr$r/rassertNotIsInstancerIndarrayr1rrr test_propertyrkz!TestCreateFillvalue.test_propertycCr7)z0 .fillvalue property works correctly if not set r!rrN)rr$r/rr1rrrtest_property_noner4z&TestCreateFillvalue.test_property_nonecCsVtddg}tjd|dd}|jjdd||d}||j|||d |d S) z& Fill value works with compound types )rf4)ri8r5rOrr!rrr#N)rIrHrdrr$r/rassertAlmostEqual)rdtvr2rrr test_compounds z!TestCreateFillvalue.test_compoundcCsH|t|jjddddgdd}WddS1swYdS)z$ Bogus fill value raises ValueError r!rri)rr*rNrr1rrrtest_excs  "zTestCreateFillvalue.test_excN) rrrr+rrrrr#rrrrrs rz dt,expectedutf-8rencodingrasciir'cCs"|jdd|d}|j|ksJdS)Nr!rrO)r$r)rrrr2rrrtest_get_unset_fill_valuesr+c@r)TestCreateNamedTypezC Feature: Datasets created from an existing named type cCsntd|jd<|jjdd|jdd}||jtd||j|jdj||jdS)z7 Named type object works and links the dataset to type f8typexrrON) rIrHrr$r/rvrxri committedr1rrr test_nameds zTestCreateNamedType.test_namedN)rrrr+r1rrrrr, r,gzipzDEFLATE is not installedc@s0eZdZdZddZddZddZdd Zd S) TestCreateGzipz= Feature: Datasets created with gzip compression cCs4|jjddddd}||jd||jddS)z# Create with explicit gzip options r!rrlr3r compressioncompression_optsNrr$r/r7r8r1rrr test_gzips zTestCreateGzip.test_gzipcCs2|jjdddd}||jd||jddS)z+ Create with implicit gzip level (level 4) r!r5r3r7r#Nr9r1rrrtest_gzip_implicitz!TestCreateGzip.test_gzip_implicitcCs|jjdddd}||jd||jdtjjj}z4t tjj_| t |jjdddd}Wdn1sr?r@rrr$h5zFILTER_DEFLATEr/r7r8)rrAr2rrrtest_compression_number5s  z3TestCreateCompressionNumber.test_compression_numbercCs |t}|jjddddWdn1swY|dt|j|t}|jjddddWdn1s@wY|dt|jtjj j }z6t tjj _ |t |jjddtj jdWdn1suwYW|tjj _ dSW|tjj _ dS|tjj _ w) z* Create with invalid compression numbers r!r5ir;NzInvalid filterrzUnknown compression)rLrurr$assertInr' exceptionrhr>r?r@r IndexErrorrIrJ)rerArrrtest_compression_number_invalidAs"     z;TestCreateCompressionNumber.test_compression_number_invalidN)rrrr+rKrPrrrrrH.s rHlzfzLZF is not installedc@rG) TestCreateLZF< Feature: Datasets created with LZF compression cCs|jjdddd}||jd||jdtd}|jjd|dd}||jd||jd|j|jdd }|||dS) z Create with explicit lzf r!r5rQr;Nrr<)rbr7r) rr$r/r7r8rIrpflushrq)rr2testdatareaddatarrrtest_lzf]s  zTestCreateLZF.test_lzfcCr)z& Giving lzf options raises ValueError r!r5rQr#r6Nrrrrr test_lzf_excms  "zTestCreateLZF.test_lzf_excN)rrrr+rWrXrrrrrRVs rRszipzSZIP is not installedc@r)TestCreateSZIPrScCs|jjddddd}dS)z Create with explicit szip r!r5rY)ecr6N)rr$r1rrr test_szip{s  zTestCreateSZIP.test_szipN)rrrr+r]rrrrrZt rZshufflezSHUFFLE is not installedc@r)TestCreateShufflez8 Feature: Datasets can use shuffling filter cC"|jjdddd}||jdS)z Enable shuffle filter r!r5T)r_N)rr$rir_r1rrr test_shufflezTestCreateShuffle.test_shuffleN)rrrr+rbrrrrr`r^r` fletcher32zFLETCHER32 is not installedc@r)TestCreateFletcher32z= Feature: Datasets can use the fletcher32 filter cCra)z Enable fletcher32 filter r!r5T)rdN)rr$rirdr1rrrtest_fletcher32rcz$TestCreateFletcher32.test_fletcher32N)rrrr+rfrrrrrer2re scaleoffsetzSCALEOFFSET is not installedc@sHeZdZdZddZddZddZdd Zd d Zd d Z ddZ dS)TestCreateScaleOffsetz? Feature: Datasets can use the scale/offset filter cCD|t|jjddtdd}WddS1swYdS)z[ Ensure that a scale factor is required for scaleoffset compression of floating point data r!r5TrHrgNrLrurr$floatr1rrr test_float_fails_without_options "z6TestCreateScaleOffset.test_float_fails_without_optionscCri)z# Check when scaleoffset is negetiver!r5grjNrkr1rrrtest_non_integerrnz&TestCreateScaleOffset.test_non_integercCri)z% Check when dtype is unsupported typer!r5TrjN)rLrMrr$boolr1rrrtest_unsupport_dtypernz*TestCreateScaleOffset.test_unsupport_dtypecCsd}d}dd|}tjj|d|}|jjd|t|d}|jdus%J||d <|jj}|jt |d |_|jdd }|j ||d| d ||k rSJdS) z2 Scaleoffset filter works for floating point data r#ri,rr?g?r!rjN.r) precision) rIrandomrandrr$rlrgfilenamerrhrrqr )rscalefacr0rangerUr2rwrVrrr test_floats  z TestCreateScaleOffset.test_floatcCsd}d}tjjdd|d|d}|jjd|tdd }|jd us"J||d <|jj}|jt |d |_|jdd }| ||d S) zB Scaleoffset filter works for integer data with default precision r rrrrYrrBr!TrjN.rs) rIrurandintrr$intrgrwrrhrrqrnbitsr0rUr2rwrVrrrtest_ints zTestCreateScaleOffset.test_intcCsd}d}tjjdd||d}|jjd|t|d}||jdk||d<|jj}|j t |d |_|jdd}| ||d S) D Scaleoffset filter works for integer data with specified precision r rrrrYr{r!rj.rsN) rIrur|rr$r}rirgrwrrhrrqr~rrrtest_int_with_minbitss z+TestCreateScaleOffset.test_int_with_minbitscCsd}d}tjjdd|dd|d}|jjd|t|d}||jdk||d <|jj}|j t |d |_|jdd }||k rHJd S) rr rrrrYrr{r!rj.rsN) rIrur|rr$r}rirgrwrrhrr r~rrrtest_int_with_minbits_lossys z1TestCreateScaleOffset.test_int_with_minbits_lossyN) rrrr+rmrorqrzrrrrrrrrhs rhc@s@eZdZdZddZddZddZdd Zd d Zd d Z dS) TestExternalzB Feature: Datasets with the external storage property c Csd}tj|}|}|dtjfg}|jjd||j|dd}||d<|jdus*Jt |d }| }Wdn1s>wY|| ksKJt j jd krot|j}t|jjj} || ksqJdSdS) z' Create and access an external dataset r>rrr!z ${ORIGIN}rHexternal efile_prefix.Nrbrr?r)rIrurr UNLIMITEDrr$rHropenreadtobytesrhrhdf5_version_tuplepathlibPathrvget_access_plistget_efile_prefixdecodeas_posixrwparent) rr0rUext_filerr2fidcontentsrrrrr test_contentss"     zTestExternal.test_contentsc Cs&d}tj|}|}tj|dtjfg}|jj d||j |tj |d}||d<|j dus2Jt |d }|}Wdn1sFwY||ksSJtjjdkrut|j}t|j} || ksuJ|jjd||j tj |d } | j dusJ| d |kdS) z< Create and access an external dataset using an efile_prefixrrr!r.Nrr)rr)rIrurospathbasenamer rrr$rHdirnamerrrrrhrrrrrvrrrrrr) rr0rUrrr2rrrrr[rrrtest_contents_efile_prefix*s$      z'TestExternal.test_contents_efile_prefixcCs|jjdd|ddS)z/ External argument may be a file name str only r!rrN)rr$rrrrr test_name_strHszTestExternal.test_name_strcCs |jjddt|ddS)z0 External argument may be a file name path only r!rrN)rr$rrrrrrrtest_name_pathMs   zTestExternal.test_name_pathcsN|d}tfddt|D}|jjdd|d}t|j|ks%JdS)z9 External argument may be an iterable of multiple tuples rc3s|] }|ddfVqdS)Nr.0r/rrr Xsz/TestExternal.test_iter_multi..poorrN)riterryrr$rr)rNrr2rrrtest_iter_multiSs zTestExternal.test_iter_multic Csd}|}t|gft|dgft|dtjgft|fgft|dfgft|dtjdfgft|ddfgffD]"\}}|||jjd||dWdn1sPwYq3dS)z" Test with invalid external lists rrz h5f.UNLIMITEDr!rN)rrMr rrurLrr$)rr0rexc_typerrrr test_invalid\s    zTestExternal.test_invalidN) rrrr+rrrrrrrrrrr s rc@s2eZdZdZd ddZddZddZd d Zd S) TestAutoCreatezP Feature: Datasets auto-created from data produce the correct types TcCsF|j}|t|tjj||||r|s!JdSdSr) rvrxr/r.rhr rzr{r|)rr)r~variablerrrrassert_string_typets  z!TestAutoCreate.assert_string_typecCsxd|jd<||jdtjjddg|jd<||jdtjjtjddgtjd|jd<||jdtjjdS) z Datasets may be resized up to maxshape using integer maxshaper!rr rr)rlNrr1rrrtest_resize_1Ds zTestResize.test_resize_1DcCsL|jjdddd}|t|dWddS1swYdS)z. Resizing past maxshape triggers an exception r!r5rr)rrN)rr$rL Exceptionrr1rrrtest_resize_overs  "zTestResize.test_resize_overcCsH|jdd}|t|dWddS1swYdS)z/ Resizing non-chunked dataset raises TypeError r!r5rN)rr$rLrMrr1rrrtest_resize_nonchunkeds  "z!TestResize.test_resize_nonchunkedcCs2|jjdddd}|jddd||jdd S) z Resize specified axis r!r5rrrraxisrN)rr$rr/r0r1rrrtest_resize_axisr=zTestResize.test_resize_axiscCsP|jjdddd}|t|jdddWddS1s!wYdS) z Illegal axis raises ValueError r!r5rrrrYrN)rr$rLrurr1rrr test_axis_exc s "zTestResize.test_axis_exccCs2|jjdddd}||jd||jddS)z? Allow zero-length initial dims for unlimited axes (issue 111) r!)r)rNrN)rr$r/r0rr1rrr test_zero_dimr=zTestResize.test_zero_dimN) rrrr+rrrrrrrrrrrrrrs   rc@rG) TestDtypezD Feature: Dataset dtype is available as .dtype property cCs(|jddd}||jtddS)z Retrieve dtype from dataset r!r|S10NrGr1rrr test_dtypeszTestDtype.test_dtypecCs<tdtjfdtjfg}|jdd|}||j|dS)z7 Retrieve dtype from complex float16 dataset (gh-2156) rsr!r!rN)rIrHfloat16rr$r/)r complex32r2rrrtest_dtype_complex32#szTestDtype.test_dtype_complex32N)rrrr+rrrrrrr rc@rG)TestLenzG Feature: Size of first axis is available via Python's len cCs"|jdd}|t|ddS)z Python len() (under 32 bits) r!)8rrN)rr$r/rr1rrrtest_len1szTestLen.test_lencCs~|jdd}||jdtjdkr-|t t|Wdn1s'wYn|t|d||ddS)z Python len() vs Dataset.len() r!)riNr) rr$r/r0sysmaxsizerL OverflowErrorrr1rrr test_len_big6s   zTestLen.test_len_bigN)rrrr+rr rrrrr+rrc@rG)TestIterz; Feature: Iterating over a dataset yields rows cCsXtjdddd}|jjd|d}t||D]\}}|t|d|||qdS) z& Iterating over a dataset yields rows rlrrOrmr!rarnN) rIrprrrr$zipr/rrq)rrbr2r/rrrr test_iterHs zTestIter.test_itercCsN|jjddd}|tdd|DWddS1s wYdS)z0 Iterating over scalar dataset raises TypeError r!r)r0cSsg|]}|qSrrrrrr Tsz-TestIter.test_iter_scalar..N)rr$rLrMr1rrrtest_iter_scalarPs "zTestIter.test_iter_scalarN)rrrr+r rrrrrr Bs r c@seZdZdZddZddZddZdd Zd d Zd d Z ddZ ddZ ddZ ddZ ddZddZddZddZddZd d!Zd"d#Zd$S)% TestStringszr Feature: Datasets created with vlen and fixed datatypes correctly translate to and from HDF5 cCsntjdd}|jjdd|d}|j}|t|tjj || tjj t |j }||jddS)z3 Vlen bytes dataset maps to vlen ascii in the file r)r*r/rrON)rh string_dtyperr$rvrxr/r.r rzr{rcheck_string_dtyperHr'rrr)r string_inforrrr^s   zTestStrings.test_vlen_bytescCsptjdd}d}|jjdd||d}||jdd|||jdd|||jdj|dS) & Vlen bytes dataset handles fillvalue r)r*barr/rrrNrhrrr$r/asstrrrrr fill_valuer)rrrtest_vlen_bytes_fillvaluehs z%TestStrings.test_vlen_bytes_fillvaluecCsjt}|jjdd|d}|j}|t|tjj || tjj t |j }||jddS)z5 Vlen unicode dataset maps to vlen utf-8 in the file r/rrOr%N)rhrrr$rvrxr/r.r rzr{rrrHr'rrrrrqs  zTestStrings.test_vlen_unicodecCstt}d}|jjdd||d}||jdd|d||jdd|||jdj|ddS)( Vlen unicode dataset handles fillvalue bárr/rrrr%N)rhrrr$r/encoderrrrrrtest_vlen_unicode_fillvalue{s z'TestStrings.test_vlen_unicode_fillvaluecCstd}|jjdd|d}|j}|t|tj j | | || d||tj jt|j}||jd||jddS)zK Fixed-length bytes dataset maps to fixed-length ascii in the file rr/rrOr?r)N)rIrHrr$rvrxr/r.rhr rz assertFalser|r}r{rrr'rrrrrtest_fixed_asciis   zTestStrings.test_fixed_asciicCsrtjddd}d}|jjdd||d}||jdd|||jdd|||jdj|d S) rr)r?r&rr/rrrNrrrrrtest_fixed_bytes_fillvalues z&TestStrings.test_fixed_bytes_fillvaluecCstjddd}|jjdd|d}|j}||tjj d}| d|d<||d <||g|d d <t j ||gt d|d d <t j | dgd |d|d d <|tt j ||gdd|d d<Wdn1sowYt j|dd t j | dgd dddS)Nr%rr&r/rrOcùrrrYr#r>Ur?S)rhrrr$rvrxr/r{r rrrIrrrLrMrr)rrr)rsrrrtest_fixed_utf8s " 0zTestStrings.test_fixed_utf8cCsztjddd}dd}|jjdd||d}||jdd|||jdd|d||jdj|d S) rr%r?r&rr/rrrN) rhrrrr$r/rrrrrrrtest_fixed_utf_8_fillvalues   z&TestStrings.test_fixed_utf_8_fillvaluecCsLtd}|t|jjdd|d}WddS1swYdS)zA Fixed-length unicode datasets are unsupported (raise TypeError) z|U10r/rrON)rIrHrLrMrr$rrr)rrrtest_fixed_unicodes  "zTestStrings.test_fixed_unicodecCsRtjdd}|jjdd|d}d}||d<|d}|t|t|||dS) zN writing and reading to vlen bytes dataset preserves type and content r)r*r/rrOHellorN)rhrrr$r/r.bytesrrr)rboutrrrtest_roundtrip_vlen_bytess z%TestStrings.test_roundtrip_vlen_bytescCsRtd}|jjdd|d}d}||d<|d}|t|tj|||dS)z[ Writing to and reading from fixed-length bytes dataset preserves type and content rr/rrOr+rN)rIrHrr$r/r.rr-rrrtest_roundtrip_fixed_bytess z&TestStrings.test_roundtrip_fixed_bytescCs^t}|jjdd|d}d}||d<||dt|d}||t|||dS)Nr/rrOfàilter) rhrrr$r%r,rr'r/r-rrrtest_retrieve_vlen_unicodes  z&TestStrings.test_retrieve_vlen_unicodecCs|jjddtd}d}||d<|d}|t |d}Wdn1s*wY||dddd||d d||d t |t j |dd t j |gtdt j t |dd t j |gtddS) Nr/rrOr1rr)ignorefiltelatin-1r?r)rr$rhrrrLUnicodeDecodeErrorr/rrrIrrrrasarray)rr)rbstrwrap1r.rrr test_asstrs    "zTestStrings.test_asstrcCstjdd}|jjdd|d}d}tj|d|d|d<||dtj| d}||t | ||| | d d dd | | d d|tj | dd tj|gtddS)Nrrr/rrOr"r%rr)r3rr5r)rhrrr$rIrrr%bytes_rr'r/rrrrr-rrrtest_asstr_fixeds    zTestStrings.test_asstr_fixedcCsXtd}|jjdd|d}d}|t ||d<WddS1s%wYdS)zGEncoding error when writing a non-ASCII string to an ASCII vlen datasetr)r/rrOr1rN)rhrrr$rLUnicodeEncodeError)rrr)rbrrrtest_unicode_write_error s   "z$TestStrings.test_unicode_write_errorcCs\t}|jjdd|d}dtdd}||d<|d}|t|t|||dS) zJ Writing valid utf-8 byte strings to a unicode vlen dataset is OK r/rrOrrutf8rN) rhrrr$rrr/r.r,r-rrrtest_unicode_write_bytessz$TestStrings.test_unicode_write_bytescCsVtd}|jjdd|d}d}||d<|d}|t|t|||ddS)z: Writing an ascii str to ascii vlen dataset is OK r)r/rrOz ASCII stringrN)rhrrr$r/r.r,rr-rrrtest_vlen_bytes_write_ascii_strs z+TestStrings.test_vlen_bytes_write_ascii_strN)rrrr+rrrrr r!r'r(r*r/r0r2r9r;r=r?r@rrrrrWs&          rc@(eZdZdZddZddZddZdS) TestCompoundz: Feature: Compound types correctly round-trip c Cstdtjfdtjfdtjfdtjfdtjfdtjfdtjfg}tjd|d }|jD] }tjdd ||<q+||jd <|jd d }| t ||k| |j|jd S)z: Compound types are read back in correct order (issue 236)weightcputimewalltimeparents_offset n_parentsstatus endpoint_typer\rOrtest.N) rIrHfloat64uint32uint8rfieldsrurrir r/)rrrUkeyoutdatarrrtest_rt0s   zTestCompound.test_rtcCstdtjdffdtjfg}tjd|d}|jD]}tjj||jdd||<q|jj dd|d}|jD]}||||<q6|jdd }| t ||k| |j|jdS) NrCrnrIrJrOr{rrK.) rIrHrLrNrrOrur0rr$rir r/)rrrUrPr)rQrrr test_assignDs  zTestCompound.test_assigncCs>tdtjfdtjfdtjfg}tjd|d}|jD] }tjdd||<q||jd<tj|jdddgdd|ddgtj|jdddd|dtjt |jdddg|ddgtdtj fg}tjtj |jddg|d|dg |t |jdddksJdS) Nr/rrrJrOrrKr\) rIrHrLrrOrurrrr7rastyper)rrrUrPdt_intrrr test_fieldsUs0  $" zTestCompound.test_fieldsN)rrrr+rRrSrVrrrrrB*s  rBc@r) TestSubarraycCsl|jjdddd}gd|d<tj|ddgdggdg|dd<tj|ddgdgdSNrr53int8rOrrYrnr)r#rr>)rr$rIrrr(rrrtest_write_listzs   zTestSubarray.test_write_listcCsx|jjdddd}tgd|d<tj|ddgdgtgdg|dd<tj|ddgdgdSrX)rr$rIrrrr(rrrtest_write_arrays  zTestSubarray.test_write_arrayN)rrrr[r\rrrrrWys rWc@s,eZdZdZddddZddZdd Zd S) TestEnumzM Feature: Enum datatype info is preserved, read/write as integer rrr")REDGREENBLUEcCsDtj|jdd}|jjdd|d}|j}t|}|||jdS)z= Enum datasets can be created and type correctly round-trips r!basetyper/rrON)rh enum_dtypeEDICTrr$rHcheck_enum_dtyper/)rrr)dt2dict2rrrrs  zTestEnum.test_createcCsptj|jdd}|jjdd|d}d|d<d|dd d f<||dd||dd d ftjd ddd S) z/ Enum datasets can be read/written as integers rrar/rrOr")#%rN)drrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr) rhrcrdrr$r/rqrIrr)rrrtest_readwrites &zTestEnum.test_readwriteN)rrrr+rdrrjrrrrr]s   r]c@s:eZdZdZddZeeeddddZ dd Z d S) TestFloatszA Test support for mini and extended-precision floats cCsJ|jjdd|d}||j|tjd|d}||d<||d|dS)Nr/rrO.)rr$r/rHrIrdrq)rrr2rbrrr _exectests zTestFloats._exectestrzNumPy float16 support requiredcCs|tddS)z Mini-floats round trip rN)rlrIrHrrrr test_miniszTestFloats.test_minicCs>ttdr|tjjtddS|tjjtddS)z Test mapping for float16 rzz+Issue with empty region refs in HDF5 1.10.6reasonc@s@eZdZdZddZddZeddZedd Zd d Z d S) TestRegionRefsz3 Various features of region references cCs@t|tdd|_|jjd|jd|_|j|jd<dS)Ni'rr/ra.) rrrIrprrrbrr$r2rrrrrs zTestRegionRefs.setUpcCs@tjdddddf}|jj|}||j||j|dS)z4 Region references can be used as slicing arguments rrhr?rrN)rIrr2 regionrefrqrbrslicrefrrrtest_create_refs zTestRegionRefs.test_create_refcCs,|jjdd}|j|}|jdksJdS)Nr)r2rrB)rrr.rrrtest_empty_regions z TestRegionRefs.test_empty_regioncCs|jjdddd}tjtjj}|tj|jdtjj |}||t t dks/J| tj|jdtjj |}|||dksIJdS)Nscalar?rrbrH.r)rr$rhh5screateSCALAR select_noneh5rrvDATASET_REGIONrrIrH select_all)rr)sidrrrrtest_scalar_dataset sz"TestRegionRefs.test_scalar_datasetcCsXtjdddddf}|jj|}||jj||jj||jj|ddS)z, Region reference shape and selection shape rrhr?rr)r?N)rIrr2rr/r0 selectionrrrrtest_ref_shapes zTestRegionRefs.test_ref_shapeN) rrrr+rrempty_regionref_xfailrrrrrrrrs   rc@rA) TestAstypez(.astype() wrapper & context manager cCsL|jjdddd}td|d<|ddd}||tjddddSNr/rrrOr.r)rr$rIrprTrqrr2rrrrtest_astype_wrapper'szTestAstype.test_astype_wrappercCs:|jjdddd}td|d<|dt|ddSr)rr$rIrpr/rrTr1rrrtest_astype_wrapper_len.sz"TestAstype.test_astype_wrapper_lencCsN|jjdddd}td|d<tj|ddd}||tjddddSr)rr$rIrpr7rTrqrrrrtest_astype_wrapper_asarray3sz&TestAstype.test_astype_wrapper_asarrayN)rrrr+rrrrrrrr$s  rc@r)TestScalarCompoundzl Retrieval of a single field from a scalar compound dataset should strip the field info cCs:tdg}|jjdd|d}||djtddS)Nr r/rrOrr!)rIrHrr$r/)rrr2rrrtest_scalar_compoundAs z'TestScalarCompound.test_scalar_compoundN)rrrr+rrrrrr:s rc@sfeZdZddZddZddZddZd d Zdd d ZddZ ddZ ddZ ddZ ddZ dS)TestVlencCsRtt}|jjdd|d}td|d<td|d<gd|d<td|d<||dtd||dtd||dtgd||dtdtjtd td gt d|dd<||dtd ||dtd ttdtdg|dd<||dtd||dtddS) Nvlenr"rOrnrrrZrYrr#) rh vlen_dtyper}rr$rIrprqrrr)rrrrIs  &"zTestVlen.test_intcCs6tt}|jjdd|d}|jdd|djdS)Nrr5rOvlen2r)rhrr}rr$rHr)rrrtest_reuse_from_other[s zTestVlen.test_reuse_from_othercCsldtfdttfg}|jjdd|d}|jj}|jt|d|_|jdd|jdddjdS)Nrrrr5rOrr) r}rhrrr$rwrrrH)rrr)fnamerrrtest_reuse_struct_from_other`s  $z%TestVlen.test_reuse_struct_from_othercCs@tt}|jjdd|d}tddg|d<tdg|d<gd|d <||dtddg||dtdg||d tgd tjgd td gt d|dd <||dtd ||dtd ttgdtgdg|dd <||dtd||dtddS)NrrrOgffffff?333333?rr)rrYrnrYrZ)皙?g?g@g@r#r#r)rr皙@)皙?rrrn) rhrr}rr$rIrrqrprr)rrr test_converths  $ zTestVlen.test_convertcCstt}|jjdd|d}td|d<tjtdtdgtdtdggtd|ddddf<ttdtdgtdtdgg|ddddf<dS)Nr)rYrYrOrryrnrY) rhrr}rr$rIrprrr)rrr test_multidimys zTestVlen.test_multidimrc CsPt|}|jj|d|d}tjgd|d}tjgd|d}tjgdtdd}||}gd} tj| |d} gd} tj| |d} ||d <||d <||d <| |d <| |d <|||d |||d |||d || |d || |d tjddg|d} | dd|d <|| |d |j |j dS)z Helper for testing various vlen numpy data types. :param np_dt: Numpy datatype to test :param dataset_name: String name of the dataset to create for testing. rrO)r@g>@)g33333Y@g i@gfffffX@g%gr)rrYr#r)rrg @r$g)rirri'rrrrYrnr#g333333?rN) rhrrr$rIrrHrTrqrTr) rnp_dt dataset_namerr)array_0array_1array_2casted_array_2list_3 list_array_3list_4 list_array_4rrr_help_float_testings2   zTestVlen._help_float_testingcCtd}||dS)NrrIrHrrrrrrtest_numpy_float16 zTestVlen.test_numpy_float16cCr)Nfloat32rrrrrtest_numpy_float32rzTestVlen.test_numpy_float32cCr)NrLrrrrrtest_numpy_float64_from_dtyperz&TestVlen.test_numpy_float64_from_dtypecCstj}||dSr)rIrLrrrrrtest_numpy_float64_2szTestVlen.test_numpy_float64_2cCs |jjddtddtgd}|ddd|jdd<t|jdd|dddksBJ|jddd |ddd|jjd dtd dtgd }|ddd|jd d<t|jd d|dddksJ|jd dd |ddddS) z4Test that non-contiguous arrays are stored correctlyncrrprO)TFTTFFFNrYrz != nc2int8)rYr#rrrrnr)rr$rhrrIrr )rr/rrrrtest_non_contiguous_arrayssFJz#TestVlen.test_non_contiguous_arraysN)r)rrrrrrrrrrrrrrrrrrrHs 0 rc@r) TestLowOpencCs|jdd}|j}dS)z Test H5Dget_access_plist r!r"N)rr$rvr)rr)p_listrrrtest_get_access_listsz TestLowOpen.test_get_access_listcCsHtjtjj}|jdd}~tj|jjd|}| |tjj dS)z# Test the dapl keyword to h5d.open r/rxN) rhh5prDATASET_ACCESSrr$h5drrvr% DatasetID)rdaplr2dsidrrr test_dapls zTestLowOpen.test_daplN)rrrrrrrrrrs r)rr?rz#chunk info requires HDF5 >= 1.10.5c Csbddlm}|}t|d}|jdddddd |ddd<Wdn1s+wY|dt|d l}|dj}|d ksIJtd D]0}t t t |dd }| |}|j|kshJ|jdksoJ|jdusvJ|jdks}JqM|d }|jd ksJ|jdksJ|jdusJ|jdksJWddS1swYdS)NrBytesIOrrKrrgrr0rrHrrsrr?ry)iorrhrr$seekrvget_num_chunksryrrIr unravel_indexget_chunk_info chunk_offset filter_mask byte_offsetrBget_chunk_info_by_coord)rbuffoutfinr)joffsetsirrrtest_get_chunk_detailss.      "r)rr rn)rr?r?)rr?czJchunk iteration requires HDF5 1.10.10 and later 1.10, or 1.12.3 and latercsddlm}|}t|d}|jdddddd |dd d <Wd n1s+wY|dt|d 6}|dj}|}|d ksKJit|D] }| |}||j <qQfd d}| |Wd d S1stwYd S)z%H5Dchunk_iter() for chunk informationrrrrKrrgrrrNrsrcsN|j}|j|jks J|j|jksJ|j|jksJ|j|jks%JdSr)rrrrB) chunk_infoknowncirrcallbacks z!test_chunk_iter..callback) rrrhrr$rrvrryrr chunk_iter)rrrr num_chunksrrrrrrtest_chunk_iters$         "rcCs.|jddd}|jdusJ|jdusJdS)Nr{rrO)r$r0r)rr)rrrtest_empty_shapesrcCsddlm}|}t|d}|jdddWdn1s!wY|dt|d(}|djdus;J|djdusFJ|dj dksQJWddS1s\wYdS)Nrrrr{rNrOrs) rrrhrr$rrrv get_offsetget_storage_size)rrrrrrrtest_zero_storage_size%s  "rcCsttjjttjjdg}|jd|tjd}|jttjks$Jtj|ddtj |tjd||dd<tj|ddtj |tjddS)Nrr/rrO) rIiinformaxr$uint64rHrrr)rrbr)rrrtest_python_int_uint643s " &rcCs<|jddtjd}tjjddd}||ddgddf<dS) Nrb)rrrYrO)rrnrYr{)rrYr#.)r$rIrNrur|)rrblockrrrtest_setitem_fancy_indexingAsrcCFttd}|dddksJWddS1swYdS)Nzvlen_string_dset.h5DS1rsPartingrrrrrrtest_vlen_spacepadH"rcCr)Nzvlen_string_dset_utc.h5ds1rs2009-12-20T10:16:18.662409Zrrrrrtest_vlen_nulltermMrr)rr?rnz>Appears you cannot pass an unknown filter id for HDF5 < 1.10.3cCs0d}|jddtj|dd}t||jvsJdS)NrbrgT)r0rHr7allow_unknown_filter)r$rIrNr'_filters)rfake_filter_idr)rrrtest_allow_unknown_filterRs  rc Csddlm}|}t|d0}|jdddddd d d }|j}|jd d |ks2J|dks8JWd n1sBwY| dt|d0}|j ddddddd}|j}|jd d |kstJ|dkszJWd d S1swYd S)z2Chunk cache configuration for individual datasets.rrrr/)r?r)rr#r r)r0rrH rdcc_nbytesrdcc_w0 rdcc_nslotsrN)rrrrs0q= ףp?)r0rHrrr)r r r ) rrrhrr$rvrget_chunk_cache get_cacherr)rrrr)ds_chunk_cacherrrrtest_dset_chunk_cache`s*  "rc@rG)TestCommutativez| Test the symmetry of operators, at least with the numpy types. Issue: https://github.com/h5py/h5py/issues/1947 cCsd}|jjd|ttjj|d}t|d}t||k||kks$Jt||k||kks1Jd}t||}t||k||kksGJt||k||kksTJdS)z Create a h5py dataset, extract one element convert to numpy Check that it returns symmetric response to == and != rrrKrHrbrgMbP?N) rr$rlrIrurvrLr nanmax)rr0r2valdeltanvalrrrtest_numpy_commutative{s  z&TestCommutative.test_numpy_commutativecCsTd}|jjd|ttjj|d}td}||k||kksJ||k||kks(JdS)z Create a h5py dataset and check basetype compatibility. Check that operation is symmetric, even if it is potentially not meaningful. rrKrr$N)rr$rlrIrurv)rr0r2rrrrtest_basetype_commutatives  z)TestCommutative.test_basetype_commutativeN)rrrr+rrrrrrrvs rc@sHeZdZdZejejdkddddZejejdkddddZ d S) TestVirtualPrefixz% Test setting virtual prefix )rr?rYz8Virtual prefix does not exist before HDF5 version 1.10.2r}cCsRd}d}|jjd|ttjj||d}t|j  }||ks'JdS)Nr/path/to/virtualrK)rHrbvirtual_prefix) rr$rlrIrurvrrrvrget_virtual_prefixrr)rr0rr2virtual_prefix_readbackrrrtest_virtual_prefix_creates  z,TestVirtualPrefix.test_virtual_prefix_createcCs^d}|jjddd|d}t|j}| ||| |t | |j ddS)Nrr!rmr)r) rrrrrvrrrrr/r%rr0)rrr2rrrrtest_virtual_prefix_requires   z-TestVirtualPrefix.test_virtual_prefix_requireN) rrrr+rr]rrrrrrrrrs   r)ar+rrrnumpyrIrRrTwarningscommonrr data_filesrrhrrr h5py._hl.baser r r r h5py.h5py_warningsr rh5py._hl.selectionsr> selectionsselrr r,r_rrrrrrrr}rrrlrrLrr+r,r]filtersrr4rHrRrZr`rerhrrrrrrrr rrBrWr]rkrqrwrUrrrrrrrrprrrrrrrrskipifrrrrrrrrs       FY0(CC&  ,'   qc,DTO 0      -