AzureDLFile Class

  • Reference

Open ADL key as a file. Data is only loaded and cached on demand.

Inheritance
builtins.object
AzureDLFile

Constructor

AzureDLFile(azure, path, mode='rb', blocksize=33554432, delimiter=None)

Parameters

Name Description
azure
Required
<xref:<xref:azure.datalake.store.core.azure connection>>
path
Required
AzureDLPath

location of file
mode
Default value: rb
blocksize
int

Size of the write or read-ahead buffer. For writing(and appending, will be truncated to 4MB (2**22).

Default value: 33554432
delimiter
bytes or None

If specified and in write mode, each flush will send data terminating on this bytestring, potentially leaving some data in the buffer.

Default value: None

Examples


>>> adl = AzureDLFileSystem()  
>>> with adl.open('my-dir/my-file.txt', mode='rb') as f:  
...     f.read(10)

Methods

close

Close file

If in write mode, causes flush of any unwritten data.
flush

Write buffered data to ADL.

Without delimiter: Uploads the current buffer.

With delimiter: writes an amount of data less than or equal to the block-size, which ends on the delimiter, until buffer is smaller than the blocksize. If there is no delimiter in a block uploads whole block.

If force=True, flushes all data in the buffer, even if it doesn't end with a delimiter; appropriate when closing the file.
info

File information about this path
next
read

Return data from cache, or fetch pieces as necessary
read1

Return data from cache, or fetch pieces as necessary
readable

Return whether the AzureDLFile was opened for reading
readinto

Reads data into buffer b
readline

Read and return a line from the stream.

If length is specified, at most size bytes will be read.
readlines

Return all lines in a file as a list
seek

Set current file location
seekable

Return whether the AzureDLFile is seekable (only in read mode)
tell

Current file location
writable

Return whether the AzureDLFile was opened for writing
write

Write data to buffer.

Buffer only sent to ADL on flush() or if buffer is bigger than blocksize.

close

Close file

If in write mode, causes flush of any unwritten data.

close()

flush

Write buffered data to ADL.

Without delimiter: Uploads the current buffer.

With delimiter: writes an amount of data less than or equal to the block-size, which ends on the delimiter, until buffer is smaller than the blocksize. If there is no delimiter in a block uploads whole block.

If force=True, flushes all data in the buffer, even if it doesn't end with a delimiter; appropriate when closing the file.

flush(syncFlag='METADATA', force=False)

Parameters

Name Description
syncFlag
Default value: METADATA
force
Default value: False

info

File information about this path

info()

next 

next()

read

Return data from cache, or fetch pieces as necessary

read(length=-1)

Parameters

Name Description
length
int(<xref:-1>)

Number of bytes to read; if <0, all remaining bytes.

Default value: -1

read1

Return data from cache, or fetch pieces as necessary

read1(length=-1)

Parameters

Name Description
length
int(<xref:-1>)

Number of bytes to read; if <0, all remaining bytes.

Default value: -1

readable

Return whether the AzureDLFile was opened for reading

readable()

readinto

Reads data into buffer b

readinto(b)

Parameters

Name Description
b
Required
bytearray

Buffer to which bytes are read into

Returns

Type Description
<xref:Returns> <xref:number> <xref:of> bytes <xref:read.>

readline

Read and return a line from the stream.

If length is specified, at most size bytes will be read.

readline(length=-1)

Parameters

Name Description
length
Default value: -1

readlines

Return all lines in a file as a list

readlines()

seek

Set current file location

seek(loc, whence=0)

Parameters

Name Description
loc
Required
int

byte location
whence

from start of file, current location or end of file, resp.

Default value: 0

seekable

Return whether the AzureDLFile is seekable (only in read mode)

seekable()

tell

Current file location

tell()

writable

Return whether the AzureDLFile was opened for writing

writable()

write

Write data to buffer.

Buffer only sent to ADL on flush() or if buffer is bigger than blocksize.

write(data)

Parameters

Name Description
data
Required
bytes

Set of bytes to be written.