cloudsync.Provider

class cloudsync.Provider

File storage provider.

Override this to implement a provider capable of using the sync engine.

Implementors are responsible for normalizing behavior, errors thrown, any needed caching.

Some helpers are provided in this base class for oauth and path manipulation.

alt_sep = '\\'

Alternate path delimiter

authenticate() → Dict[str, Union[str, int]]

Authenticate a connection.

Returns:

Creds: A JSON serializable object that can be used to log in.

Raises:

CloudTokenError on failure

basename(path: str)

Just like os.basename, but for provider paths.

case_sensitive = True

Provider is case sensitive

connect(creds)

Connect to provider.

Generally providers should overload connect_impl, instead.

connect_impl(creds) → str

Connection implementation.

Some providers don’t need connections, so just don’t implement/overload this method.

Returns:

Unique connection id that should be the same each time the same user connects. A combination of a provider name and a login/userid could be sufiicient, but it is suggested to use a provider specific identity, if available.

property connected

True if connected, false if not.

If False, any use of the provider except the connect() function, must raise a CloudDisconnectedError

connection_id = None

Must remain constant between logins and must be unique to the login

abstract create(path, file_like: BinaryIO, metadata=None) → dataclasses.dataclass

Create a file at the specified path, setting contents and optionally setting metadata

abstract property current_cursor

Get the current cursor for the events generator

default_sleep = 0.01

Per event loop sleep time

abstract delete(oid)

Delete an object

dirname(path: str)

Just like os.dirname, but for provider paths.

disconnect()

Invalidates current connection, closes sockets, etc.

abstract download(oid, file_like: BinaryIO)

Get the bytes of a specified object id

download_path(path, io)

abstract events() → Generator[[dataclasses.dataclass, None], None]

Yields events, possibly forever.

If stopped, the event poller will sleep for self.default_sleep, and call this again.

abstract exists_oid(oid) → bool

Returns true of object exists with specified oid

abstract exists_path(path) → bool

Returns true of object exists at the specified path

get_quota() → dict

Returns a dict with of used (bytes), limit (bytes), optional login, and possibly other provider-specific info

globalize_oid(oid: str) → str

Converts an oid that may be account specific to one that can be used in other accounts.

abstract hash_data(file_like: BinaryIO) → Union[Dict[str, Union[Dict[str, Hash], Tuple[Hash, ...], str, int, bytes, float, None]], Tuple[Union[Dict[str, Hash], Tuple[Hash, ...], str, int, bytes, float, None], ...], str, int, bytes, float, None]

Returns a provider specific hash from data

hash_oid(oid) → Union[Dict[str, Union[Dict[str, Hash], Tuple[Hash, ...], str, int, bytes, float, None]], Tuple[Union[Dict[str, Hash], Tuple[Hash, ...], str, int, bytes, float, None], ...], str, int, bytes, float, None]

Returns a provider specific hash associated with the object referred to

abstract info_oid(oid: str, use_cache=True) → Optional[dataclasses.dataclass]

Returns info for an object with specified oid, or None if not found

abstract info_path(path: str, use_cache=True) → Optional[dataclasses.dataclass]

Returns info for an object at a path, or None if not found

interrupt_auth()

Iterrupt/stop a blocking authentication call.

is_subpath(folder, target, strict=False)

True if the target is within the folder.

Args:

folder: the directory target: the potential sub-file or folder strict: whether to return True if folder==target

is_subpath_of_root(target, strict=False)

True if the target is within the root folder.

Args:

folder: the directory target: the potential sub-file or folder strict: whether to return True if folder==root

classmethod join(*paths: Union[str, List[str], Tuple[str]])

Joins a list of path strings in a provider-specific manner.

Args:

paths: zero or more paths

large_file_size = 0

Used for testing providers with separate large file handling

abstract property latest_cursor

Get the latest cursor as of now.

list_ns(recursive: bool = True, parent: dataclasses.dataclass = None) → List[dataclasses.dataclass]

Yield one entry for each namespace supported, or None if namespaces are not needed

abstract listdir(oid) → Generator[[dataclasses.dataclass, None], None]

Yield one entry for each file at the directory pointed to by the specified object id

listdir_oid(oid, path=None) → Generator[[dataclasses.dataclass, None], None]

listdir_path(path) → Generator[[dataclasses.dataclass, None], None]

localize_oid(global_oid: str)

Converts a globalized oid to one that can be used locally.

All regular provider functions use ‘local oids’ only unless otherwise specified.

abstract mkdir(path) → str

Create a folder

mkdirs(path)

Makes a directory and intervening directories, returns the oid of the leaf

name = None

Provider name

property namespace

Some providers have multiple ‘namespaces’, that can be listed and changed.

Cannot be set when not connected.

property namespace_id

Unique id corresponding to a namespace name.

Can be set when not connected.

normalize_path(path: str, for_display: bool = False)

Used internally for comparing paths in a case and sep insensitive manner, as appropriate.

Args:

path: the path to normalize for_display: when true, preserve case of path’s leaf node

classmethod normalize_path_separators(path: str)

Normalizes path separators only.

Replaces alternate separators with primary, strips separators from end of path string.

classmethod oauth_test_instance(prefix: str, token_key='refresh_token', token_sep='|', port_range: Tuple[int, int] = None, host_name=None)

Helper function for oauth providers.

Args:

prefix: environment variable prefix token_key: creds dict key token_sep: multi-env var token separator port_range: if any, specify tuple

oid_is_path = False

Objects stored in cloud are only referenced by path

paths_match(patha, pathb, for_display=False)

True if two paths are equal, uses normalize_path().

reconnect()

Reconnect to provider, using existing creds.

If a provider was previously connected, it should retain the creds used. This function should restore the connection if the creds are still valid

Raises:

CloudDisconnectedError on failure

abstract rename(oid, path) → str

Rename an object to specified path

replace_path(path, from_dir, to_dir)

Replaces from_dir with to_dir in path, but only if from_dir is_subpath of path.

rmtree(oid)

Recursively remove all folders including the folder/file specified.

Override this if your provider has a more efficient implementation.

property root_oid

The root oid, if any

property root_path

The root path, if any

sep = '/'

Path delimiter

set_creds(creds)

Set credentials without connecting.

set_root(root_path=None, root_oid=None)

Set sync root path and oid. Once set, these values cannot be changed.

split(path)

Splits a path into a dirname, filename, just like 1os.path.split()1

sync_state = None

Access to sync engine state for this provider

classmethod test_instance()

Override to enable CI testing of your class, see oauth_test_instance code for an example

Returns:

Provider: an instance of an provider, with “creds” set to the creds blob

test_root = '/'

Root folder to use during provider tests

test_short_poll_only(short_poll_only: bool)

abstract upload(oid, file_like: BinaryIO, metadata=None) → dataclasses.dataclass

Upload a filelike to an existing object id, optionally setting metadata

upload_block_size = 0

Used for testing providers with separate large file handling

classmethod uses_oauth()

Return True if provider uses OAuthConfig initialization

walk(path, recursive=True)

List all files recursively, yielded as events

walk_oid(oid, recursive=True)

List all files recursively, yielded as events

win_paths = False

C: drive letter stuff needed for paths