Skip to main content

handlers

I/O handlers for various file formats

Handler Objects#

class Handler(ABC)

Base class for file type loaders

ABC for loaders

load#

def load(path: Path, s3_config: Optional[_T] = None) -> Dict

Load function for file type

This handles s3 path conversion for all handler types pre load call

Arguments:

  • path - path to file
  • s3_config - optional s3 config object if using s3 storage

Returns:

dictionary of read file

_post_process_config_paths#

@staticmethoddef _post_process_config_paths(payload)

Transform path string into path object

_load#

@abstractmethoddef _load(path: str) -> Dict

Private load function for file type

Arguments:

  • path - path to file

Returns:

dictionary of read file

_write_crypto#

def _write_crypto(value: Union[str, ByteString], path: Path, name: str, crypto_name: str, create_path: bool, s3_config: Optional[_T])

Write values of the underlying cryptography data used to encode some spock values

Arguments:

  • value - current crypto attribute
  • path - path to write out
  • name - spock generated file name
  • create_path - boolean to create the path if non-existent (for non S3)
  • s3_config - optional s3 config object if using s3 storage
  • crypto_name - name of the crypto attribute

Returns:

None

save#

def save(out_dict: Dict, info_dict: Optional[Dict], library_dict: Optional[Dict], path: Path, name: str, create_path: bool = False, s3_config: Optional[_T] = None, salt: Optional[str] = None, key: Optional[ByteString] = None)

Write function for file type

This will handle local or s3 writes with the boolean is_s3 flag. If detected it will conditionally import the necessary addons to handle the upload

Arguments:

  • out_dict - payload to write
  • info_dict - info payload to write
  • library_dict - package info to write
  • path - path to write out
  • name - spock generated file name
  • create_path - boolean to create the path if non-existent (for non S3)
  • s3_config - optional s3 config object if using s3 storage
  • salt - string of the salt used for crypto
  • key - ByteString of the key used for crypto

_check_s3_write#

@staticmethoddef _check_s3_write(write_path: str, path: Path, name: str, s3_config: Optional[_T])

Handles writing to S3 if necessary

Arguments:

  • write_path - path the file was written to locally
  • path - original path specified
  • name - original file name
  • s3_config - optional s3 config object if using s3 storage

_save#

@abstractmethoddef _save(out_dict: Dict, info_dict: Optional[Dict], library_dict: Optional[Dict], path: str) -> str

Write function for file type

Arguments:

  • out_dict - payload to write
  • info_dict - info payload to write
  • library_dict - package info to write
  • path - path to write out

_handle_possible_s3_load_path#

@staticmethoddef _handle_possible_s3_load_path(path: Path, s3_config: Optional[_T] = None) -> Union[str, Path]

Handles the possibility of having to handle loading from a S3 path

Checks to see if it detects a S3 uri and if so triggers imports of s3 functionality and handles the file download

Arguments:

  • path - spock config path
  • s3_config - optional s3 configuration object

Returns:

  • path - current path for the configuration file

_handle_possible_s3_save_path#

@staticmethoddef _handle_possible_s3_save_path(path: Path, name: str, create_path: bool, s3_config: Optional[_T] = None) -> Tuple[str, bool]

Handles the possibility of having to save to a S3 path

Checks to see if it detects a S3 uri and if so generates a tmp location to write the file to pre-upload

Arguments:

  • path - save path
  • name - spock generated file name
  • create_path - create the path for non s3 data
  • s3_config - s3 config object

write_extra_info#

@staticmethoddef write_extra_info(path: str, info_dict: Dict, version: bool = True, write_mode: str = "w+", newlines: Optional[int] = None, header: Optional[str] = None)

Writes extra info to commented newlines

Arguments:

  • path - path to write out
  • info_dict - info payload to write
  • version - write the spock version string first
  • write_mode - write mode for the file
  • newlines - number of new lines to add to start

YAMLHandler Objects#

class YAMLHandler(Handler)

YAML class for loading YAML config files

Base YAML class

_load#

def _load(path: str) -> Dict

YAML load function

Arguments:

  • path - path to YAML file

Returns:

  • base_payload - dictionary of read file

_save#

def _save(out_dict: Dict, info_dict: Optional[Dict], library_dict: Optional[Dict], path: str) -> str

Write function for YAML type

Arguments:

  • out_dict - payload to write
  • info_dict - info payload to write
  • library_dict - package info to write
  • path - path to write out

TOMLHandler Objects#

class TOMLHandler(Handler)

TOML class for loading TOML config files

Base TOML class

_load#

def _load(path: str) -> Dict

TOML load function

Arguments:

  • path - path to TOML file

Returns:

  • base_payload - dictionary of read file

_save#

def _save(out_dict: Dict, info_dict: Optional[Dict], library_dict: Optional[Dict], path: str) -> str

Write function for TOML type

Arguments:

  • out_dict - payload to write
  • info_dict - info payload to write
  • library_dict - package info to write
  • path - path to write out

JSONHandler Objects#

class JSONHandler(Handler)

JSON class for loading JSON config files

Base JSON class

_load#

def _load(path: str) -> Dict

JSON load function

Arguments:

  • path - path to JSON file

Returns:

  • base_payload - dictionary of read file

_save#

def _save(out_dict: Dict, info_dict: Optional[Dict], library_dict: Optional[Dict], path: str) -> str

Write function for JSON type

Arguments:

  • out_dict - payload to write
  • info_dict - info payload to write
  • library_dict - package info to write
  • path - path to write out