pythonfix
Packages Code Examples Errors News Tags Blog
pythonfix

Contents

black library internals

0
 Python Package Internals
   1797 words   9 minutes   0 views 
Contents

This page is all about the black python library, the contents below is auto-generated by inspecting a python package’s code base.

This page is experimental: if you are seeking help with the `black` python package it's usually best to reference the libraries offical documentation page.

Package Files

  • black/__init__.py
  • black/__main__.py
  • black/brackets.py
  • black/cache.py
  • black/comments.py
  • black/concurrency.py
  • black/const.py
  • black/debug.py
  • black/files.py
  • black/handle_ipynb_magics.py
  • black/linegen.py
  • black/lines.py
  • black/mode.py
  • black/nodes.py
  • black/numerics.py
  • black/output.py
  • black/parsing.py
  • black/py.typed
  • black/report.py
  • black/rusty.py
  • black/strings.py
  • black/trans.py

Package Classes

  • <enum ‘Changed’>
  • <class ‘black.lines.EmptyLineTracker’>
  • <class ‘str’>
  • <enum ‘Enum’>
  • <class ‘concurrent.futures._base.Executor’>
  • <enum ‘Feature’>
  • <class ‘str’>
  • <class ‘black.mode.Mode’>
  • <class ‘pathspec.patterns.gitwildmatch.GitWildMatchPatternError’>
  • <class ‘black.parsing.InvalidInput’>
  • <class ‘json.decoder.JSONDecodeError’>
  • <class ‘blib2to3.pytree.Leaf’>
  • <class ‘black.lines.Line’>
  • <class ‘black.linegen.LineGenerator’>
  • <class ‘black.mode.Mode’>
  • <class ‘str’>
  • <class ‘blib2to3.pytree.Node’>
  • <class ‘black.report.NothingChanged’>
  • <class ‘pathlib.Path’>
  • <class ‘concurrent.futures.process.ProcessPoolExecutor’>
  • <class ‘black.report.Report’>
  • <enum ‘TargetVersion’>
  • <class ‘concurrent.futures.thread.ThreadPoolExecutor’>
  • <enum ‘WriteBack’>
  • <class ‘datetime.datetime’>

Package Methods

black.assert_equivalent()

black.assert_equivalent(src: str, dst: str, *, pass_num: int = 1) -> None

black.assert_equivalent(
  src,
  dst,
  pass_num,
)

black.assert_stable()

black.assert_stable(src: str, dst: str, mode: black.mode.Mode) -> None

black.assert_stable(
  src,
  dst,
  mode,
)

black.cancel()

black.cancel(tasks: Iterable[ForwardRef('asyncio.Task[Any]')]) -> None

black.cancel(
  tasks,
)

black.check_stability_and_equivalence()

Raise AssertionError if source and destination contents are not equivalent, or if a second pass of the formatter would format the content differently.

black.check_stability_and_equivalence(src_contents: str, dst_contents: str, *, mode: black.mode.Mode) -> None

black.check_stability_and_equivalence(
  src_contents,
  dst_contents,
  mode,
)

black.color_diff()

black.color_diff(contents: str) -> str

black.color_diff(
  contents,
)

black.contextmanager()

black.contextmanager(func)

black.contextmanager(
  func,
)

black.decode_bytes()

newline is either CRLF or LF but decoded_contents is decoded with universal newlines (i.e. only contains LF).

black.decode_bytes(src: bytes) -> Tuple[str, str, str]

black.decode_bytes(
  src,
)

black.detect_target_versions()

black.detect_target_versions(node: blib2to3.pytree.Node) -> Set[black.mode.TargetVersion]

black.detect_target_versions(
  node,
)

black.diff()

black.diff(a: str, b: str, a_name: str, b_name: str) -> str

black.diff(
  a,
  b,
  a_name,
  b_name,
)

black.dump_to_file()

black.dump_to_file(*output: str, ensure_final_newline: bool = True) -> str

black.dump_to_file(
  output,
  ensure_final_newline,
)

black.err()

black.err(message: Optional[str] = None, nl: bool = True, **styles: Any) -> None

black.err(
  message,
  nl,
  styles,
)

black.filter_cached()

The first contains paths of files that modified on disk or are not in the cache. The other contains paths to non-modified files.

black.filter_cached(cache: Dict[str, Tuple[float, int]], sources: Iterable[pathlib.Path]) -> Tuple[Set[pathlib.Path], Set[pathlib.Pa
th]]

black.filter_cached(
  cache,
  sources,
)

black.find_pyproject_toml()

black.find_pyproject_toml(path_search_start: Tuple[str, ...]) -> Optional[str]

black.find_pyproject_toml(
  path_search_start,
)

black.format_cell()

General idea is:

  • if cell has trailing semicolon, remove it;
  • if cell has IPython magics, mask them;
  • format cell;
  • reinstate IPython magics;
  • reinstate trailing semicolon (if originally present);
  • strip trailing newlines.

Cells with syntax errors will not be processed, as they could potentially be automagics or multi-line magics, which are currently not supported.

black.format_cell(src: str, *, fast: bool, mode: black.mode.Mode) -> str

black.format_cell(
  src,
  fast,
  mode,
)

black.format_file_contents()

If fast is False, additionally confirm that the reformatted code is valid by calling :func:assert_equivalent and :func:assert_stable on it. mode is passed to :func:format_str.

black.format_file_contents(src_contents: str, *, fast: bool, mode: black.mode.Mode) -> str

black.format_file_contents(
  src_contents,
  fast,
  mode,
)

black.format_file_in_place()

If write_back is DIFF, write a diff to stdout. If it is YES, write reformatted code to the file. mode and fast options are passed to :func:format_file_contents.

black.format_file_in_place(src: pathlib.Path, fast: bool, mode: black.mode.Mode, write_back: black.WriteBack = <WriteBack.NO: 0>, lo
ck: Any = None) -> bool

black.format_file_in_place(
  src,
  fast,
  mode,
  write_back,
  lock,
)

black.format_ipynb_string()

Operate cell-by-cell, only on code cells, only for Python notebooks. If the .ipynb originally had a trailing newline, it’ll be preserved.

black.format_ipynb_string(src_contents: str, *, fast: bool, mode: black.mode.Mode) -> str

black.format_ipynb_string(
  src_contents,
  fast,
  mode,
)

black.format_stdin_to_stdout()

If content is None, it’s read from sys.stdin.

If write_back is YES, write reformatted code back to stdout. If it is DIFF, write a diff to stdout. The mode argument is passed to

black.format_stdin_to_stdout(fast: bool, *, content: Optional[str] = None, write_back: black.WriteBack = <WriteBack.NO: 0>, mode: bl
ack.mode.Mode) -> bool

black.format_stdin_to_stdout(
  fast,
  content,
  write_back,
  mode,
)

black.format_str()

mode determines formatting options, such as how many characters per line are allowed. Example:

>>> import black >>> print(black.format_str(“def f(arg:str=’’)->None:…”, mode=black.Mode())) def f(arg: str = “”) -> None: …

A more complex example:

>>> print( … black.format_str( … “def f(arg:str=’’)->None: hey”, … mode=black.Mode( … target_versions={black.TargetVersion.PY36}, … line_length=10, … string_normalization=False, … is_pyi=False, … ), … ), … ) def f( arg: str = ‘’, ) -> None: hey

black.format_str(src_contents: str, *, mode: black.mode.Mode) -> str

black.format_str(
  src_contents,
  mode,
)

black.gen_python_files()

exclude_regex, extend_exclude, or force_exclude regexes, but are included by the include regex.

Symbolic links pointing outside of the root directory are ignored.

report is where output about exclusions goes.

black.gen_python_files(paths: Iterable[pathlib.Path], root: pathlib.Path, include: Pattern[str], exclude: Pattern[str], extend_exclu
de: Optional[Pattern[str]], force_exclude: Optional[Pattern[str]], report: black.report.Report, gitignore: Optional[pathspec.pathspe
c.PathSpec], *, verbose: bool, quiet: bool) -> Iterator[pathlib.Path]

black.gen_python_files(
  paths,
  root,
  include,
  exclude,
  extend_exclude,
  force_exclude,
  report,
  gitignore,
  verbose,
  quiet,
)

black.get_cache_info()

black.get_cache_info(path: pathlib.Path) -> Tuple[float, int]

black.get_cache_info(
  path,
)

black.get_features_used()

Currently looking for:

  • f-strings;
  • underscores in numeric literals;
  • trailing commas after * or ** in function signatures and calls;
  • positional only arguments in function signatures and lambdas;
  • assignment expression;
  • relaxed decorator syntax;
  • print / exec statements;
black.get_features_used(node: blib2to3.pytree.Node) -> Set[black.mode.Feature]

black.get_features_used(
  node,
)

black.get_future_imports()

black.get_future_imports(node: blib2to3.pytree.Node) -> Set[str]

black.get_future_imports(
  node,
)

black.get_sources()

black.get_sources(*, ctx: click.core.Context, src: Tuple[str, ...], quiet: bool, verbose: bool, include: Pattern[str], exclude: Opti
onal[Pattern[str]], extend_exclude: Optional[Pattern[str]], force_exclude: Optional[Pattern[str]], report: 'Report', stdin_filename:
 Optional[str]) -> Set[pathlib.Path]

black.get_sources(
  ctx,
  src,
  quiet,
  verbose,
  include,
  exclude,
  extend_exclude,
  force_exclude,
  report,
  stdin_filename,
)

black.ipynb_diff()

black.ipynb_diff(a: str, b: str, a_name: str, b_name: str) -> str

black.ipynb_diff(
  a,
  b,
  a_name,
  b_name,
)

black.is_simple_decorator_expression()

This function takes the node of the ’namedexpr_test’ of the new decorator grammar and test if it would be valid under the old decorator grammar.

The old grammar was: decorator: @ dotted_name [arguments] NEWLINE The new grammar is : decorator: @ namedexpr_test NEWLINE

black.is_simple_decorator_expression(node: Union[blib2to3.pytree.Leaf, blib2to3.pytree.Node]) -> bool

black.is_simple_decorator_expression(
  node,
)

black.lib2to3_parse()

black.lib2to3_parse(src_txt: str, target_versions: Iterable[black.mode.TargetVersion] = ()) -> blib2to3.pytree.Node

black.lib2to3_parse(
  src_txt,
  target_versions,
)

black.mask_cell()

For example,

%matplotlib inline
'foo'

becomes

"25716f358c32750e"
'foo'

The replacements are returned, along with the transformed code.

black.mask_cell(src: str) -> Tuple[str, List[black.handle_ipynb_magics.Replacement]]

black.mask_cell(
  src,
)

black.maybe_install_uvloop()

This is called only from command-line entry points to avoid interfering with the parent process if Black is used as a library.

black.maybe_install_uvloop() -> None

black.maybe_install_uvloop(
)

black.mypyc_attr()

black.mypyc_attr(*attrs, **kwattrs)

black.mypyc_attr(
  attrs,
  kwattrs,
)

black.normalize_fmt_off()

black.normalize_fmt_off(node: blib2to3.pytree.Node) -> None

black.normalize_fmt_off(
  node,
)

black.normalize_path_maybe_ignore()

report is where “path ignored” output goes.

black.normalize_path_maybe_ignore(path: pathlib.Path, root: pathlib.Path, report: black.report.Report) -> Optional[str]

black.normalize_path_maybe_ignore(
  path,
  root,
  report,
)

black.nullcontext()

To be used like nullcontext in Python 3.7.

black.nullcontext() -> Iterator[NoneType]

black.nullcontext(
)

black.out()

black.out(message: Optional[str] = None, nl: bool = True, **styles: Any) -> None

black.out(
  message,
  nl,
  styles,
)

black.parse_ast()

black.parse_ast(src: str) -> ast.AST

black.parse_ast(
  src,
)

black.parse_pyproject_toml()

If parsing fails, will raise a tomli.TOMLDecodeError

black.parse_pyproject_toml(path_config: str) -> Dict[str, Any]

black.parse_pyproject_toml(
  path_config,
)

black.patch_click()

On certain misconfigured environments, Python 3 selects the ASCII encoding as the default which restricts paths that it can access during the lifetime of the application. Click refuses to work in this scenario by raising a RuntimeError.

In case of Black the likelihood that non-ASCII characters are going to be used in file paths is minimal since it’s Python source code. Moreover, this crash was spurious on Python 3.7 thanks to PEP 538 and PEP 540.

black.patch_click() -> None

black.patch_click(
)

black.patched_main()

black.patched_main() -> None

black.patched_main(
)

black.path_empty()

black.path_empty(src: Sized, msg: str, quiet: bool, verbose: bool, ctx: click.core.Context) -> None

black.path_empty(
  src,
  msg,
  quiet,
  verbose,
  ctx,
)

black.put_trailing_semicolon_back()

Mirrors the logic in quiet from IPython.core.displayhook, but uses tokenize_rt so that round-tripping works fine.

black.put_trailing_semicolon_back(src: str, has_trailing_semicolon: bool) -> str

black.put_trailing_semicolon_back(
  src,
  has_trailing_semicolon,
)

black.re_compile_maybe_verbose()

If it contains newlines, use verbose mode.

black.re_compile_maybe_verbose(regex: str) -> Pattern[str]

black.re_compile_maybe_verbose(
  regex,
)

black.read_cache()

If it is not well formed, the call to write_cache later should resolve the issue.

black.read_cache(mode: black.mode.Mode) -> Dict[str, Tuple[float, int]]

black.read_cache(
  mode,
)

black.read_pyproject_toml()

Returns the path to a successfully found and read configuration file, None otherwise.

black.read_pyproject_toml(ctx: click.core.Context, param: click.core.Parameter, value: Optional[str]) -> Optional[str]

black.read_pyproject_toml(
  ctx,
  param,
  value,
)

black.reformat_code()

Similar to reformat_one, but for string content.

fast, write_back, and mode options are passed to

black.reformat_code(content: str, fast: bool, write_back: black.WriteBack, mode: black.mode.Mode, report: black.report.Report) -> No
ne

black.reformat_code(
  content,
  fast,
  write_back,
  mode,
  report,
)

black.reformat_many()

black.reformat_many(sources: Set[pathlib.Path], fast: bool, write_back: black.WriteBack, mode: black.mode.Mode, report: 'Report', wo
rkers: Optional[int]) -> None

black.reformat_many(
  sources,
  fast,
  write_back,
  mode,
  report,
  workers,
)

black.reformat_one()

fast, write_back, and mode options are passed to

black.reformat_one(src: pathlib.Path, fast: bool, write_back: black.WriteBack, mode: black.mode.Mode, report: 'Report') -> None

black.reformat_one(
  src,
  fast,
  write_back,
  mode,
  report,
)

black.remove_trailing_semicolon()

For example,

fig, ax = plt.subplots()
ax.plot(x_data, y_data);  # plot data

would become

fig, ax = plt.subplots()
ax.plot(x_data, y_data)  # plot data

Mirrors the logic in quiet from IPython.core.displayhook, but uses tokenize_rt so that round-tripping works fine.

black.remove_trailing_semicolon(src: str) -> Tuple[str, bool]

black.remove_trailing_semicolon(
  src,
)

black.replace()

This is especially useful for frozen classes. Example usage:

@dataclass(frozen=True) class C: x: int y: int

c = C(1, 2) c1 = replace(c, x=3) assert c1.x == 3 and c1.y == 2

black.replace(obj, /, **changes)

black.replace(
  obj,
  changes,
)

black.schedule_formatting()

(Use ProcessPoolExecutors for actual parallelism.)

write_back, fast, and mode options are passed to

black.schedule_formatting(sources: Set[pathlib.Path], fast: bool, write_back: black.WriteBack, mode: black.mode.Mode, report: 'Repor
t', loop: asyncio.events.AbstractEventLoop, executor: concurrent.futures._base.Executor) -> None

black.schedule_formatting(
  sources,
  fast,
  write_back,
  mode,
  report,
  loop,
  executor,
)

black.shutdown()

black.shutdown(loop: asyncio.events.AbstractEventLoop) -> None

black.shutdown(
  loop,
)

black.stringify_ast()

black.stringify_ast(node: ast.AST, depth: int = 0) -> Iterator[str]

black.stringify_ast(
  node,
  depth,
)

black.supports_feature()

black.supports_feature(target_versions: Set[black.mode.TargetVersion], feature: black.mode.Feature) -> bool

black.supports_feature(
  target_versions,
  feature,
)

black.target_version_option_callback()

This is its own function because mypy couldn’t infer the type correctly when it was a lambda, causing mypyc trouble.

black.target_version_option_callback(c: click.core.Context, p: Union[click.core.Option, click.core.Parameter], v: Tuple[str, ...]) -
> List[black.mode.TargetVersion]

black.target_version_option_callback(
  c,
  p,
  v,
)

black.transform_line()

They should fit in the allotted line_length but might not be able to.

features are syntactical features that may be used in the output.

black.transform_line(line: black.lines.Line, mode: black.mode.Mode, features: Collection[black.mode.Feature] = ()) -> Iterator[black
.lines.Line]

black.transform_line(
  line,
  mode,
  features,
)

black.unmask_cell()

For example

"9b20"
foo = bar

becomes

%%time
foo = bar

black.unmask_cell(src: str, replacements: List[black.handle_ipynb_magics.Replacement]) -> str

black.unmask_cell(
  src,
  replacements,
)

black.validate_cell()

or non-Python cell magics, which might cause tokenizer_rt to break because of indentations.

If a cell contains ``!ls``, then it'll be transformed to
``get_ipython().system('ls')``. However, if the cell originally contained
``get_ipython().system('ls')``, then it would get transformed in the same way:

    \>\>\> TransformerManager().transform_cell("get_ipython().system('ls')")
    "get_ipython().system('ls')

" >>> TransformerManager().transform_cell("!ls") “get_ipython().system(’ls’) "

Due to the impossibility of safely roundtripping in such situations, cells
containing transformed magics will be ignored.

black.validate_cell(src: str) -> None

black.validate_cell(
  src,
)

black.validate_metadata()

All notebook metadata fields are optional, see https://nbformat.readthedocs.io/en/latest/format_description.html. So if a notebook has empty metadata, we will try to parse it anyway.

black.validate_metadata(nb: MutableMapping[str, Any]) -> None

black.validate_metadata(
  nb,
)

black.validate_regex()

black.validate_regex(ctx: click.core.Context, param: click.core.Parameter, value: Optional[str]) -> Optional[Pattern[str]]

black.validate_regex(
  ctx,
  param,
  value,
)

black.wrap_stream_for_windows()

If colorama is unavailable, the original stream is returned unmodified. Otherwise, the wrap_stream() function determines whether the stream needs to be wrapped for a Windows environment and will accordingly either return an AnsiToWin32 wrapper or the original stream.

black.wrap_stream_for_windows(f: _io.TextIOWrapper) -> Union[_io.TextIOWrapper, ForwardRef('colorama.AnsiToWin32')]

black.wrap_stream_for_windows(
  f,
)

black.write_cache()

black.write_cache(cache: Dict[str, Tuple[float, int]], sources: Iterable[pathlib.Path], mode: black.mode.Mode) -> None

black.write_cache(
  cache,
  sources,
  mode,
)
Updated on 01-15-2022
Back | Home

Tags

Black