{"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef get_call_signature(fn: FunctionType,\n args: ArgsType,\n kwargs: KwargsType,\n debug_cache: bool = False) -> str:\n \"\"\"\n Takes a function and its args/kwargs, and produces a string description\n of the function call (the call signature) suitable for use indirectly as a\n cache key. The string is a JSON representation. See ``make_cache_key`` for\n a more suitable actual cache key.\n \"\"\"\n # Note that the function won't have the __self__ argument (as in\n # fn.__self__), at this point, even if it's a member function.\n try:\n call_sig = json_encode((fn.__qualname__, args, kwargs))\n except TypeError:\n log.critical(\n \"\\nTo decorate using @django_cache_function without specifying \"\n \"cache_key, the decorated function's owning class and its \"\n \"parameters must be JSON-serializable (see jsonfunc.py, \"\n \"django_cache_fn.py).\\n\")\n raise\n if debug_cache:\n log.debug(\"Making call signature {!r}\", call_sig)\n return call_sig", "response": "Returns a string that can be used to generate a cache key for the function call."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef make_cache_key(call_signature: str,\n debug_cache: bool = False) -> str:\n \"\"\"\n Takes a function and its args/kwargs, and produces a string description\n of the function call (the call signature) suitable for use as a cache key.\n The string is an MD5 hash of the JSON-encoded call signature.\n The logic behind these decisions is as follows:\n\n - We have a bunch of components of arbitrary type, and we need to get\n a unique string out.\n - We shouldn't use ``str()``, because that is often poorly specified; e.g.\n is ``'a.b.c'`` a ``TableId``, or is it a ``ColumnId`` with no ``'db'``\n field?\n - We could use ``repr()``: sometimes that gives us helpful things that\n could in principle be passed to ``eval()``, in which case ``repr()`` would\n be fine, but sometimes it doesn't, and gives unhelpful things like\n ``'<__main__.Thing object at 0x7ff3093ebda0>'``.\n - However, if something encodes to JSON, that representation should\n be reversible and thus contain the right sort of information.\n - Note also that bound methods will come with a ``self`` argument, for\n which the address may be very relevant...\n - Let's go with ``repr()``. Users of the cache decorator should not pass\n objects whose ``repr()`` includes the memory address of the object unless\n they want those objects to be treated as distinct.\n - Ah, no. The cache itself will pickle and unpickle things, and this\n will change memory addresses of objects. So we can't store a reference\n to an object using ``repr()`` and using ``cache.add()``/``pickle()`` and\n hope they'll come out the same.\n - Use the JSON after all.\n - And do it in ``get_call_signature()``, not here.\n - That means that any class we wish to decorate WITHOUT specifying a\n cache key manually must support JSON.\n \"\"\"\n key = hashlib.md5(call_signature.encode(\"utf-8\")).hexdigest()\n if debug_cache:\n log.debug(\"Making cache key {} from call_signature {!r}\",\n key, call_signature)\n return key", "response": "Generates a unique cache key for the given function call signature."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef django_cache_function(timeout: int = 5 * 60,\n cache_key: str = '',\n debug_cache: bool = False):\n \"\"\"\n Decorator to add caching to a function in Django.\n Uses the Django default cache.\n\n Args:\n\n timeout: timeout in seconds; use None for \"never expire\", as 0 means\n \"do not cache\".\n\n cache_key: optional cache key to use (if falsy, we'll invent one)\n debug_cache: show hits/misses?\n \"\"\"\n cache_key = cache_key or None\n\n def decorator(fn):\n def wrapper(*args, **kwargs):\n # - NOTE that Django returns None from cache.get() for \"not in\n # cache\", so can't cache a None value;\n # https://docs.djangoproject.com/en/1.10/topics/cache/#basic-usage # noqa\n # - We need to store a bit more than just the function result\n # anyway, to detect hash collisions when the user doesn't specify\n # the cache_key, so we may as well use that format even if the\n # user does specify the cache_key, and then we can store a None\n # result properly as well.\n if cache_key:\n # User specified a cache key. This is easy.\n call_sig = ''\n _cache_key = cache_key\n check_stored_call_sig = False\n else:\n # User didn't specify a cache key, so we'll do one\n # automatically. Since we do this via a hash, there is a small\n # but non-zero chance of a hash collision.\n call_sig = get_call_signature(fn, args, kwargs)\n _cache_key = make_cache_key(call_sig)\n check_stored_call_sig = True\n if debug_cache:\n log.critical(\"Checking cache for key: \" + _cache_key)\n cache_result_tuple = cache.get(_cache_key) # TALKS TO CACHE HERE\n if cache_result_tuple is None:\n if debug_cache:\n log.debug(\"Cache miss\")\n else:\n if debug_cache:\n log.debug(\"Cache hit\")\n cached_call_sig, func_result = cache_result_tuple\n if (not check_stored_call_sig) or cached_call_sig == call_sig:\n return func_result\n log.warning(\n \"... Cache hit was due to hash collision; cached_call_sig \"\n \"{} != call_sig {}\".format(\n repr(cached_call_sig), repr(call_sig)))\n # If we get here, either it wasn't in the cache, or something\n # was in the cache that matched by cache_key but was actually a\n # hash collision. Either way, we must do the real work.\n func_result = fn(*args, **kwargs)\n cache_result_tuple = (call_sig, func_result)\n cache.set(key=_cache_key, value=cache_result_tuple,\n timeout=timeout) # TALKS TO CACHE HERE\n return func_result\n\n return wrapper\n\n return decorator", "response": "A decorator that returns a function that caches the result of the function in Django s cache."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\ncreates an affine transformation matrix that maps vector 1 onto vector 2.", "response": "def transform(v1, v2):\n \"\"\"\n Create an affine transformation matrix that maps vector 1\n onto vector 2\n\n https://math.stackexchange.com/questions/293116/rotating-one-3d-vector-to-another\n \"\"\"\n theta = angle(v1,v2)\n x = N.cross(v1,v2)\n x = x / N.linalg.norm(x)\n\n A = N.array([\n [0, -x[2], x[1]],\n [x[2], 0, -x[0]],\n [-x[1], x[0], 0]])\n R = N.exp(A*theta)\n\n return R"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef cartesian(lon, lat):\n return N.array([\n N.cos(lat)*N.cos(lon),\n N.cos(lat)*N.sin(lon),\n N.sin(lat)\n ])", "response": "Converts spherical positions in ( lon lat ) to cartesian coordiantes [ x y z )"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef x_11paths_authorization(app_id, secret, context, utc=None):\n utc = utc or context.headers[X_11PATHS_DATE_HEADER_NAME]\n\n url_path = ensure_url_path_starts_with_slash(context.url_path)\n url_path_query = url_path\n if context.query_params:\n url_path_query += \"?%s\" % (url_encode(context.query_params, sort=True))\n\n string_to_sign = (context.method.upper().strip() + \"\\n\" +\n utc + \"\\n\" +\n _get_11paths_serialized_headers(context.headers) + \"\\n\" +\n url_path_query.strip())\n\n if context.body_params and isinstance(context.renderer, FormRenderer):\n string_to_sign = string_to_sign + \"\\n\" + url_encode(context.body_params, sort=True).replace(\"&\", \"\")\n\n authorization_header_value = (AUTHORIZATION_METHOD + AUTHORIZATION_HEADER_FIELD_SEPARATOR + app_id +\n AUTHORIZATION_HEADER_FIELD_SEPARATOR + _sign_data(secret, string_to_sign))\n\n return authorization_header_value", "response": "Calculate the authentication headers needed to send a request to the Latch API."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nsigns data using the secret as cipher key.", "response": "def _sign_data(secret, data):\n \"\"\"\n Sign data.\n\n :param data: the string to sign\n :return: string base64 encoding of the HMAC-SHA1 hash of the data parameter using {@code secretKey} as cipher key.\n \"\"\"\n sha1_hash = hmac.new(secret.encode(), data.encode(), sha1)\n return binascii.b2a_base64(sha1_hash.digest())[:-1].decode('utf8')"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _get_11paths_serialized_headers(x_headers):\n if x_headers:\n headers = to_key_val_list(x_headers, sort=True, insensitive=True)\n serialized_headers = \"\"\n for key, value in headers:\n if key.lower().startswith(X_11PATHS_HEADER_PREFIX.lower()) and \\\n key.lower() != X_11PATHS_DATE_HEADER_NAME.lower():\n serialized_headers += key.lower() + X_11PATHS_HEADER_SEPARATOR + value.replace(\"\\n\", \" \") + \" \"\n return serialized_headers.strip()\n else:\n return \"\"", "response": "Returns a string ready to be signed from the 11 - paths specific HTTP headers received."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nreturns a 2x2 transformation matrix to rotate the sequence element by an angle in two dimensions.", "response": "def rotate_2D(angle):\n \"\"\"\n Returns a 2x2 transformation matrix to rotate\n by an angle in two dimensions\n \"\"\"\n return N.array([[N.cos(angle),-N.sin(angle)],\n [N.sin(angle),N.cos(angle)]])"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef apparent_dip_correction(axes):\n a1 = axes[0].copy()\n a1[-1] = 0\n cosa = angle(axes[0],a1,cos=True)\n _ = 1-cosa**2\n if _ > 1e-12:\n sina = N.sqrt(_)\n if cosa < 0:\n sina *= -1\n # Construct rotation matrix\n R= N.array([[cosa,sina],[-sina,cosa]])\n else:\n # Small angle, don't bother\n # (small angles can lead to spurious results)\n R = N.identity(2)\n #if axes[0,0] < 0:\n # return R.T\n #else:\n return R", "response": "Generates a two - dimensional rotation matrix that rotates a projected dataset to correct for apparent dip\n "} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef hyperbolic_errors(hyp_axes, xvals,\n transformation=None, axes=None,\n means=None, correct_apparent_dip=True,\n reverse=False):\n \"\"\"\n Returns a function that can be used to create a view of the\n hyperbolic error ellipse from a specific direction.\n\n This creates a hyperbolic quadric and slices it to form a conic\n on a 2d cartesian plane aligned with the requested direction.\n\n A function is returned that takes x values (distance along nominal\n line) and returns y values (width of error hyperbola)\n\n kwargs:\n transformation rotation to apply to quadric prior to slicing\n (e.g. transformation into 'world' coordinates\n axes axes on which to slice the data\n \"\"\"\n if means is None:\n means = N.array([0,0])\n\n arr = augment_tensor(N.diag(hyp_axes))\n\n # Transform ellipsoid to dual hyperboloid\n hyp = conic(arr).dual()\n\n if len(hyp_axes) == 3:\n # Three_dimensional case\n\n if transformation is None:\n transformation = N.identity(3)\n if axes is None:\n axes = N.array([[0,1,0],[0,0,1]])\n\n hyp = hyp.transform(augment_tensor(transformation))\n\n n_ = N.cross(axes[0],axes[1])\n\n # Create a plane containing the two axes specified\n # in the function call\n p = plane(n_) # no offset (goes through origin)\n h1 = hyp.slice(p, axes=axes)[0]\n else:\n # We have a 2d geometry\n h1 = hyp\n\n # Major axes of the conic sliced in the requested viewing\n # geometry\n A = N.sqrt(h1.semiaxes())\n\n yvals = A[1]*N.cosh(N.arcsinh(xvals/A[0]))\n\n vals = N.array([xvals,yvals]).transpose()\n nom = N.array([xvals,N.zeros(xvals.shape)]).transpose()\n\n # Rotate the whole result if the PCA axes aren't aligned to the\n # major axes of the view coordinate system\n ax1 = apparent_dip_correction(axes)\n\n # This is a dirty hack to flip things left to right\n if reverse:\n ax1 = ax1.T\n # Top\n t = dot(vals,ax1).T+means[:,N.newaxis]\n # Btm\n vals[:,-1] *= -1\n b = dot(vals,ax1).T+means[:,N.newaxis]\n\n nom = dot(nom,ax1).T+means[:,N.newaxis]\n return nom, b, t[:,::-1]", "response": "Create a function that can be used to create a view of a hyperbolic error ellipse from a specific direction."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\ncomputes hyperbola values for the given set of values.", "response": "def hyperbola_values(hyp_axes, xvals):\n \"\"\"\n kwargs:\n transformation rotation to apply to quadric prior to slicing\n (e.g. transformation into 'world' coordinates\n axes axes on which to slice the data\n \"\"\"\n A = N.sqrt(hyp_axes)\n return A[1]*N.cosh(N.arcsinh(xvals/A[0]))"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nformats a number of bytes in a human - readable binary format.", "response": "def sizeof_fmt(num: float, suffix: str = 'B') -> str:\n \"\"\"\n Formats a number of bytes in a human-readable binary format (e.g. ``2048``\n becomes ``'2 KiB'``); from http://stackoverflow.com/questions/1094841.\n \"\"\"\n for unit in ('', 'Ki', 'Mi', 'Gi', 'Ti', 'Pi', 'Ei', 'Zi'):\n if abs(num) < 1024.0:\n return \"%3.1f%s%s\" % (num, unit, suffix)\n num /= 1024.0\n return \"%.1f%s%s\" % (num, 'Yi', suffix)"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nreturns a human - readable version of the number of bytes in a format specification string.", "response": "def bytes2human(n: Union[int, float],\n format: str = '%(value).1f %(symbol)s',\n symbols: str = 'customary') -> str:\n \"\"\"\n Converts a number of bytes into a human-readable format.\n From http://code.activestate.com/recipes/578019-bytes-to-human-human-to-bytes-converter/.\n \n Args:\n n: number of bytes\n format: a format specification string\n symbols: can be one of ``\"customary\"``, ``\"customary_ext\"``, ``\"iec\"``\n or ``\"iec_ext\"``; see http://goo.gl/kTQMs\n\n Returns:\n the formatted number\n \n Examples:\n\n >>> bytes2human(0)\n '0.0 B'\n >>> bytes2human(0.9)\n '0.0 B'\n >>> bytes2human(1)\n '1.0 B'\n >>> bytes2human(1.9)\n '1.0 B'\n >>> bytes2human(1024)\n '1.0 K'\n >>> bytes2human(1048576)\n '1.0 M'\n >>> bytes2human(1099511627776127398123789121)\n '909.5 Y'\n\n >>> bytes2human(9856, symbols=\"customary\")\n '9.6 K'\n >>> bytes2human(9856, symbols=\"customary_ext\")\n '9.6 kilo'\n >>> bytes2human(9856, symbols=\"iec\")\n '9.6 Ki'\n >>> bytes2human(9856, symbols=\"iec_ext\")\n '9.6 kibi'\n\n >>> bytes2human(10000, \"%(value).1f %(symbol)s/sec\")\n '9.8 K/sec'\n\n >>> # precision can be adjusted by playing with %f operator\n >>> bytes2human(10000, format=\"%(value).5f %(symbol)s\")\n '9.76562 K'\n\n \"\"\" # noqa\n n = int(n)\n if n < 0:\n raise ValueError(\"n < 0\")\n symbols = SYMBOLS[symbols]\n prefix = {}\n for i, s in enumerate(symbols[1:]):\n prefix[s] = 1 << (i + 1) * 10\n for symbol in reversed(symbols[1:]):\n if n >= prefix[symbol]:\n value = float(n) / prefix[symbol]\n return format % locals()\n return format % dict(symbol=symbols[0], value=n)"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nconvert a human - readable string to bytes.", "response": "def human2bytes(s: str) -> int:\n \"\"\"\n Modified from\n http://code.activestate.com/recipes/578019-bytes-to-human-human-to-bytes-converter/.\n \n Attempts to guess the string format based on default symbols\n set and return the corresponding bytes as an integer.\n When unable to recognize the format, :exc:`ValueError` is raised.\n\n >>> human2bytes('0 B')\n 0\n >>> human2bytes('1 K')\n 1024\n >>> human2bytes('1 M')\n 1048576\n >>> human2bytes('1 Gi')\n 1073741824\n >>> human2bytes('1 tera')\n 1099511627776\n\n >>> human2bytes('0.5kilo')\n 512\n >>> human2bytes('0.1 byte')\n 0\n >>> human2bytes('1 k') # k is an alias for K\n 1024\n >>> human2bytes('12 foo')\n Traceback (most recent call last):\n ...\n ValueError: can't interpret '12 foo'\n \"\"\" # noqa\n if not s:\n raise ValueError(\"Can't interpret {!r} as integer\".format(s))\n try:\n return int(s)\n except ValueError:\n pass\n init = s\n num = \"\"\n while s and s[0:1].isdigit() or s[0:1] == '.':\n num += s[0]\n s = s[1:]\n num = float(num)\n letter = s.strip()\n for name, sset in SYMBOLS.items():\n if letter in sset:\n break\n else:\n if letter == 'k':\n # treat 'k' as an alias for 'K' as per https://en.wikipedia.org/wiki/Binary_prefix # noqa\n sset = SYMBOLS['customary']\n letter = letter.upper()\n else:\n raise ValueError(\"can't interpret %r\" % init)\n prefix = {sset[0]: 1}\n for i, s in enumerate(sset[1:]):\n prefix[s] = 1 << (i + 1) * 10\n return int(num * prefix[letter])"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nreturns a list of all tables that depend on each other.", "response": "def get_all_dependencies(metadata: MetaData,\n extra_dependencies: List[TableDependency] = None,\n sort: bool = True) \\\n -> List[TableDependency]:\n \"\"\"\n Describes how the tables found in the metadata depend on each other.\n (If table B contains a foreign key to table A, for example, then B depends\n on A.)\n\n Args:\n metadata: the metadata to inspect\n extra_dependencies: additional table dependencies to specify manually\n sort: sort into alphabetical order of (parent, child) table names?\n\n Returns:\n a list of :class:`TableDependency` objects\n\n See :func:`sort_tables_and_constraints` for method.\n \"\"\"\n extra_dependencies = extra_dependencies or [] # type: List[TableDependency] # noqa\n for td in extra_dependencies:\n td.set_metadata_if_none(metadata)\n dependencies = set([td.sqla_tuple() for td in extra_dependencies])\n\n tables = list(metadata.tables.values()) # type: List[Table]\n\n for table in tables:\n for fkc in table.foreign_key_constraints:\n if fkc.use_alter is True:\n # http://docs.sqlalchemy.org/en/latest/core/constraints.html#sqlalchemy.schema.ForeignKeyConstraint.params.use_alter # noqa\n continue\n\n dependent_on = fkc.referred_table\n if dependent_on is not table:\n dependencies.add((dependent_on, table))\n\n if hasattr(table, \"_extra_dependencies\"):\n # noinspection PyProtectedMember\n dependencies.update(\n (parent, table) for parent in table._extra_dependencies\n )\n\n dependencies = [\n TableDependency(parent_table=parent, child_table=child)\n for parent, child in dependencies\n ]\n if sort:\n dependencies.sort(key=lambda td_: (td_.parent_tablename,\n td_.child_tablename))\n return dependencies"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nclassify a metadata object by their dependencies.", "response": "def classify_tables_by_dependency_type(\n metadata: MetaData,\n extra_dependencies: List[TableDependency] = None,\n sort: bool = True) \\\n -> List[TableDependencyClassification]:\n \"\"\"\n Inspects a metadata object (optionally adding other specified dependencies)\n and returns a list of objects describing their dependencies.\n\n Args:\n metadata: the :class:`MetaData` to inspect\n extra_dependencies: additional dependencies\n sort: sort the results by table name?\n\n Returns:\n list of :class:`TableDependencyClassification` objects, one for each\n table\n\n \"\"\"\n tables = list(metadata.tables.values()) # type: List[Table]\n all_deps = get_all_dependencies(metadata, extra_dependencies)\n tdcmap = {} # type: Dict[Table, TableDependencyClassification]\n for table in tables:\n parents = [td.parent_table for td in all_deps\n if td.child_table == table]\n children = [td.child_table for td in all_deps\n if td.parent_table == table]\n tdcmap[table] = TableDependencyClassification(\n table, parents=parents, children=children\n )\n\n # Check for circularity\n def parents_contain(start: Table,\n probe: Table) -> Tuple[bool, List[Table]]:\n tdc_ = tdcmap[start]\n if probe in tdc_.parents:\n return True, [start, probe]\n for parent in tdc_.parents:\n contains_, chain_ = parents_contain(start=parent, probe=probe)\n if contains_:\n return True, [start] + chain_\n return False, []\n\n def children_contain(start: Table,\n probe: Table) -> Tuple[bool, List[Table]]:\n tdc_ = tdcmap[start]\n if probe in tdc_.children:\n return True, [start, probe]\n for child in tdc_.children:\n contains_, chain_ = children_contain(start=child, probe=probe)\n if contains_:\n return True, [start] + chain_\n return False, []\n\n for table in tables:\n tdc = tdcmap[table]\n contains, chain = parents_contain(start=table, probe=table)\n if contains:\n tdc.set_circular(contains, chain)\n else:\n contains, chain = children_contain(start=table, probe=table)\n if contains:\n tdc.set_circular(contains, chain)\n else:\n tdc.set_circular(False)\n\n classifications = list(tdcmap.values())\n if sort:\n classifications.sort(key=lambda c: c.tablename)\n return classifications"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef merge_db(base_class: Type,\n src_engine: Engine,\n dst_session: Session,\n allow_missing_src_tables: bool = True,\n allow_missing_src_columns: bool = True,\n translate_fn: Callable[[TranslationContext], None] = None,\n skip_tables: List[TableIdentity] = None,\n only_tables: List[TableIdentity] = None,\n tables_to_keep_pks_for: List[TableIdentity] = None,\n extra_table_dependencies: List[TableDependency] = None,\n dummy_run: bool = False,\n info_only: bool = False,\n report_every: int = 1000,\n flush_per_table: bool = True,\n flush_per_record: bool = False,\n commit_with_flush: bool = False,\n commit_at_end: bool = True,\n prevent_eager_load: bool = True,\n trcon_info: Dict[str, Any] = None) -> None:\n \"\"\"\n Copies an entire database as far as it is described by ``metadata`` and\n ``base_class``, from SQLAlchemy ORM session ``src_session`` to\n ``dst_session``, and in the process:\n\n - creates new primary keys at the destination, or raises an error if it\n doesn't know how (typically something like: ``Field 'name' doesn't have a\n default value``)\n\n - maintains relationships, or raises an error if it doesn't know how\n\n Basic method:\n\n - Examines the metadata for the SQLAlchemy ORM base class you provide.\n\n - Assumes that the tables exist (in the destination).\n\n - For each table/ORM class found in the metadata:\n\n - Queries (via the ORM) from the source.\n\n - For each ORM instance retrieved:\n\n - Writes information to the destination SQLAlchemy session.\n\n - If that ORM object has relationships, process them too.\n\n If a table is missing in the source, then that's OK if and only if\n ``allow_missing_src_tables`` is set. (Similarly with columns and\n ``allow_missing_src_columns``; we ask the ORM to perform a partial load,\n of a subset of attributes only.)\n\n Args:\n base_class:\n your ORM base class, e.g. from ``Base = declarative_base()``\n\n src_engine:\n SQLALchemy :class:`Engine` for the source database\n\n dst_session:\n SQLAlchemy :class:`Session` for the destination database\n\n allow_missing_src_tables:\n proceed if tables are missing from the source (allowing you to\n import from older, incomplete databases)\n\n allow_missing_src_columns:\n proceed if columns are missing from the source (allowing you to\n import from older, incomplete databases)\n\n translate_fn:\n optional function called with each instance, so you can modify\n instances in the pipeline. Signature:\n\n .. code-block:: python\n\n def my_translate_fn(trcon: TranslationContext) -> None:\n # We can modify trcon.newobj, or replace it (including\n # setting trcon.newobj = None to omit this object).\n pass\n\n skip_tables:\n tables to skip (specified as a list of :class:`TableIdentity`)\n\n only_tables:\n tables to restrict the processor to (specified as a list of\n :class:`TableIdentity`)\n\n tables_to_keep_pks_for:\n tables for which PKs are guaranteed to be safe to insert into the\n destination database, without modification (specified as a list of\n :class:`TableIdentity`)\n\n extra_table_dependencies:\n optional list of :class:`TableDependency` objects (q.v.)\n\n dummy_run:\n don't alter the destination database\n\n info_only:\n show info, then stop\n\n report_every:\n provide a progress report every *n* records\n\n flush_per_table:\n flush the session after every table (reasonable)\n\n flush_per_record:\n flush the session after every instance (AVOID this if tables may\n refer to themselves)\n\n commit_with_flush:\n ``COMMIT`` with each flush?\n\n commit_at_end:\n ``COMMIT`` when finished?\n\n prevent_eager_load:\n disable any eager loading (use lazy loading instead)\n\n trcon_info:\n additional dictionary passed to ``TranslationContext.info``\n (see :class:`.TranslationContext`)\n \"\"\"\n\n log.info(\"merge_db(): starting\")\n if dummy_run:\n log.warning(\"Dummy run only; destination will not be changed\")\n\n # Check parameters before we modify them\n if only_tables is not None and not only_tables:\n log.warning(\"... only_tables == []; nothing to do\")\n return\n\n # Finalize parameters\n skip_tables = skip_tables or [] # type: List[TableIdentity]\n only_tables = only_tables or [] # type: List[TableIdentity]\n tables_to_keep_pks_for = tables_to_keep_pks_for or [] # type: List[TableIdentity] # noqa\n extra_table_dependencies = extra_table_dependencies or [] # type: List[TableDependency] # noqa\n trcon_info = trcon_info or {} # type: Dict[str, Any]\n\n # We need both Core and ORM for the source.\n # noinspection PyUnresolvedReferences\n metadata = base_class.metadata # type: MetaData\n src_session = sessionmaker(bind=src_engine)() # type: Session\n dst_engine = get_engine_from_session(dst_session)\n tablename_to_ormclass = get_orm_classes_by_table_name_from_base(base_class)\n\n # Tell all TableIdentity objects about their metadata\n for tilist in [skip_tables, only_tables, tables_to_keep_pks_for]:\n for ti in tilist:\n ti.set_metadata_if_none(metadata)\n for td in extra_table_dependencies:\n td.set_metadata_if_none(metadata)\n\n # Get all lists of tables as their names\n skip_table_names = [ti.tablename for ti in skip_tables]\n only_table_names = [ti.tablename for ti in only_tables]\n tables_to_keep_pks_for = [ti.tablename for ti in tables_to_keep_pks_for] # type: List[str] # noqa\n # ... now all are of type List[str]\n\n # Safety check: this is an imperfect check for source == destination, but\n # it is fairly easy to pass in the wrong URL, so let's try our best:\n _src_url = get_safe_url_from_engine(src_engine)\n _dst_url = get_safe_url_from_session(dst_session)\n assert _src_url != _dst_url or _src_url == SQLITE_MEMORY_URL, (\n \"Source and destination databases are the same!\"\n )\n\n # Check the right tables are present.\n src_tables = sorted(get_table_names(src_engine))\n dst_tables = sorted(list(tablename_to_ormclass.keys()))\n log.debug(\"Source tables: {!r}\", src_tables)\n log.debug(\"Destination tables: {!r}\", dst_tables)\n if not allow_missing_src_tables:\n missing_tables = sorted(\n d for d in dst_tables\n if d not in src_tables and d not in skip_table_names\n )\n if missing_tables:\n raise RuntimeError(\"The following tables are missing from the \"\n \"source database: \" + repr(missing_tables))\n\n table_num = 0\n overall_record_num = 0\n\n tables = list(metadata.tables.values()) # type: List[Table]\n # Very helpfully, MetaData.sorted_tables produces tables in order of\n # relationship dependency (\"each table is preceded by all tables which\n # it references\");\n # http://docs.sqlalchemy.org/en/latest/core/metadata.html\n # HOWEVER, it only works if you specify ForeignKey relationships\n # explicitly.\n # We can also add in user-specified dependencies, and therefore can do the\n # sorting in one step with sqlalchemy.schema.sort_tables:\n ordered_tables = sort_tables(\n tables,\n extra_dependencies=[td.sqla_tuple() for td in extra_table_dependencies]\n )\n # Note that the ordering is NOT NECESSARILY CONSISTENT, though (in that\n # the order of stuff it doesn't care about varies across runs).\n all_dependencies = get_all_dependencies(metadata, extra_table_dependencies)\n dep_classifications = classify_tables_by_dependency_type(\n metadata, extra_table_dependencies)\n circular = [tdc for tdc in dep_classifications if tdc.circular]\n assert not circular, \"Circular dependencies! {!r}\".format(circular)\n log.debug(\"All table dependencies: {}\",\n \"; \".join(str(td) for td in all_dependencies))\n log.debug(\"Table dependency classifications: {}\",\n \"; \".join(str(c) for c in dep_classifications))\n log.info(\"Processing tables in the order: {!r}\",\n [table.name for table in ordered_tables])\n\n objmap = {}\n\n def flush() -> None:\n if not dummy_run:\n log.debug(\"Flushing session\")\n dst_session.flush()\n if commit_with_flush:\n log.debug(\"Committing...\")\n dst_session.commit()\n\n def translate(oldobj_: object, newobj_: object) -> object:\n if translate_fn is None:\n return newobj_\n tc = TranslationContext(oldobj=oldobj_,\n newobj=newobj_,\n objmap=objmap,\n table=table,\n tablename=tablename,\n src_session=src_session,\n dst_session=dst_session,\n src_engine=src_engine,\n dst_engine=dst_engine,\n missing_src_columns=missing_columns,\n src_table_names=src_tables,\n info=trcon_info)\n translate_fn(tc)\n if tc.newobj is None:\n log.debug(\"Instance skipped by user-supplied translate_fn\")\n return tc.newobj\n\n # -------------------------------------------------------------------------\n # Now, per table/ORM class...\n # -------------------------------------------------------------------------\n for table in ordered_tables:\n tablename = table.name\n\n if tablename in skip_table_names:\n log.info(\"... skipping table {!r} (as per skip_tables)\", tablename)\n continue\n if only_table_names and tablename not in only_table_names:\n log.info(\"... ignoring table {!r} (as per only_tables)\", tablename)\n continue\n if allow_missing_src_tables and tablename not in src_tables:\n log.info(\"... ignoring table {!r} (not in source database)\",\n tablename)\n continue\n table_num += 1\n table_record_num = 0\n\n src_columns = sorted(get_column_names(src_engine, tablename))\n dst_columns = sorted([column.name for column in table.columns])\n missing_columns = sorted(list(set(dst_columns) - set(src_columns)))\n\n if not allow_missing_src_columns:\n if missing_columns:\n raise RuntimeError(\n \"The following columns are missing from source table \"\n \"{!r}: {!r}\".format(tablename, missing_columns))\n\n orm_class = tablename_to_ormclass[tablename]\n pk_attrs = get_pk_attrnames(orm_class)\n c2a = colname_to_attrname_dict(orm_class)\n missing_attrs = map_keys_to_values(missing_columns, c2a)\n tdc = [tdc for tdc in dep_classifications if tdc.table == table][0]\n\n log.info(\"Processing table {!r} via ORM class {!r}\",\n tablename, orm_class)\n log.debug(\"PK attributes: {!r}\", pk_attrs)\n log.debug(\"Table: {!r}\", table)\n log.debug(\"Dependencies: parents = {!r}; children = {!r}\",\n tdc.parent_names, tdc.child_names)\n\n if info_only:\n log.debug(\"info_only; skipping table contents\")\n continue\n\n def wipe_primary_key(inst: object) -> None:\n for attrname in pk_attrs:\n setattr(inst, attrname, None)\n\n query = src_session.query(orm_class)\n\n if allow_missing_src_columns and missing_columns:\n src_attrs = map_keys_to_values(src_columns, c2a)\n log.info(\"Table {} is missing columns {} in the source\",\n tablename, missing_columns)\n log.debug(\"... using only columns {} via attributes {}\",\n src_columns, src_attrs)\n query = query.options(load_only(*src_attrs))\n # PROBLEM: it will not ignore the PK.\n\n if prevent_eager_load:\n query = query.options(lazyload(\"*\"))\n\n wipe_pk = tablename not in tables_to_keep_pks_for\n\n # How best to deal with relationships?\n #\n # This doesn't work:\n # - process tables in order of dependencies, eager-loading\n # relationships with\n # for relationship in insp.mapper.relationships: # type: RelationshipProperty # noqa\n # related_col = getattr(orm_class, relationship.key)\n # query = query.options(joinedload(related_col))\n # - expunge from old session / make_transient / wipe_primary_key/ add\n # to new session\n # ... get errors like\n # sqlalchemy.exc.InvalidRequestError: Object '' is already attached to session '7' (this is\n # '6')\n #\n # ... at the point of dst_session.add(instance)\n # ... when adding the object on the child side of the relationship\n # ... I suspect that we move the Parent from session S to session D,\n # but when we eager-load the Parent from the Child, that makes\n # another in session S, so when we add the Child to session D, its\n # parent is in session S, which is wrong.\n #\n # We must, therefore, take a more interventional approach, in which we\n # maintain a copy of the old object, make a copy using\n # copy_sqla_object, and re-assign relationships accordingly.\n\n for instance in query.all():\n # log.debug(\"Source instance: {!r}\", instance)\n table_record_num += 1\n overall_record_num += 1\n if table_record_num % report_every == 0:\n log.info(\"... progress{}: on table {} ({}); record {} this \"\n \"table; overall record {}\",\n \" (DUMMY RUN)\" if dummy_run else \"\",\n table_num, tablename,\n table_record_num, overall_record_num)\n\n if tdc.standalone:\n # Our table has neither parents nor children. We can therefore\n # simply move the instance from one session to the other,\n # blanking primary keys.\n\n # https://stackoverflow.com/questions/14636192/sqlalchemy-modification-of-detached-object # noqa\n src_session.expunge(instance)\n make_transient(instance)\n if wipe_pk:\n wipe_primary_key(instance)\n\n instance = translate(instance, instance)\n if not instance:\n continue # translate_fn elected to skip it\n\n if not dummy_run:\n dst_session.add(instance)\n # new PK will be created when session is flushed\n\n else:\n # Our table has either parents or children. We therefore make\n # a copy and place the COPY in the destination session. If\n # this object may be a parent, we maintain a log (in objmap)\n # of the old-to-new mapping. If this object is a child, we\n # re-assign its relationships based on the old-to-new mapping\n # (since we will have processed the parent table first, having\n # carefully ordered them in advance).\n\n oldobj = instance # rename for clarity\n newobj = copy_sqla_object(\n oldobj, omit_pk=wipe_pk, omit_fk=True,\n omit_attrs=missing_attrs, debug=False\n )\n\n rewrite_relationships(oldobj, newobj, objmap, debug=False,\n skip_table_names=skip_table_names)\n\n newobj = translate(oldobj, newobj)\n if not newobj:\n continue # translate_fn elected to skip it\n\n if not dummy_run:\n dst_session.add(newobj)\n # new PK will be created when session is flushed\n\n if tdc.is_parent:\n objmap[oldobj] = newobj # for its children's benefit\n\n if flush_per_record:\n flush()\n\n if flush_per_table:\n flush()\n\n flush()\n if commit_at_end:\n log.debug(\"Committing...\")\n dst_session.commit()\n log.info(\"merge_db(): finished\")", "response": "Merges the SQLAlchemy database with another SQLAlchemy database."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nsets the metadata for the parent and child tables.", "response": "def set_metadata(self, metadata: MetaData) -> None:\n \"\"\"\n Sets the metadata for the parent and child tables.\n \"\"\"\n self._parent.set_metadata(metadata)\n self._child.set_metadata(metadata)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef set_metadata_if_none(self, metadata: MetaData) -> None:\n self._parent.set_metadata_if_none(metadata)\n self._child.set_metadata_if_none(metadata)", "response": "Sets the metadata for the parent and child tables unless they were set already."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nmark this table as circular.", "response": "def set_circular(self, circular: bool, chain: List[Table] = None) -> None:\n \"\"\"\n Mark this table as circular (or not).\n\n Args:\n circular: is it circular?\n chain: if it's circular, this should be the list of tables\n participating in the circular chain\n \"\"\"\n self.circular = circular\n self.circular_chain = chain or []"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef description(self) -> str:\n if self.is_parent and self.is_child:\n desc = \"parent+child\"\n elif self.is_parent:\n desc = \"parent\"\n elif self.is_child:\n desc = \"child\"\n else:\n desc = \"standalone\"\n if self.circular:\n desc += \"+CIRCULAR({})\".format(self.circular_description)\n return desc", "response": "Short description of the current object."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nreturning a binary version of an openpyxl. Workbook representation of an Excel file.", "response": "def excel_to_bytes(wb: Workbook) -> bytes:\n \"\"\"\n Obtain a binary version of an :class:`openpyxl.Workbook` representation of\n an Excel file.\n \"\"\"\n memfile = io.BytesIO()\n wb.save(memfile)\n return memfile.getvalue()"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef spherical(coordinates):\n c = coordinates\n r = N.linalg.norm(c,axis=0)\n theta = N.arccos(c[2]/r)\n phi = N.arctan2(c[1],c[0])\n return N.column_stack((r,theta,phi))", "response": "Returns a spherical formula from a list of coordinates."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef centered(coordinates):\n coordinates = N.array(coordinates)\n means = N.mean(coordinates,axis=0)\n return coordinates - means", "response": "Returns the center of the given coordinates."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nadds HTTP headers to a Django response class object.", "response": "def add_http_headers_for_attachment(response: HttpResponse,\n offered_filename: str = None,\n content_type: str = None,\n as_attachment: bool = False,\n as_inline: bool = False,\n content_length: int = None) -> None:\n \"\"\"\n Add HTTP headers to a Django response class object.\n\n Args:\n\n response: ``HttpResponse`` instance\n offered_filename: filename that the client browser will suggest\n content_type: HTTP content type\n as_attachment: if True, browsers will generally save to disk.\n If False, they may display it inline.\n http://www.w3.org/Protocols/rfc2616/rfc2616-sec19.html\n as_inline: attempt to force inline (only if not as_attachment)\n content_length: HTTP content length\n\n \"\"\"\n if offered_filename is None:\n offered_filename = ''\n if content_type is None:\n content_type = 'application/force-download'\n response['Content-Type'] = content_type\n if as_attachment:\n prefix = 'attachment; '\n elif as_inline:\n prefix = 'inline; '\n else:\n prefix = ''\n fname = 'filename=%s' % smart_str(offered_filename)\n response['Content-Disposition'] = prefix + fname\n if content_length is not None:\n response['Content-Length'] = content_length"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nserve up a file from disk.", "response": "def serve_file(path_to_file: str,\n offered_filename: str = None,\n content_type: str = None,\n as_attachment: bool = False,\n as_inline: bool = False) -> HttpResponseBase:\n \"\"\"\n Serve up a file from disk.\n Two methods (chosen by ``settings.XSENDFILE``):\n (a) serve directly\n (b) serve by asking the web server to do so via the X-SendFile directive.\n \"\"\"\n # http://stackoverflow.com/questions/1156246/having-django-serve-downloadable-files # noqa\n # https://docs.djangoproject.com/en/dev/ref/request-response/#telling-the-browser-to-treat-the-response-as-a-file-attachment # noqa\n # https://djangosnippets.org/snippets/365/\n if offered_filename is None:\n offered_filename = os.path.basename(path_to_file) or ''\n if settings.XSENDFILE:\n response = HttpResponse()\n response['X-Sendfile'] = smart_str(path_to_file)\n content_length = os.path.getsize(path_to_file)\n else:\n response = FileResponse(open(path_to_file, mode='rb'))\n content_length = None\n add_http_headers_for_attachment(response,\n offered_filename=offered_filename,\n content_type=content_type,\n as_attachment=as_attachment,\n as_inline=as_inline,\n content_length=content_length)\n return response"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef serve_buffer(data: bytes,\n offered_filename: str = None,\n content_type: str = None,\n as_attachment: bool = True,\n as_inline: bool = False) -> HttpResponse:\n \"\"\"\n Serve up binary data from a buffer.\n Options as for ``serve_file()``.\n \"\"\"\n response = HttpResponse(data)\n add_http_headers_for_attachment(response,\n offered_filename=offered_filename,\n content_type=content_type,\n as_attachment=as_attachment,\n as_inline=as_inline,\n content_length=len(data))\n return response", "response": "Serve up binary data from a buffer."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nadding a Content - Disposition header to the HTTP response to say that there is an attachment with the specified filename.", "response": "def add_download_filename(response: HttpResponse, filename: str) -> None:\n \"\"\"\n Adds a ``Content-Disposition`` header to the HTTP response to say that\n there is an attachment with the specified filename.\n \"\"\"\n # https://docs.djangoproject.com/en/1.9/howto/outputting-csv/\n add_http_headers_for_attachment(response)\n response['Content-Disposition'] = 'attachment; filename=\"{}\"'.format(\n filename)"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef file_response(data: Union[bytes, str], # HttpResponse encodes str if req'd\n content_type: str,\n filename: str) -> HttpResponse:\n \"\"\"\n Returns an ``HttpResponse`` with an attachment containing the specified\n data with the specified filename as an attachment.\n \"\"\"\n response = HttpResponse(data, content_type=content_type)\n add_download_filename(response, filename)\n return response", "response": "Returns an HttpResponse with an attachment containing the specified data with the specified content type and filename as an attachment."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nserves a single PDF from disk.", "response": "def serve_concatenated_pdf_from_disk(\n filenames: Iterable[str],\n offered_filename: str = \"crate_download.pdf\",\n **kwargs) -> HttpResponse:\n \"\"\"\n Concatenates PDFs from disk and serves them.\n \"\"\"\n pdf = get_concatenated_pdf_from_disk(filenames, **kwargs)\n return serve_buffer(pdf,\n offered_filename=offered_filename,\n content_type=MimeType.PDF,\n as_attachment=False,\n as_inline=True)"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef serve_pdf_from_html(html: str,\n offered_filename: str = \"test.pdf\",\n **kwargs) -> HttpResponse:\n \"\"\"\n Same args as ``pdf_from_html()``.\n WATCH OUT: may not apply e.g. wkhtmltopdf options as you'd wish.\n \"\"\"\n pdf = get_pdf_from_html(html, **kwargs)\n return serve_buffer(pdf,\n offered_filename=offered_filename,\n content_type=MimeType.PDF,\n as_attachment=False,\n as_inline=True)", "response": "Serve a PDF from a HTML page."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nserving a single PDF from a list of PDFs into memory and serve it.", "response": "def serve_concatenated_pdf_from_memory(\n pdf_plans: Iterable[PdfPlan],\n start_recto: bool = True,\n offered_filename: str = \"crate_download.pdf\") -> HttpResponse:\n \"\"\"\n Concatenates PDFs into memory and serves it.\n WATCH OUT: may not apply e.g. wkhtmltopdf options as you'd wish.\n \"\"\"\n pdf = get_concatenated_pdf_in_memory(pdf_plans, start_recto=start_recto)\n return serve_buffer(pdf,\n offered_filename=offered_filename,\n content_type=MimeType.PDF,\n as_attachment=False,\n as_inline=True)"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\ndeletes files from filesystem when object is deleted.", "response": "def auto_delete_files_on_instance_delete(instance: Any,\n fieldnames: Iterable[str]) -> None:\n \"\"\"\n Deletes files from filesystem when object is deleted.\n \"\"\"\n for fieldname in fieldnames:\n filefield = getattr(instance, fieldname, None)\n if filefield:\n if os.path.isfile(filefield.path):\n os.remove(filefield.path)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef auto_delete_files_on_instance_change(\n instance: Any,\n fieldnames: Iterable[str],\n model_class) -> None:\n \"\"\"\n Deletes files from filesystem when object is changed.\n\n model_class: ``Type[Model]``\n\n ... only the type checker in Py3.5 is broken; v.s.\n \"\"\"\n\n if not instance.pk:\n return # instance not yet saved in database\n # noinspection PyUnresolvedReferences\n try:\n # noinspection PyUnresolvedReferences\n old_instance = model_class.objects.get(pk=instance.pk)\n except model_class.DoesNotExist:\n return # old version gone from database entirely\n for fieldname in fieldnames:\n old_filefield = getattr(old_instance, fieldname, None)\n if not old_filefield:\n continue\n new_filefield = getattr(instance, fieldname, None)\n if old_filefield != new_filefield:\n if os.path.isfile(old_filefield.path):\n os.remove(old_filefield.path)", "response": "Deletes files from filesystem when instance is changed."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef axis_transform(pca_axes):\n from_ = N.identity(3)\n to_ = pca_axes\n\n # Find inverse transform for forward transform\n # y = M x -> M = y (x)^(-1)\n # We don't need to do least-squares since\n # there is a simple transformation\n trans_matrix = N.linalg.lstsq(from_,to_)[0]\n return trans_matrix", "response": "Creates an affine transformation matrix to rotate data in PCA axes into Cartesian plane."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nconstructing the covariance matrix of the input data from the singular value decomposition.", "response": "def covariance_matrix(self):\n \"\"\"\n Constructs the covariance matrix of\n input data from\n the singular value decomposition. Note\n that this is different than a covariance\n matrix of residuals, which is what we want\n for calculating fit errors.\n\n Using SVD output to compute covariance matrix\n X=U\u03a3V\u22a4\n XX\u22a4XX\u22a4=(U\u03a3V\u22a4)(U\u03a3V\u22a4)\u22a4=(U\u03a3V\u22a4)(V\u03a3U\u22a4)\n V is an orthogonal matrix (V\u22a4V=I),\n covariance matrix of input data: XX\u22a4=U\u03a32U\u22a4\n\n Because the axes represent identity in the\n PCA coordinate system, the PCA major axes\n themselves represent an affine transformation\n matrix from PCA to Cartesian space\n \"\"\"\n\n a = N.dot(self.U,self.sigma)\n cv = N.dot(a,a.T)\n # This yields the covariance matrix in Cartesian\n # coordinates\n return cv"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef U(self):\n if self._U is None:\n sinv = N.diag(1/self.singular_values)\n self._U = dot(self.arr,self.V.T,sinv)\n return self._U", "response": "Property to support lazy evaluation of residuals\n "} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef residuals(self):\n _ = self.rotated()\n _[:,-1] = 0\n _ = N.dot(_,self.axes)\n return self.arr - _", "response": "Returns residuals of fit against all\n "} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef angular_error(self, axis_length):\n hyp_axes = self.method(self)\n return N.arctan2(hyp_axes[-1],axis_length)", "response": "Returns the angular error for an in - plane axis of\n given length."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef angular_errors(self, degrees=True):\n hyp_axes = self.method(self)\n v = angular_errors(hyp_axes)\n if degrees:\n v = N.degrees(v)\n return tuple(v)", "response": "Returns the minimum and maximum angular errors of the current class."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _covariance_matrix(self, type='noise'):\n if type == 'sampling':\n return self.sigma**2/(self.n-1)\n elif type == 'noise':\n return 4*self.sigma*N.var(self.rotated(), axis=0)", "response": "Constructs the covariance matrix from PCA\n residuals."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef explained_variance(self):\n v = N.diagonal(self.covariance_matrix)\n return v[0:2].sum()/v.sum()", "response": "Returns the explained variance of the log - likelihood of the current cluster entry."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\ncomputing strike and dip from a normal vector.", "response": "def strike_dip(self, degrees=True):\n \"\"\"\n Computes strike and dip from a normal vector.\n Results are usually the same as LLSQ\n in strike (to a few decimal places) and close in dip.\n Sometimes, dips are greater by as much as 45 degrees,\n reflecting inclusion of errors in x-y plane.\n \"\"\"\n n = self.axes[2]\n r = N.linalg.norm(n)\n strike = N.degrees(N.arctan2(n[0],n[1]))-90\n dip = N.degrees(N.arccos(n[2]/r))\n\n # Since PCA errors are not pinned to the XYZ plane,\n # we need to make sure our results are in the\n # right quadrant.\n if dip > 90:\n dip = 180 - dip\n strike += 180\n\n # Proper azimuth notation\n if strike < 0:\n strike += 360\n\n return strike, dip"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef as_hyperbola(self, rotated=False):\n idx = N.diag_indices(3)\n _ = 1/self.covariance_matrix[idx]\n d = list(_)\n d[-1] *= -1\n\n arr = N.identity(4)*-1\n arr[idx] = d\n hyp = conic(arr)\n if rotated:\n R = augment(self.axes)\n hyp = hyp.transform(R)\n return hyp", "response": "Return the hyperbolic error area of the current object."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nextracting words from tweet 69.", "response": "def run(self, line):\n \"\"\"\n Extract words from tweet\n\n 1. Remove non-ascii characters\n 2. Split line into individual words\n 3. Clean up puncuation characters\n \n \"\"\"\n words = []\n for word in self.clean_unicode(line.lower()).split():\n if word.startswith('http'):\n continue\n cleaned = self.clean_punctuation(word)\n if len(cleaned) > 1 and cleaned not in self.stopwords:\n words.append(cleaned)\n return words"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef get_rows_fieldnames_from_query(\n session: Union[Session, Engine, Connection],\n query: Query) -> Tuple[Sequence[Sequence[Any]], Sequence[str]]:\n \"\"\"\n Returns results and column names from a query.\n\n Args:\n session: SQLAlchemy :class:`Session`, :class:`Engine`, or\n :class:`Connection` object\n query: SQLAlchemy :class:`Query`\n\n Returns:\n ``(rows, fieldnames)`` where ``rows`` is the usual set of results and\n ``fieldnames`` are the name of the result columns/fields.\n\n \"\"\"\n # https://stackoverflow.com/questions/6455560/how-to-get-column-names-from-sqlalchemy-result-declarative-syntax # noqa\n # No! Returns e.g. \"User\" for session.Query(User)...\n # fieldnames = [cd['name'] for cd in query.column_descriptions]\n result = session.execute(query) # type: ResultProxy\n fieldnames = result.keys()\n # ... yes! Comes out as \"_table_field\", which is how SQLAlchemy SELECTs\n # things.\n rows = result.fetchall()\n return rows, fieldnames", "response": "Returns the rows and column names from a SQLAlchemy query."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nconverting a SQLAlchemy Exists clause to a boolean answer.", "response": "def bool_from_exists_clause(session: Session,\n exists_clause: Exists) -> bool:\n \"\"\"\n Database dialects are not consistent in how ``EXISTS`` clauses can be\n converted to a boolean answer. This function manages the inconsistencies.\n\n See:\n \n - https://bitbucket.org/zzzeek/sqlalchemy/issues/3212/misleading-documentation-for-queryexists\n - http://docs.sqlalchemy.org/en/latest/orm/query.html#sqlalchemy.orm.query.Query.exists\n \n Specifically, we want this:\n \n *SQL Server*\n \n .. code-block:: sql\n \n SELECT 1 WHERE EXISTS (SELECT 1 FROM table WHERE ...)\n -- ... giving 1 or None (no rows)\n -- ... fine for SQL Server, but invalid for MySQL (no FROM clause)\n \n *Others, including MySQL*\n \n .. code-block:: sql\n \n SELECT EXISTS (SELECT 1 FROM table WHERE ...)\n -- ... giving 1 or 0\n -- ... fine for MySQL, but invalid syntax for SQL Server\n \n \"\"\" # noqa\n if session.get_bind().dialect.name == SqlaDialectName.MSSQL:\n # SQL Server\n result = session.query(literal(True)).filter(exists_clause).scalar()\n else:\n # MySQL, etc.\n result = session.query(exists_clause).scalar()\n return bool(result)"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\ndetermine whether a database record exists for the specified ormclass and criteria.", "response": "def exists_orm(session: Session,\n ormclass: DeclarativeMeta,\n *criteria: Any) -> bool:\n \"\"\"\n Detects whether a database record exists for the specified ``ormclass``\n and ``criteria``.\n\n Example usage:\n\n .. code-block:: python\n\n bool_exists = exists_orm(session, MyClass, MyClass.myfield == value)\n \"\"\"\n # http://docs.sqlalchemy.org/en/latest/orm/query.html\n q = session.query(ormclass)\n for criterion in criteria:\n q = q.filter(criterion)\n exists_clause = q.exists()\n return bool_from_exists_clause(session=session,\n exists_clause=exists_clause)"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef get_or_create(session: Session,\n model: DeclarativeMeta,\n defaults: Dict[str, Any] = None,\n **kwargs: Any) -> Tuple[Any, bool]:\n \"\"\"\n Fetches an ORM object from the database, or creates one if none existed.\n\n Args:\n session: an SQLAlchemy :class:`Session`\n model: an SQLAlchemy ORM class\n defaults: default initialization arguments (in addition to relevant\n filter criteria) if we have to create a new instance\n kwargs: optional filter criteria\n\n Returns:\n a tuple ``(instance, newly_created)``\n\n See http://stackoverflow.com/questions/2546207 (this function is a\n composite of several suggestions).\n \"\"\"\n instance = session.query(model).filter_by(**kwargs).first()\n if instance:\n return instance, False\n else:\n params = dict((k, v) for k, v in kwargs.items()\n if not isinstance(v, ClauseElement))\n params.update(defaults or {})\n instance = model(**params)\n session.add(instance)\n return instance, True", "response": "Fetches an ORM object from the database or creates one if none existed."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nreturns the number of starred entries in the database.", "response": "def count_star(self) -> int:\n \"\"\"\n Implements the ``COUNT(*)`` specialization.\n \"\"\"\n count_query = (self.statement.with_only_columns([func.count()])\n .order_by(None))\n return self.session.execute(count_query).scalar()"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\ndownload GeoJSON file of US counties from url and return it as string.", "response": "def read(self, src):\n \"\"\" Download GeoJSON file of US counties from url (S3 bucket) \"\"\"\n geojson = None\n if not self.is_valid_src(src):\n error = \"File < {0} > does not exists or does start with 'http'.\"\n raise ValueError(error.format(src))\n if not self.is_url(src):\n return open(src, 'r').read().decode('latin-1').encode('utf-8')\n tmp = self.get_location(src)\n # if src poits to url that was already downloaded\n # read from local file instead\n if os.path.isfile(tmp):\n with open(tmp, 'r') as f:\n return f.read()\n # download file and write to local filesystem before returning\n response = urllib2.urlopen(src)\n data = response.read().decode('latin-1').encode('utf-8')\n with open(tmp, 'w') as f:\n f.write(data)\n return data"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef create_base64encoded_randomness(num_bytes: int) -> str:\n randbytes = os.urandom(num_bytes) # YES\n return base64.urlsafe_b64encode(randbytes).decode('ascii')", "response": "Create and return num_bytes of random data."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nreturn a SQLAlchemy Table object that is used for initialization.", "response": "def table(self) -> Table:\n \"\"\"\n Returns a SQLAlchemy :class:`Table` object. This is either the\n :class:`Table` object that was used for initialization, or one that\n was constructed from the ``tablename`` plus the ``metadata``.\n \"\"\"\n if self._table is not None:\n return self._table\n assert self._metadata, (\n \"Must specify metadata (in constructor or via set_metadata()/\"\n \"set_metadata_if_none() before you can get a Table from a \"\n \"tablename\"\n )\n for table in self._metadata.tables.values(): # type: Table\n if table.name == self._tablename:\n return table\n raise ValueError(\"No table named {!r} is present in the \"\n \"metadata\".format(self._tablename))"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nreturns the string name of the table.", "response": "def tablename(self) -> str:\n \"\"\"\n Returns the string name of the table.\n \"\"\"\n if self._tablename:\n return self._tablename\n return self.table.name"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef is_running(process_id: int) -> bool:\n pstr = str(process_id)\n encoding = sys.getdefaultencoding()\n s = subprocess.Popen([\"ps\", \"-p\", pstr], stdout=subprocess.PIPE)\n for line in s.stdout:\n strline = line.decode(encoding)\n if pstr in strline:\n return True\n return False", "response": "Checks if a process is running."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef main() -> None:\n parser = ArgumentParser(\n description=\"Pauses and resumes a process by disk space; LINUX ONLY.\"\n )\n parser.add_argument(\n \"process_id\", type=int,\n help=\"Process ID.\"\n )\n parser.add_argument(\n \"--path\", required=True,\n help=\"Path to check free space for (e.g. '/')\"\n )\n parser.add_argument(\n \"--pause_when_free_below\", type=str, required=True,\n help=\"Pause process when free disk space below this value (in bytes \"\n \"or as e.g. '50G')\"\n )\n parser.add_argument(\n \"--resume_when_free_above\", type=str, required=True,\n help=\"Resume process when free disk space above this value (in bytes \"\n \"or as e.g. '70G')\"\n )\n parser.add_argument(\n \"--check_every\", type=int, required=True,\n help=\"Check every n seconds (where this is n)\"\n )\n parser.add_argument(\n \"--verbose\", action=\"store_true\",\n help=\"Verbose output\"\n )\n args = parser.parse_args()\n main_only_quicksetup_rootlogger(\n level=logging.DEBUG if args.verbose else logging.INFO)\n\n minimum = human2bytes(args.pause_when_free_below)\n maximum = human2bytes(args.resume_when_free_above)\n path = args.path\n process_id = args.process_id\n period = args.check_every\n pause_args = [\"kill\", \"-STOP\", str(process_id)]\n resume_args = [\"kill\", \"-CONT\", str(process_id)]\n\n assert minimum < maximum, \"Minimum must be less than maximum\"\n\n log.info(\n \"Starting: controlling process {proc}; \"\n \"checking disk space every {period} s; \"\n \"will pause when free space on {path} is less than {minimum} and \"\n \"resume when free space is at least {maximum}; \"\n \"pause command will be {pause}; \"\n \"resume command will be {resume}.\".format(\n proc=process_id,\n period=period,\n path=path,\n minimum=sizeof_fmt(minimum),\n maximum=sizeof_fmt(maximum),\n pause=pause_args,\n resume=resume_args,\n ))\n log.debug(\"Presuming that the process is RUNNING to begin with.\")\n\n paused = False\n while True:\n if not is_running(process_id):\n log.info(\"Process {} is no longer running\", process_id)\n sys.exit(0)\n space = shutil.disk_usage(path).free\n log.debug(\"Disk space on {} is {}\", path, sizeof_fmt(space))\n if space < minimum and not paused:\n log.info(\"Disk space down to {}: pausing process {}\",\n sizeof_fmt(space), process_id)\n subprocess.check_call(pause_args)\n paused = True\n elif space >= maximum and paused:\n log.info(\"Disk space up to {}: resuming process {}\",\n sizeof_fmt(space), process_id)\n subprocess.check_call(resume_args)\n paused = False\n log.debug(\"Sleeping for {} seconds...\", period)\n sleep(period)", "response": "Main function for the pause_process_by_disk_space tool."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nbootstrap noise on the data.", "response": "def bootstrap_noise(data, func, n=10000, std=1, symmetric=True):\n \"\"\"\n Bootstrap by adding noise\n \"\"\"\n boot_dist = []\n arr = N.zeros(data.shape)\n for i in range(n):\n if symmetric:\n # Noise on all three axes\n arr = N.random.randn(*data.shape)*std\n else:\n # Only z-coordinate noise\n arr[:,-1] = N.random.randn(data.shape[0])*std\n boot_dist.append(func(data+arr))\n return N.array(boot_dist)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nreturning a string representation of a Django model object.", "response": "def modelrepr(instance) -> str:\n \"\"\"\n Default ``repr`` version of a Django model object, for debugging.\n \"\"\"\n elements = []\n # noinspection PyProtectedMember\n for f in instance._meta.get_fields():\n # https://docs.djangoproject.com/en/2.0/ref/models/meta/\n if f.auto_created:\n continue\n if f.is_relation and f.related_model is None:\n continue\n fieldname = f.name\n try:\n value = repr(getattr(instance, fieldname))\n except ObjectDoesNotExist:\n value = \"\"\n elements.append(\"{}={}\".format(fieldname, value))\n return \"<{} <{}>>\".format(type(instance).__name__,\n \", \".join(elements))"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef assert_processor_available(processor: str) -> None:\n if processor not in [Processors.XHTML2PDF,\n Processors.WEASYPRINT,\n Processors.PDFKIT]:\n raise AssertionError(\"rnc_pdf.set_pdf_processor: invalid PDF processor\"\n \" specified\")\n if processor == Processors.WEASYPRINT and not weasyprint:\n raise RuntimeError(\"rnc_pdf: Weasyprint requested, but not available\")\n if processor == Processors.XHTML2PDF and not xhtml2pdf:\n raise RuntimeError(\"rnc_pdf: xhtml2pdf requested, but not available\")\n if processor == Processors.PDFKIT and not pdfkit:\n raise RuntimeError(\"rnc_pdf: pdfkit requested, but not available\")", "response": "Assert that a specific PDF processor is available."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nmakes a PDF from HTML.", "response": "def make_pdf_from_html(\n # Mandatory parameters:\n on_disk: bool,\n html: str,\n # Disk options:\n output_path: str = None,\n # Shared options:\n header_html: str = None,\n footer_html: str = None,\n wkhtmltopdf_filename: str = _WKHTMLTOPDF_FILENAME,\n wkhtmltopdf_options: Dict[str, Any] = None,\n file_encoding: str = \"utf-8\",\n debug_options: bool = False,\n debug_content: bool = False,\n debug_wkhtmltopdf_args: bool = True,\n fix_pdfkit_encoding_bug: bool = None,\n processor: str = _DEFAULT_PROCESSOR) -> Union[bytes, bool]:\n \"\"\"\n Takes HTML and either returns a PDF in memory or makes one on disk.\n\n For preference, uses ``wkhtmltopdf`` (with ``pdfkit``):\n\n - faster than ``xhtml2pdf``\n - tables not buggy like ``Weasyprint``\n - however, doesn't support CSS Paged Media, so we have the\n ``header_html`` and ``footer_html`` options to allow you to pass\n appropriate HTML content to serve as the header/footer (rather than\n passing it within the main HTML).\n\n Args:\n on_disk: make file on disk (rather than returning it in memory)?\n\n html: main HTML\n\n output_path: if ``on_disk``, the output filename\n\n header_html: optional page header, as HTML\n\n footer_html: optional page footer, as HTML\n\n wkhtmltopdf_filename: filename of the ``wkhtmltopdf`` executable\n\n wkhtmltopdf_options: options for ``wkhtmltopdf``\n\n file_encoding: encoding to use when writing the header/footer to disk\n\n debug_options: log ``wkhtmltopdf`` config/options passed to ``pdfkit``?\n\n debug_content: log the main/header/footer HTML?\n\n debug_wkhtmltopdf_args: log the final command-line arguments to\n that will be used by ``pdfkit`` when it calls ``wkhtmltopdf``?\n\n fix_pdfkit_encoding_bug: attempt to work around bug in e.g.\n ``pdfkit==0.5.0`` by encoding ``wkhtmltopdf_filename`` to UTF-8\n before passing it to ``pdfkit``? If you pass ``None`` here, then\n a default value is used, from\n :func:`get_default_fix_pdfkit_encoding_bug`.\n\n processor: a PDF processor type from :class:`Processors`\n\n Returns:\n the PDF binary as a ``bytes`` object\n\n Raises:\n AssertionError: if bad ``processor``\n RuntimeError: if requested processor is unavailable\n\n \"\"\"\n wkhtmltopdf_options = wkhtmltopdf_options or {} # type: Dict[str, Any]\n assert_processor_available(processor)\n\n if debug_content:\n log.debug(\"html: {}\", html)\n log.debug(\"header_html: {}\", header_html)\n log.debug(\"footer_html: {}\", footer_html)\n\n if fix_pdfkit_encoding_bug is None:\n fix_pdfkit_encoding_bug = get_default_fix_pdfkit_encoding_bug()\n\n if processor == Processors.XHTML2PDF:\n\n if on_disk:\n with open(output_path, mode='wb') as outfile:\n # noinspection PyUnresolvedReferences\n xhtml2pdf.document.pisaDocument(html, outfile)\n return True\n else:\n memfile = io.BytesIO()\n # noinspection PyUnresolvedReferences\n xhtml2pdf.document.pisaDocument(html, memfile)\n # ... returns a document, but we don't use it, so we don't store it\n # to stop pychecker complaining\n # http://xhtml2pdf.appspot.com/static/pisa-en.html\n memfile.seek(0)\n return memfile.read()\n # http://stackoverflow.com/questions/3310584\n\n elif processor == Processors.WEASYPRINT:\n\n if on_disk:\n return weasyprint.HTML(string=html).write_pdf(output_path)\n else:\n # http://ampad.de/blog/generating-pdfs-django/\n return weasyprint.HTML(string=html).write_pdf()\n\n elif processor == Processors.PDFKIT:\n\n # Config:\n if not wkhtmltopdf_filename:\n config = None\n else:\n if fix_pdfkit_encoding_bug: # needs to be True for pdfkit==0.5.0\n log.debug(\"Attempting to fix bug in pdfkit (e.g. version 0.5.0)\"\n \" by encoding wkhtmltopdf_filename to UTF-8\")\n config = pdfkit.configuration(\n wkhtmltopdf=wkhtmltopdf_filename.encode('utf-8'))\n # the bug is that pdfkit.pdfkit.PDFKit.__init__ will attempt to\n # decode the string in its configuration object;\n # https://github.com/JazzCore/python-pdfkit/issues/32\n else:\n config = pdfkit.configuration(wkhtmltopdf=wkhtmltopdf_filename)\n\n # Temporary files that a subprocess can read:\n # http://stackoverflow.com/questions/15169101\n # wkhtmltopdf requires its HTML files to have \".html\" extensions:\n # http://stackoverflow.com/questions/5776125\n h_filename = None\n f_filename = None\n try:\n if header_html:\n h_fd, h_filename = tempfile.mkstemp(suffix='.html')\n os.write(h_fd, header_html.encode(file_encoding))\n os.close(h_fd)\n wkhtmltopdf_options[\"header-html\"] = h_filename\n if footer_html:\n f_fd, f_filename = tempfile.mkstemp(suffix='.html')\n os.write(f_fd, footer_html.encode(file_encoding))\n os.close(f_fd)\n wkhtmltopdf_options[\"footer-html\"] = f_filename\n if debug_options:\n log.debug(\"wkhtmltopdf config: {!r}\", config)\n log.debug(\"wkhtmltopdf_options: {}\",\n pformat(wkhtmltopdf_options))\n kit = pdfkit.pdfkit.PDFKit(html, 'string', configuration=config,\n options=wkhtmltopdf_options)\n\n if on_disk:\n path = output_path\n else:\n path = None\n # With \"path=None\", the to_pdf() function directly returns\n # stdout from a subprocess.Popen().communicate() call (see\n # pdfkit.py). Since universal_newlines is not set, stdout will\n # be bytes in Python 3.\n\n if debug_wkhtmltopdf_args:\n log.debug(\"Probable current user: {!r}\", getpass.getuser())\n log.debug(\"wkhtmltopdf arguments will be: {!r}\",\n kit.command(path=path))\n\n return kit.to_pdf(path=path)\n\n finally:\n if h_filename:\n os.remove(h_filename)\n if f_filename:\n os.remove(f_filename)\n\n else:\n raise AssertionError(\"Unknown PDF engine\")"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\ntake a HTML string and returns a PDF.", "response": "def get_pdf_from_html(html: str,\n header_html: str = None,\n footer_html: str = None,\n wkhtmltopdf_filename: str = _WKHTMLTOPDF_FILENAME,\n wkhtmltopdf_options: Dict[str, Any] = None,\n file_encoding: str = \"utf-8\",\n debug_options: bool = False,\n debug_content: bool = False,\n debug_wkhtmltopdf_args: bool = True,\n fix_pdfkit_encoding_bug: bool = None,\n processor: str = _DEFAULT_PROCESSOR) -> bytes:\n \"\"\"\n Takes HTML and returns a PDF.\n\n See the arguments to :func:`make_pdf_from_html` (except ``on_disk``).\n\n Returns:\n the PDF binary as a ``bytes`` object\n \"\"\"\n result = make_pdf_from_html(\n on_disk=False,\n html=html,\n header_html=header_html,\n footer_html=footer_html,\n wkhtmltopdf_filename=wkhtmltopdf_filename,\n wkhtmltopdf_options=wkhtmltopdf_options,\n file_encoding=file_encoding,\n debug_options=debug_options,\n debug_content=debug_content,\n debug_wkhtmltopdf_args=debug_wkhtmltopdf_args,\n fix_pdfkit_encoding_bug=fix_pdfkit_encoding_bug,\n processor=processor,\n ) # type: bytes\n return result"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nextract a PDF from a PyPDF2 writer or merger object.", "response": "def pdf_from_writer(writer: Union[PdfFileWriter, PdfFileMerger]) -> bytes:\n \"\"\"\n Extracts a PDF (as binary data) from a PyPDF2 writer or merger object.\n \"\"\"\n memfile = io.BytesIO()\n writer.write(memfile)\n memfile.seek(0)\n return memfile.read()"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nappending a PDF to a PyPDF2 writer.", "response": "def append_memory_pdf_to_writer(input_pdf: bytes,\n writer: PdfFileWriter,\n start_recto: bool = True) -> None:\n \"\"\"\n Appends a PDF (as bytes in memory) to a PyPDF2 writer.\n\n Args:\n input_pdf: the PDF, as ``bytes``\n writer: the writer\n start_recto: start a new right-hand page?\n \"\"\"\n if not input_pdf:\n return\n if start_recto and writer.getNumPages() % 2 != 0:\n writer.addBlankPage()\n # ... suitable for double-sided printing\n infile = io.BytesIO(input_pdf)\n reader = PdfFileReader(infile)\n for page_num in range(reader.numPages):\n writer.addPage(reader.getPage(page_num))"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef append_pdf(input_pdf: bytes, output_writer: PdfFileWriter):\n append_memory_pdf_to_writer(input_pdf=input_pdf,\n writer=output_writer)", "response": "Append a PDF to a pyPDF writer."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef get_concatenated_pdf_from_disk(filenames: Iterable[str],\n start_recto: bool = True) -> bytes:\n \"\"\"\n Concatenates PDFs from disk and returns them as an in-memory binary PDF.\n\n Args:\n filenames: iterable of filenames of PDFs to concatenate\n start_recto: start a new right-hand page for each new PDF?\n\n Returns:\n concatenated PDF, as ``bytes``\n\n \"\"\"\n # http://stackoverflow.com/questions/17104926/pypdf-merging-multiple-pdf-files-into-one-pdf # noqa\n # https://en.wikipedia.org/wiki/Recto_and_verso\n if start_recto:\n writer = PdfFileWriter()\n for filename in filenames:\n if filename:\n if writer.getNumPages() % 2 != 0:\n writer.addBlankPage()\n writer.appendPagesFromReader(\n PdfFileReader(open(filename, 'rb')))\n return pdf_from_writer(writer)\n else:\n merger = PdfFileMerger()\n for filename in filenames:\n if filename:\n merger.append(open(filename, 'rb'))\n return pdf_from_writer(merger)", "response": "Returns a PDF that is concatenated from the given files."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nreturn a concatenation of the PDFs in - memory.", "response": "def get_concatenated_pdf_in_memory(\n pdf_plans: Iterable[PdfPlan],\n start_recto: bool = True) -> bytes:\n \"\"\"\n Concatenates PDFs and returns them as an in-memory binary PDF.\n\n Args:\n pdf_plans: iterable of :class:`PdfPlan` objects\n start_recto: start a new right-hand page for each new PDF?\n\n Returns:\n concatenated PDF, as ``bytes``\n\n \"\"\"\n writer = PdfFileWriter()\n for pdfplan in pdf_plans:\n pdfplan.add_to_writer(writer, start_recto=start_recto)\n return pdf_from_writer(writer)"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nadding the PDF described by this class to a PDF writer.", "response": "def add_to_writer(self,\n writer: PdfFileWriter,\n start_recto: bool = True) -> None:\n \"\"\"\n Add the PDF described by this class to a PDF writer.\n\n Args:\n writer: a :class:`PyPDF2.PdfFileWriter`\n start_recto: start a new right-hand page?\n\n \"\"\"\n if self.is_html:\n pdf = get_pdf_from_html(\n html=self.html,\n header_html=self.header_html,\n footer_html=self.footer_html,\n wkhtmltopdf_filename=self.wkhtmltopdf_filename,\n wkhtmltopdf_options=self.wkhtmltopdf_options)\n append_memory_pdf_to_writer(pdf, writer, start_recto=start_recto)\n elif self.is_filename:\n if start_recto and writer.getNumPages() % 2 != 0:\n writer.addBlankPage()\n writer.appendPagesFromReader(PdfFileReader(\n open(self.filename, 'rb')))\n else:\n raise AssertionError(\"PdfPlan: shouldn't get here!\")"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\ntruncate a number to its integer representation.", "response": "def trunc_if_integer(n: Any) -> Any:\n \"\"\"\n Truncates floats that are integers to their integer representation.\n That is, converts ``1.0`` to ``1``, etc.\n Otherwise, returns the starting value.\n Will raise an exception if the input cannot be converted to ``int``.\n \"\"\"\n if n == int(n):\n return int(n)\n return n"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef repr_result(obj: Any, elements: List[str],\n with_addr: bool = False, joiner: str = COMMA_SPACE) -> str:\n \"\"\"\n Internal function to make a :func:`repr`-style representation of an object.\n\n Args:\n obj: object to display\n elements: list of object ``attribute=value`` strings\n with_addr: include the memory address of ``obj``\n joiner: string with which to join the elements\n\n Returns:\n string: :func:`repr`-style representation\n\n \"\"\"\n if with_addr:\n return \"<{qualname}({elements}) at {addr}>\".format(\n qualname=obj.__class__.__qualname__,\n elements=joiner.join(elements),\n addr=hex(id(obj)))\n else:\n return \"{qualname}({elements})\".format(\n qualname=obj.__class__.__qualname__,\n elements=joiner.join(elements))", "response": "Internal function to make a : func:`repr` - style representation of an object."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef auto_repr(obj: Any, with_addr: bool = False,\n sort_attrs: bool = True, joiner: str = COMMA_SPACE) -> str:\n \"\"\"\n Convenience function for :func:`__repr__`.\n Works its way through the object's ``__dict__`` and reports accordingly.\n\n Args:\n obj: object to display\n with_addr: include the memory address of ``obj``\n sort_attrs: sort the attributes into alphabetical order?\n joiner: string with which to join the elements\n\n Returns:\n string: :func:`repr`-style representation\n \"\"\"\n if sort_attrs:\n keys = sorted(obj.__dict__.keys())\n else:\n keys = obj.__dict__.keys()\n elements = [\"{}={}\".format(k, repr(getattr(obj, k))) for k in keys]\n return repr_result(obj, elements, with_addr=with_addr, joiner=joiner)", "response": "Convenience function for printing the object s attributes into a string - style representation."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef simple_repr(obj: Any, attrnames: List[str],\n with_addr: bool = False, joiner: str = COMMA_SPACE) -> str:\n \"\"\"\n Convenience function for :func:`__repr__`.\n Works its way through a list of attribute names, and creates a ``repr()``\n representation assuming that parameters to the constructor have the same\n names.\n\n Args:\n obj: object to display\n attrnames: names of attributes to include\n with_addr: include the memory address of ``obj``\n joiner: string with which to join the elements\n\n Returns:\n string: :func:`repr`-style representation\n\n \"\"\"\n elements = [\"{}={}\".format(name, repr(getattr(obj, name)))\n for name in attrnames]\n return repr_result(obj, elements, with_addr=with_addr, joiner=joiner)", "response": "Simple repr function for the base class."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nreturn a string representation of the object with the given attribute names stripping underscore.", "response": "def mapped_repr_stripping_underscores(\n obj: Any, attrnames: List[str],\n with_addr: bool = False, joiner: str = COMMA_SPACE) -> str:\n \"\"\"\n Convenience function for :func:`__repr__`.\n Here, you pass a list of internal attributes, and it assumes that the\n :func:`__init__` parameter names have the leading underscore dropped.\n\n Args:\n obj: object to display\n attrnames: list of attribute names\n with_addr: include the memory address of ``obj``\n joiner: string with which to join the elements\n\n Returns:\n string: :func:`repr`-style representation\n\n \"\"\"\n attributes = []\n for attr_name in attrnames:\n if attr_name.startswith('_'):\n init_param_name = attr_name[1:]\n else:\n init_param_name = attr_name\n attributes.append((attr_name, init_param_name))\n return mapped_repr(obj, attributes, with_addr=with_addr, joiner=joiner)"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nreturns a string representation of the object with the given attribute list.", "response": "def ordered_repr(obj: object, attrlist: Iterable[str],\n joiner: str = COMMA_SPACE) -> str:\n \"\"\"\n Shortcut to make :func:`repr` functions ordered.\n Define your :func:`__repr__` like this:\n\n .. code-block:: python\n\n def __repr__(self):\n return ordered_repr(self, [\"field1\", \"field2\", \"field3\"])\n\n Args:\n obj: object to display\n attrlist: iterable of attribute names\n joiner: string with which to join the elements\n\n Returns:\n string: :func:`repr`-style representation\n \"\"\"\n return \"<{classname}({kvp})>\".format(\n classname=type(obj).__name__,\n kvp=joiner.join(\"{}={}\".format(a, repr(getattr(obj, a)))\n for a in attrlist)\n )"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef auto_str(obj: Any, indent: int = 4, width: int = 80, depth: int = None,\n compact: bool = False) -> str:\n \"\"\"\n Make a pretty :func:`str()` representation using :func:`pprint.pformat`\n and the object's ``__dict__`` attribute.\n\n Args:\n obj: object to display\n indent: see\n https://docs.python.org/3/library/pprint.html#pprint.PrettyPrinter\n width: as above\n depth: as above\n compact: as above\n\n Returns:\n string: :func:`str`-style representation\n\n \"\"\"\n return pprint.pformat(obj.__dict__, indent=indent, width=width,\n depth=depth, compact=compact)", "response": "Return a pretty string representation of the object."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef timedelta_days(days: int) -> timedelta64:\n int_days = int(days)\n if int_days != days:\n raise ValueError(\"Fractional days passed to timedelta_days: \"\n \"{!r}\".format(days))\n try:\n # Do not pass e.g. 27.0; that will raise a ValueError.\n # Must be an actual int:\n return timedelta64(int_days, 'D')\n except ValueError as e:\n raise ValueError(\"Failure in timedelta_days; value was {!r}; original \"\n \"error was: {}\".format(days, e))", "response": "Convert a duration in days to a NumPy timedelta64 object."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _get_generic_two_antidep_episodes_result(\n rowdata: Tuple[Any, ...] = None) -> DataFrame:\n \"\"\"\n Create a results row for this application.\n \"\"\"\n # Valid data types... see:\n # - pandas.core.dtypes.common.pandas_dtype\n # - https://pandas.pydata.org/pandas-docs/stable/timeseries.html\n # - https://docs.scipy.org/doc/numpy-1.13.0/reference/arrays.datetime.html\n data = [rowdata] if rowdata else []\n return DataFrame(array(\n data, # data\n dtype=[ # column definitions:\n (RCN_PATIENT_ID, DTYPE_STRING),\n (RCN_DRUG_A_NAME, DTYPE_STRING),\n (RCN_DRUG_A_FIRST_MENTION, DTYPE_DATE),\n (RCN_DRUG_A_SECOND_MENTION, DTYPE_DATE),\n (RCN_DRUG_B_NAME, DTYPE_STRING),\n (RCN_DRUG_B_FIRST_MENTION, DTYPE_DATE),\n (RCN_DRUG_B_SECOND_MENTION, DTYPE_DATE),\n (RCN_EXPECT_RESPONSE_BY_DATE, DTYPE_DATE),\n (RCN_END_OF_SYMPTOM_PERIOD, DTYPE_DATE),\n ]\n ))", "response": "Create a result row for this application."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef two_antidepressant_episodes_single_patient(\n patient_id: str,\n patient_drug_date_df: DataFrame,\n patient_colname: str = DEFAULT_SOURCE_PATIENT_COLNAME,\n drug_colname: str = DEFAULT_SOURCE_DRUG_COLNAME,\n date_colname: str = DEFAULT_SOURCE_DATE_COLNAME,\n course_length_days: int = DEFAULT_ANTIDEPRESSANT_COURSE_LENGTH_DAYS,\n expect_response_by_days: int = DEFAULT_EXPECT_RESPONSE_BY_DAYS,\n symptom_assessment_time_days: int = DEFAULT_SYMPTOM_ASSESSMENT_TIME_DAYS, # noqa\n first_episode_only: bool = True) -> Optional[DataFrame]:\n \"\"\"\n Processes a single patient for ``two_antidepressant_episodes()`` (q.v.).\n\n Implements the key algorithm.\n \"\"\"\n log.debug(\"Running two_antidepressant_episodes_single_patient() for \"\n \"patient {!r}\".format(patient_id))\n all_episodes_for_patient = _get_blank_two_antidep_episodes_result()\n flush_stdout_stderr()\n # Get column details from source data\n sourcecolnum_drug = patient_drug_date_df.columns.get_loc(drug_colname)\n sourcecolnum_date = patient_drug_date_df.columns.get_loc(date_colname)\n\n # -------------------------------------------------------------------------\n # Get data for this patient\n # -------------------------------------------------------------------------\n # ... this is pretty quick (e.g. 4ms for 1150 rows\n patient_mask = patient_drug_date_df[patient_colname].values == patient_id\n tp = patient_drug_date_df[patient_mask] # type: DataFrame\n\n # -------------------------------------------------------------------------\n # Sort by date, then drug.\n # ... arbitrary drug name order to make the output stable\n # -------------------------------------------------------------------------\n # ... this is about 2ms for small lists; probably not limiting\n # ... seems slower if \"inplace=True\" is used.\n tp = tp.sort_values(by=[date_colname, drug_colname], ascending=True)\n\n # log.critical(\"{!r}\", tp)\n\n nrows_all = len(tp) # https://stackoverflow.com/questions/15943769/\n if nrows_all < 4: # need A, A, B, B; so minimum #rows is 4\n return None\n end_date = tp.iat[nrows_all - 1, sourcecolnum_date] # date of last row\n\n # -------------------------------------------------------------------------\n # Get antidepressants, in the order they appear\n # -------------------------------------------------------------------------\n for first_b_rownum in range(1, nrows_all):\n # ... skip rows 0 and 1, because a drug can't be the second (B) drug\n # unless there are two mentions of A beforehand.\n # ... modified 2019-04-01 from range(2, nrows_all) to\n # range(1, nrows_all) because we now support A ending on the same\n # day that B starts (so first possible \"first B\" row is the second\n # row, row 1)\n # ---------------------------------------------------------------------\n # Check candidate B drug\n # ---------------------------------------------------------------------\n antidepressant_b_name = tp.iat[first_b_rownum, sourcecolnum_drug]\n antidepressant_b_first_mention = tp.iat[first_b_rownum,\n sourcecolnum_date]\n earliest_possible_b_second_mention = (\n antidepressant_b_first_mention +\n timedelta_days(course_length_days - 1)\n )\n if earliest_possible_b_second_mention > end_date:\n # Impossible for this to be a B course.\n # Logically unnecessary test, but improves efficiency by skipping\n # the slice operation that follows.\n continue # try another B\n b_second_mentions = tp[\n (tp[drug_colname] == antidepressant_b_name) & # same drug\n (tp[date_colname] >= earliest_possible_b_second_mention)\n ]\n if len(b_second_mentions) == 0:\n # No second mention of antidepressant_b_name\n continue # try another B\n # We only care about the earliest qualifying (completion-of-course)\n # B second mention.\n antidepressant_b_second_mention = b_second_mentions.iat[\n 0, sourcecolnum_date]\n # ... this statement could be moved to after the A loop, but that\n # would sacrifice clarity.\n\n # ---------------------------------------------------------------------\n # Now find preceding A drug\n # ---------------------------------------------------------------------\n preceding_other_antidepressants = tp[\n (tp[drug_colname] != antidepressant_b_name) &\n # ... A is a different drug to B\n\n # Changed 2019-04-01: WAS THIS:\n # (tp[date_colname] < antidepressant_b_first_mention)\n # # ... A is mentioned before B starts\n # CHANGED TO THIS:\n (tp[date_colname] <= antidepressant_b_first_mention)\n # ... A is mentioned before B starts or on the same day\n ]\n nrows_a = len(preceding_other_antidepressants)\n if nrows_a < 2: # need at least two mentions of A\n # No candidates for A\n continue # try another B\n # preceding_other_antidepressants remains date-sorted (ascending)\n found_valid_a = False\n antidepressant_a_name = NaN\n antidepressant_a_first_mention = NaN\n antidepressant_a_second_mention = NaN\n for first_a_rownum in range(nrows_a - 1):\n # skip the last row, as that's impossible (the first mention of\n # A cannot be the last row since there must be two mentions of A)\n antidepressant_a_name = tp.iat[first_a_rownum, sourcecolnum_drug]\n antidepressant_a_first_mention = tp.iat[first_a_rownum,\n sourcecolnum_date]\n earliest_possible_a_second_mention = (\n antidepressant_a_first_mention +\n timedelta_days(course_length_days - 1)\n )\n\n # 2019-04-01: CHANGED FROM:\n # if (earliest_possible_a_second_mention >=\n # antidepressant_b_first_mention):\n # TO:\n if (earliest_possible_a_second_mention >\n antidepressant_b_first_mention):\n # Impossible to squeeze in the second A mention before B.\n # Logically unnecessary test, but improves efficiency by\n # skipping the slice operation that follows.\n continue # try another A\n a_second_mentions = tp[\n (tp[drug_colname] == antidepressant_a_name) &\n # ... same drug\n (tp[date_colname] >= earliest_possible_a_second_mention)\n # ... mentioned late enough after its first mention\n ]\n if len(a_second_mentions) == 0:\n # No second mention of antidepressant_a_name\n continue # try another A\n # We pick the first possible completion-of-course A second\n # mention:\n antidepressant_a_second_mention = a_second_mentions.iat[\n 0, sourcecolnum_date]\n # Make sure B is not mentioned within the A range\n mentions_of_b_within_a_range = tp[\n (tp[drug_colname] == antidepressant_b_name) &\n (tp[date_colname] >= antidepressant_a_first_mention) &\n\n # 2019-04-01: CHANGED FROM:\n # (tp[date_colname] <= antidepressant_a_second_mention)\n # TO:\n (tp[date_colname] < antidepressant_a_second_mention)\n ]\n if len(mentions_of_b_within_a_range) > 0:\n # Nope, chuck out this combination.\n continue # try another A\n found_valid_a = True\n break\n if not found_valid_a:\n continue # try another B\n\n # ---------------------------------------------------------------------\n # OK; here we have found a combination that we like.\n # Add it to the results.\n # ---------------------------------------------------------------------\n # https://stackoverflow.com/questions/19365513/how-to-add-an-extra-row-to-a-pandas-dataframe/19368360 # noqa\n # http://pandas.pydata.org/pandas-docs/stable/indexing.html#setting-with-enlargement # noqa\n\n expect_response_by_date = (\n antidepressant_b_first_mention + timedelta_days(\n expect_response_by_days)\n )\n end_of_symptom_period = (\n antidepressant_b_first_mention + timedelta_days(\n expect_response_by_days + symptom_assessment_time_days - 1)\n )\n result = _get_generic_two_antidep_episodes_result((\n patient_id,\n antidepressant_a_name,\n antidepressant_a_first_mention,\n antidepressant_a_second_mention,\n antidepressant_b_name,\n antidepressant_b_first_mention,\n antidepressant_b_second_mention,\n expect_response_by_date,\n end_of_symptom_period\n ))\n # We only care about the first episode per patient that matches, so:\n if first_episode_only:\n return result\n else:\n all_episodes_for_patient = all_episodes_for_patient.append(result)\n\n if len(all_episodes_for_patient) == 0:\n return None # nothing found\n return all_episodes_for_patient", "response": "Processes a single patient and returns a DataFrame containing the two - antidepressant episodes for that patient."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef two_antidepressant_episodes(\n patient_drug_date_df: DataFrame,\n patient_colname: str = DEFAULT_SOURCE_PATIENT_COLNAME,\n drug_colname: str = DEFAULT_SOURCE_DRUG_COLNAME,\n date_colname: str = DEFAULT_SOURCE_DATE_COLNAME,\n course_length_days: int = DEFAULT_ANTIDEPRESSANT_COURSE_LENGTH_DAYS,\n expect_response_by_days: int = DEFAULT_EXPECT_RESPONSE_BY_DAYS,\n symptom_assessment_time_days: int =\n DEFAULT_SYMPTOM_ASSESSMENT_TIME_DAYS,\n n_threads: int = DEFAULT_N_THREADS,\n first_episode_only: bool = True) -> DataFrame:\n \"\"\"\n Takes a *pandas* ``DataFrame``, ``patient_drug_date_df`` (or, via\n ``reticulate``, an R ``data.frame`` or ``data.table``). This should contain\n dated present-tense references to antidepressant drugs (only).\n\n Returns a set of result rows as a ``DataFrame``.\n \"\"\"\n # Say hello\n log.info(\"Running two_antidepressant_episodes...\")\n start = Pendulum.now()\n\n # Work through each patient\n patient_ids = sorted(list(set(patient_drug_date_df[patient_colname])))\n n_patients = len(patient_ids)\n log.info(\"Found {} patients\", n_patients)\n flush_stdout_stderr()\n\n def _get_patient_result(_patient_id: str) -> Optional[DataFrame]:\n return two_antidepressant_episodes_single_patient(\n patient_id=_patient_id,\n patient_drug_date_df=patient_drug_date_df,\n patient_colname=patient_colname,\n drug_colname=drug_colname,\n date_colname=date_colname,\n course_length_days=course_length_days,\n expect_response_by_days=expect_response_by_days,\n symptom_assessment_time_days=symptom_assessment_time_days,\n first_episode_only=first_episode_only,\n )\n\n combined_result = _get_blank_two_antidep_episodes_result()\n if n_threads > 1:\n # Farm off the work to lots of threads:\n log.info(\"Parallel processing method; {} threads\", n_threads)\n with ThreadPoolExecutor(max_workers=n_threads) as executor:\n list_of_results_frames = executor.map(_get_patient_result,\n patient_ids)\n\n log.debug(\"Recombining results from parallel processing...\")\n for patient_result in list_of_results_frames:\n if patient_result is not None:\n combined_result = combined_result.append(patient_result)\n log.debug(\"... recombined\")\n\n else:\n log.info(\"Single-thread method\")\n for ptnum, patient_id in enumerate(patient_ids, start=1):\n log.debug(\"Processing patient {} out of {}\", ptnum, n_patients)\n patient_result = _get_patient_result(patient_id)\n if patient_result is not None:\n combined_result = combined_result.append(patient_result)\n\n # For consistent results order, even with parallel processing:\n combined_result = combined_result.sort_values(\n by=[RCN_PATIENT_ID], ascending=True)\n # So that the DataFrame indices aren't all zero (largely cosmetic):\n combined_result.reset_index(inplace=True, drop=True)\n\n end = Pendulum.now()\n duration = end - start\n log.info(\"Took {} seconds for {} patients\",\n duration.total_seconds(), n_patients)\n flush_stdout_stderr()\n return combined_result", "response": "Takes a pandas DataFrame or via\n and returns a DataFrame containing the two - antidepressant drugs for each patient."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef does_unrtf_support_quiet() -> bool:\n required_unrtf_version = Version(\"0.21.9\")\n # ... probably: http://hg.savannah.gnu.org/hgweb/unrtf/\n # ... 0.21.9 definitely supports --quiet\n # ... 0.19.3 definitely doesn't support it\n unrtf_filename = shutil.which('unrtf')\n if not unrtf_filename:\n return False\n p = subprocess.Popen([\"unrtf\", \"--version\"],\n stdout=subprocess.PIPE,\n stderr=subprocess.PIPE)\n _, err_bytes = p.communicate()\n text = err_bytes.decode(sys.getdefaultencoding())\n lines = text.split()\n if len(lines) < 1:\n return False\n version_str = lines[0]\n unrtf_version = Version(version_str)\n return unrtf_version >= required_unrtf_version", "response": "Check if the unrtf tool supports the quiet argument from a version that I m not\n too sure of."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef get_filelikeobject(filename: str = None,\n blob: bytes = None) -> BinaryIO:\n \"\"\"\n Open a file-like object.\n\n Guard the use of this function with ``with``.\n\n Args:\n filename: for specifying via a filename\n blob: for specifying via an in-memory ``bytes`` object\n\n Returns:\n a :class:`BinaryIO` object\n \"\"\"\n if not filename and not blob:\n raise ValueError(\"no filename and no blob\")\n if filename and blob:\n raise ValueError(\"specify either filename or blob\")\n if filename:\n return open(filename, 'rb')\n else:\n return io.BytesIO(blob)", "response": "Returns a file - like object."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nreturn the contents of a file or of a BLOB.", "response": "def get_file_contents(filename: str = None, blob: bytes = None) -> bytes:\n \"\"\"\n Returns the binary contents of a file, or of a BLOB.\n \"\"\"\n if not filename and not blob:\n raise ValueError(\"no filename and no blob\")\n if filename and blob:\n raise ValueError(\"specify either filename or blob\")\n if blob:\n return blob\n with open(filename, 'rb') as f:\n return f.read()"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nguess the character set encoding of the specified binary contents.", "response": "def get_chardet_encoding(binary_contents: bytes) -> Optional[str]:\n \"\"\"\n Guess the character set encoding of the specified ``binary_contents``.\n \"\"\"\n if not binary_contents:\n return None\n if chardet is None or UniversalDetector is None:\n log.warning(\"chardet not installed; limits detection of encodings\")\n return None\n # METHOD 1\n # http://chardet.readthedocs.io/en/latest/\n #\n # guess = chardet.detect(binary_contents)\n #\n # METHOD 2: faster with large files\n # http://chardet.readthedocs.io/en/latest/\n # http://stackoverflow.com/questions/13857856/split-byte-string-into-lines\n\n # noinspection PyCallingNonCallable\n detector = UniversalDetector()\n for byte_line in binary_contents.split(b\"\\n\"):\n detector.feed(byte_line)\n if detector.done:\n break\n guess = detector.result\n # Handle result\n if 'encoding' not in guess:\n log.warning(\"Something went wrong within chardet; no encoding\")\n return None\n return guess['encoding']"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef get_file_contents_text(\n filename: str = None, blob: bytes = None,\n config: TextProcessingConfig = _DEFAULT_CONFIG) -> str:\n \"\"\"\n Returns the string contents of a file, or of a BLOB.\n \"\"\"\n binary_contents = get_file_contents(filename=filename, blob=blob)\n # 1. Try the encoding the user specified\n if config.encoding:\n try:\n return binary_contents.decode(config.encoding)\n except ValueError: # of which UnicodeDecodeError is more specific\n # ... https://docs.python.org/3/library/codecs.html\n pass\n # 2. Try the system encoding\n sysdef = sys.getdefaultencoding()\n if sysdef != config.encoding:\n try:\n return binary_contents.decode(sysdef)\n except ValueError:\n pass\n # 3. Try the best guess from chardet\n # http://chardet.readthedocs.io/en/latest/usage.html\n if chardet:\n guess = chardet.detect(binary_contents)\n if guess['encoding']:\n return binary_contents.decode(guess['encoding'])\n raise ValueError(\"Unknown encoding ({})\".format(\n \"filename={}\".format(repr(filename)) if filename else \"blob\"))", "response": "Returns the contents of a file or of a BLOB."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef get_cmd_output(*args, encoding: str = SYS_ENCODING) -> str:\n log.debug(\"get_cmd_output(): args = {!r}\", args)\n p = subprocess.Popen(args, stdout=subprocess.PIPE)\n stdout, stderr = p.communicate()\n return stdout.decode(encoding, errors='ignore')", "response": "Returns the output of a command."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nreturn text output of a command passing binary data in via stdin.", "response": "def get_cmd_output_from_stdin(stdint_content_binary: bytes,\n *args, encoding: str = SYS_ENCODING) -> str:\n \"\"\"\n Returns text output of a command, passing binary data in via stdin.\n \"\"\"\n p = subprocess.Popen(args, stdin=subprocess.PIPE, stdout=subprocess.PIPE)\n stdout, stderr = p.communicate(input=stdint_content_binary)\n return stdout.decode(encoding, errors='ignore')"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef convert_pdf_to_txt(filename: str = None, blob: bytes = None,\n config: TextProcessingConfig = _DEFAULT_CONFIG) -> str:\n \"\"\"\n Converts a PDF file to text.\n Pass either a filename or a binary object.\n \"\"\"\n pdftotext = tools['pdftotext']\n if pdftotext: # External command method\n if filename:\n return get_cmd_output(pdftotext, filename, '-')\n else:\n return get_cmd_output_from_stdin(blob, pdftotext, '-', '-')\n elif pdfminer: # Memory-hogging method\n with get_filelikeobject(filename, blob) as fp:\n rsrcmgr = pdfminer.pdfinterp.PDFResourceManager()\n retstr = StringIO()\n codec = ENCODING\n laparams = pdfminer.layout.LAParams()\n device = pdfminer.converter.TextConverter(\n rsrcmgr, retstr, codec=codec, laparams=laparams)\n interpreter = pdfminer.pdfinterp.PDFPageInterpreter(rsrcmgr,\n device)\n password = \"\"\n maxpages = 0\n caching = True\n pagenos = set()\n for page in pdfminer.pdfpage.PDFPage.get_pages(\n fp, pagenos, maxpages=maxpages, password=password,\n caching=caching, check_extractable=True):\n interpreter.process_page(page)\n text = retstr.getvalue().decode(ENCODING)\n return text\n else:\n raise AssertionError(\"No PDF-reading tool available\")", "response": "Converts a PDF file to text."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef availability_pdf() -> bool:\n pdftotext = tools['pdftotext']\n if pdftotext:\n return True\n elif pdfminer:\n log.warning(\"PDF conversion: pdftotext missing; \"\n \"using pdfminer (less efficient)\")\n return True\n else:\n return False", "response": "Returns True if there is a PDF - to - text tool available?"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef gen_xml_files_from_docx(fp: BinaryIO) -> Iterator[str]:\n try:\n z = zipfile.ZipFile(fp)\n filelist = z.namelist()\n for filename in filelist:\n if DOCX_HEADER_FILE_REGEX.match(filename):\n yield z.read(filename).decode(\"utf8\")\n yield z.read(DOCX_DOC_FILE)\n for filename in filelist:\n if DOCX_FOOTER_FILE_REGEX.match(filename):\n yield z.read(filename).decode(\"utf8\")\n except zipfile.BadZipFile:\n # Clarify the error:\n raise zipfile.BadZipFile(\"File is not a zip file - encrypted DOCX?\")", "response": "Generates XML files from a DOCX file."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nconverts an XML tree of a DOCX file to a string contents.", "response": "def docx_text_from_xml(xml: str, config: TextProcessingConfig) -> str:\n \"\"\"\n Converts an XML tree of a DOCX file to string contents.\n\n Args:\n xml: raw XML text\n config: :class:`TextProcessingConfig` control object\n\n Returns:\n contents as a string\n \"\"\"\n root = ElementTree.fromstring(xml)\n return docx_text_from_xml_node(root, 0, config)"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef docx_text_from_xml_node(node: ElementTree.Element,\n level: int,\n config: TextProcessingConfig) -> str:\n \"\"\"\n Returns text from an XML node within a DOCX file.\n\n Args:\n node: an XML node\n level: current level in XML hierarchy (used for recursion; start level\n is 0)\n config: :class:`TextProcessingConfig` control object\n\n Returns:\n contents as a string\n\n \"\"\"\n text = ''\n # log.debug(\"Level {}, tag {}\", level, node.tag)\n if node.tag == DOCX_TEXT:\n text += node.text or ''\n elif node.tag == DOCX_TAB:\n text += '\\t'\n elif node.tag in DOCX_NEWLINES:\n text += '\\n'\n elif node.tag == DOCX_NEWPARA:\n text += '\\n\\n'\n\n if node.tag == DOCX_TABLE:\n text += '\\n\\n' + docx_table_from_xml_node(node, level, config)\n else:\n for child in node:\n text += docx_text_from_xml_node(child, level + 1, config)\n return text", "response": "Returns a string containing the contents of an XML node within a DOCX file."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nconvert an XML node representing a DOCX table into a textual representation.", "response": "def docx_table_from_xml_node(table_node: ElementTree.Element,\n level: int,\n config: TextProcessingConfig) -> str:\n \"\"\"\n Converts an XML node representing a DOCX table into a textual\n representation.\n\n Args:\n table_node: XML node\n level: current level in XML hierarchy (used for recursion; start level\n is 0)\n config: :class:`TextProcessingConfig` control object\n\n Returns:\n string representation\n\n \"\"\"\n table = CustomDocxTable()\n for row_node in table_node:\n if row_node.tag != DOCX_TABLE_ROW:\n continue\n table.new_row()\n for cell_node in row_node:\n if cell_node.tag != DOCX_TABLE_CELL:\n continue\n table.new_cell()\n for para_node in cell_node:\n text = docx_text_from_xml_node(para_node, level, config)\n if text:\n table.add_paragraph(text)\n return docx_process_table(table, config)"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef docx_process_simple_text(text: str, width: int) -> str:\n if width:\n return '\\n'.join(textwrap.wrap(text, width=width))\n else:\n return text", "response": "Process text with word - wrapping."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef docx_process_table(table: DOCX_TABLE_TYPE,\n config: TextProcessingConfig) -> str:\n \"\"\"\n Converts a DOCX table to text.\n\n Structure representing a DOCX table:\n\n .. code-block:: none\n\n table\n .rows[]\n .cells[]\n .paragraphs[]\n .text\n\n That's the structure of a :class:`docx.table.Table` object, but also of our\n homebrew creation, :class:`CustomDocxTable`.\n\n The ``plain`` option optimizes for natural language processing, by:\n\n - removing vertical lines:\n\n .. code-block:: none\n\n +-------------+-------------+\n | AAA AAA | BBB BBB |\n | AAA AAA | BBB BBB |\n +-------------+-------------+\n\n becomes\n\n .. code-block:: none\n\n -----------------------------\n AAA AAA BBB BBB\n AAA AAA BBB BBB\n -----------------------------\n\n - and offsetting cells:\n\n .. code-block:: none\n\n AAA AAA BBB BBB CCC CCC\n AAA AAA BBB BBB CCC CCC\n\n becomes\n\n .. code-block:: none\n\n AAA AAA\n AAA AAA\n BBB BBB\n BBB BBB\n CCC CCC\n CCC CCC\n\n - Note also that the grids in DOCX files can have varying number of cells\n per row, e.g.\n\n .. code-block:: none\n \n +---+---+---+\n | 1 | 2 | 3 |\n +---+---+---+\n | 1 | 2 |\n +---+---+\n\n \"\"\"\n\n def get_cell_text(cell_) -> str:\n cellparagraphs = [paragraph.text.strip()\n for paragraph in cell_.paragraphs]\n cellparagraphs = [x for x in cellparagraphs if x]\n return '\\n\\n'.join(cellparagraphs)\n\n ncols = 1\n # noinspection PyTypeChecker\n for row in table.rows:\n ncols = max(ncols, len(row.cells))\n pt = prettytable.PrettyTable(\n field_names=list(range(ncols)),\n encoding=ENCODING,\n header=False,\n border=True,\n hrules=prettytable.ALL,\n vrules=prettytable.NONE if config.plain else prettytable.ALL,\n )\n pt.align = 'l'\n pt.valign = 't'\n pt.max_width = max(config.width // ncols, config.min_col_width)\n if config.plain:\n # noinspection PyTypeChecker\n for row in table.rows:\n for i, cell in enumerate(row.cells):\n n_before = i\n n_after = ncols - i - 1\n # ... use ncols, not len(row.cells), since \"cells per row\" is\n # not constant, but prettytable wants a fixed number.\n # (changed in v0.2.8)\n ptrow = (\n [''] * n_before +\n [get_cell_text(cell)] +\n [''] * n_after\n )\n assert(len(ptrow) == ncols)\n pt.add_row(ptrow)\n else:\n # noinspection PyTypeChecker\n for row in table.rows:\n ptrow = [] # type: List[str]\n # noinspection PyTypeChecker\n for cell in row.cells:\n ptrow.append(get_cell_text(cell))\n ptrow += [''] * (ncols - len(ptrow)) # added in v0.2.8\n assert (len(ptrow) == ncols)\n pt.add_row(ptrow)\n return pt.get_string()", "response": "This function processes a DOCX table and returns a list of text that can be used to create a new table."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\niterates through items of a DOCX file.", "response": "def docx_docx_iter_block_items(parent: DOCX_CONTAINER_TYPE) \\\n -> Iterator[DOCX_BLOCK_ITEM_TYPE]:\n # only called if docx loaded\n \"\"\"\n Iterate through items of a DOCX file.\n\n See https://github.com/python-openxml/python-docx/issues/40.\n\n Yield each paragraph and table child within ``parent``, in document order.\n Each returned value is an instance of either :class:`Table` or\n :class:`Paragraph`. ``parent`` would most commonly be a reference to a main\n :class:`Document` object, but also works for a :class:`_Cell` object, which\n itself can contain paragraphs and tables.\n\n NOTE: uses internals of the ``python-docx`` (``docx``) library; subject to\n change; this version works with ``docx==0.8.5``.\n \"\"\"\n if isinstance(parent, docx.document.Document):\n parent_elm = parent.element.body\n elif isinstance(parent, docx.table._Cell):\n parent_elm = parent._tc\n else:\n raise ValueError(\"something's not right\")\n\n for child in parent_elm.iterchildren():\n if isinstance(child, docx.oxml.text.paragraph.CT_P):\n yield docx.text.paragraph.Paragraph(child, parent)\n elif isinstance(child, docx.oxml.table.CT_Tbl):\n yield docx.table.Table(child, parent)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\niterate through a DOCX file and yield text. Args: doc: DOCX document to process config: :class:`TextProcessingConfig` control object Yields: pieces of text (paragraphs)", "response": "def docx_docx_gen_text(doc: DOCX_DOCUMENT_TYPE,\n config: TextProcessingConfig) -> Iterator[str]:\n # only called if docx loaded\n \"\"\"\n Iterate through a DOCX file and yield text.\n\n Args:\n doc: DOCX document to process\n config: :class:`TextProcessingConfig` control object\n\n Yields:\n pieces of text (paragraphs)\n\n \"\"\"\n if in_order:\n for thing in docx_docx_iter_block_items(doc):\n if isinstance(thing, docx.text.paragraph.Paragraph):\n yield docx_process_simple_text(thing.text, config.width)\n elif isinstance(thing, docx.table.Table):\n yield docx_process_table(thing, config)\n else:\n for paragraph in doc.paragraphs:\n yield docx_process_simple_text(paragraph.text, config.width)\n for table in doc.tables:\n yield docx_process_table(table, config)"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef convert_docx_to_text(\n filename: str = None, blob: bytes = None,\n config: TextProcessingConfig = _DEFAULT_CONFIG) -> str:\n \"\"\"\n Converts a DOCX file to text.\n Pass either a filename or a binary object.\n\n Args:\n filename: filename to process\n blob: binary ``bytes`` object to process\n config: :class:`TextProcessingConfig` control object\n\n Returns:\n text contents\n\n Notes:\n\n - Old ``docx`` (https://pypi.python.org/pypi/python-docx) has been\n superseded (see https://github.com/mikemaccana/python-docx).\n\n - ``docx.opendocx(file)`` uses :class:`zipfile.ZipFile`, which can take\n either a filename or a file-like object\n (https://docs.python.org/2/library/zipfile.html).\n\n - Method was:\n\n .. code-block:: python\n\n with get_filelikeobject(filename, blob) as fp:\n document = docx.opendocx(fp)\n paratextlist = docx.getdocumenttext(document)\n return '\\n\\n'.join(paratextlist)\n\n - Newer ``docx`` is python-docx\n\n - https://pypi.python.org/pypi/python-docx\n - https://python-docx.readthedocs.org/en/latest/\n - http://stackoverflow.com/questions/25228106\n\n However, it uses ``lxml``, which has C dependencies, so it doesn't always\n install properly on e.g. bare Windows machines.\n\n PERFORMANCE of my method:\n\n - nice table formatting\n - but tables grouped at end, not in sensible places\n - can iterate via ``doc.paragraphs`` and ``doc.tables`` but not in\n true document order, it seems\n - others have noted this too:\n\n - https://github.com/python-openxml/python-docx/issues/40\n - https://github.com/deanmalmgren/textract/pull/92\n\n - ``docx2txt`` is at https://pypi.python.org/pypi/docx2txt/0.6; this is\n pure Python. Its command-line function appears to be for Python 2 only\n (2016-04-21: crashes under Python 3; is due to an encoding bug). However,\n it seems fine as a library. It doesn't handle in-memory blobs properly,\n though, so we need to extend it.\n\n PERFORMANCE OF ITS ``process()`` function:\n\n - all text comes out\n - table text is in a sensible place\n - table formatting is lost.\n\n - Other manual methods (not yet implemented):\n http://etienned.github.io/posts/extract-text-from-word-docx-simply/.\n\n Looks like it won't deal with header stuff (etc.) that ``docx2txt``\n handles.\n\n - Upshot: we need a DIY version.\n\n - See also this \"compile lots of techniques\" libraries, which has C\n dependencies: http://textract.readthedocs.org/en/latest/\n\n \"\"\"\n\n if True:\n text = ''\n with get_filelikeobject(filename, blob) as fp:\n for xml in gen_xml_files_from_docx(fp):\n text += docx_text_from_xml(xml, config)\n return text", "response": "Converts a DOCX file to text."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nconvert an OpenOffice ODT file to text.", "response": "def convert_odt_to_text(filename: str = None,\n blob: bytes = None,\n config: TextProcessingConfig = _DEFAULT_CONFIG) -> str:\n \"\"\"\n Converts an OpenOffice ODT file to text.\n\n Pass either a filename or a binary object.\n \"\"\"\n # We can't use exactly the same method as for DOCX files, using docx:\n # sometimes that works, but sometimes it falls over with:\n # KeyError: \"There is no item named 'word/document.xml' in the archive\"\n with get_filelikeobject(filename, blob) as fp:\n z = zipfile.ZipFile(fp)\n tree = ElementTree.fromstring(z.read('content.xml'))\n # ... may raise zipfile.BadZipfile\n textlist = [] # type: List[str]\n for element in tree.iter():\n if element.text:\n textlist.append(element.text.strip())\n return '\\n\\n'.join(textlist)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nconverting HTML to text.", "response": "def convert_html_to_text(\n filename: str = None,\n blob: bytes = None,\n config: TextProcessingConfig = _DEFAULT_CONFIG) -> str:\n \"\"\"\n Converts HTML to text.\n \"\"\"\n with get_filelikeobject(filename, blob) as fp:\n soup = bs4.BeautifulSoup(fp)\n return soup.get_text()"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nconverts XML to text.", "response": "def convert_xml_to_text(filename: str = None,\n blob: bytes = None,\n config: TextProcessingConfig = _DEFAULT_CONFIG) -> str:\n \"\"\"\n Converts XML to text.\n \"\"\"\n with get_filelikeobject(filename, blob) as fp:\n soup = bs4.BeautifulStoneSoup(fp)\n return soup.get_text()"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nconverting RTF to text.", "response": "def convert_rtf_to_text(filename: str = None,\n blob: bytes = None,\n config: TextProcessingConfig = _DEFAULT_CONFIG) -> str:\n \"\"\"\n Converts RTF to text.\n \"\"\"\n unrtf = tools['unrtf']\n if unrtf: # Best\n args = [unrtf, '--text', '--nopict']\n if UNRTF_SUPPORTS_QUIET:\n args.append('--quiet')\n if filename:\n args.append(filename)\n return get_cmd_output(*args)\n else:\n return get_cmd_output_from_stdin(blob, *args)\n elif pyth: # Very memory-consuming:\n # https://github.com/brendonh/pyth/blob/master/pyth/plugins/rtf15/reader.py # noqa\n with get_filelikeobject(filename, blob) as fp:\n doc = pyth.plugins.rtf15.reader.Rtf15Reader.read(fp)\n return (\n pyth.plugins.plaintext.writer.PlaintextWriter.write(doc).getvalue()\n )\n else:\n raise AssertionError(\"No RTF-reading tool available\")"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\ncheck if an RTF processor is available.", "response": "def availability_rtf() -> bool:\n \"\"\"\n Is an RTF processor available?\n \"\"\"\n unrtf = tools['unrtf']\n if unrtf:\n return True\n elif pyth:\n log.warning(\"RTF conversion: unrtf missing; \"\n \"using pyth (less efficient)\")\n return True\n else:\n return False"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nconverts Microsoft Word DOC files to text.", "response": "def convert_doc_to_text(filename: str = None,\n blob: bytes = None,\n config: TextProcessingConfig = _DEFAULT_CONFIG) -> str:\n \"\"\"\n Converts Microsoft Word DOC files to text.\n \"\"\"\n antiword = tools['antiword']\n if antiword:\n if filename:\n return get_cmd_output(antiword, '-w', str(config.width), filename)\n else:\n return get_cmd_output_from_stdin(blob, antiword, '-w',\n str(config.width), '-')\n else:\n raise AssertionError(\"No DOC-reading tool available\")"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef convert_anything_to_text(\n filename: str = None,\n blob: bytes = None,\n config: TextProcessingConfig = _DEFAULT_CONFIG) -> str:\n \"\"\"\n Convert arbitrary files to text, using ``strings`` or ``strings2``.\n (``strings`` is a standard Unix command to get text from any old rubbish.)\n \"\"\"\n strings = tools['strings'] or tools['strings2']\n if strings:\n if filename:\n return get_cmd_output(strings, filename)\n else:\n return get_cmd_output_from_stdin(blob, strings)\n else:\n raise AssertionError(\"No fallback string-reading tool available\")", "response": "Convert arbitrary files to text using strings2."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef document_to_text(filename: str = None,\n blob: bytes = None,\n extension: str = None,\n config: TextProcessingConfig = _DEFAULT_CONFIG) -> str:\n \"\"\"\n Converts a document to text.\n\n This function selects a processor based on the file extension (either from\n the filename, or, in the case of a BLOB, the extension specified manually\n via the ``extension`` parameter).\n\n Pass either a filename or a binary object.\n\n - Raises an exception for malformed arguments, missing files, bad\n filetypes, etc.\n\n - Returns a string if the file was processed (potentially an empty string).\n \"\"\"\n if not filename and not blob:\n raise ValueError(\"document_to_text: no filename and no blob\")\n if filename and blob:\n raise ValueError(\"document_to_text: specify either filename or blob\")\n if blob and not extension:\n raise ValueError(\"document_to_text: need extension hint for blob\")\n if filename:\n stub, extension = os.path.splitext(filename)\n else:\n if extension[0] != \".\":\n extension = \".\" + extension\n extension = extension.lower()\n\n # Ensure blob is an appropriate type\n log.debug(\n \"filename: {}, blob type: {}, blob length: {}, extension: {}\".format(\n filename,\n type(blob),\n len(blob) if blob is not None else None,\n extension))\n\n # If we were given a filename and the file doesn't exist, don't bother.\n if filename and not os.path.isfile(filename):\n raise ValueError(\"document_to_text: no such file: {!r}\".format(\n filename))\n\n # Choose method\n info = ext_map.get(extension)\n if info is None:\n log.warning(\"Unknown filetype: {}; using generic tool\", extension)\n info = ext_map[None]\n func = info[CONVERTER]\n return func(filename, blob, config)", "response": "Converts a document to text."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef is_text_extractor_available(extension: str) -> bool:\n if extension is not None:\n extension = extension.lower()\n info = ext_map.get(extension)\n if info is None:\n return False\n availability = info[AVAILABILITY]\n if type(availability) == bool:\n return availability\n elif callable(availability):\n return availability()\n else:\n raise ValueError(\n \"Bad information object for extension: {}\".format(extension))", "response": "Returns True if a text extractor is available for the specified extension."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\ncommand - line processor.", "response": "def main() -> None:\n \"\"\"\n Command-line processor. See ``--help`` for details.\n \"\"\"\n logging.basicConfig(level=logging.DEBUG)\n parser = argparse.ArgumentParser()\n parser.add_argument(\"inputfile\", nargs=\"?\", help=\"Input file name\")\n parser.add_argument(\n \"--availability\", nargs='*',\n help=\"File extensions to check availability for (use a '.' prefix, \"\n \"and use the special extension 'None' to check the fallback \"\n \"processor\")\n parser.add_argument(\n \"--plain\", action=\"store_true\",\n help=\"Keep it plain: minimize layout (e.g. of tables) and maximize \"\n \"the likelihood of source sentence flowing properly in the \"\n \"output (e.g. for natural language processing).\")\n parser.add_argument(\n \"--width\", type=int, default=DEFAULT_WIDTH,\n help=\"Word wrapping width (default {})\".format(DEFAULT_WIDTH))\n parser.add_argument(\n \"--min-col-width\", type=int, default=DEFAULT_MIN_COL_WIDTH,\n help=\"Minimum column width for tables (default {})\".format(\n DEFAULT_MIN_COL_WIDTH))\n args = parser.parse_args()\n if args.availability:\n for ext in args.availability:\n if ext.lower() == 'none':\n ext = None\n available = is_text_extractor_available(ext)\n print(\"Extractor for extension {} present: {}\".format(ext,\n available))\n return\n if not args.inputfile:\n parser.print_help(sys.stderr)\n return\n config = TextProcessingConfig(\n width=args.width,\n min_col_width=args.min_col_width,\n plain=args.plain,\n )\n result = document_to_text(filename=args.inputfile, config=config)\n if result is None:\n return\n else:\n print(result)"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef set_verbose_logging(verbose: bool) -> None:\n if verbose:\n set_loglevel(logging.DEBUG)\n else:\n set_loglevel(logging.INFO)", "response": "Chooses basic or verbose logging."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef debug_sql(sql: str, *args: Any) -> None:\n log.debug(\"SQL: %s\" % sql)\n if args:\n log.debug(\"Args: %r\" % args)", "response": "Writes SQL and arguments to the log."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef delimit(x: str, delims: Tuple[str, str]) -> str:\n return delims[0] + x + delims[1]", "response": "Delimits x using delims [ 0 1 ) and [ right ]"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef get_sql_select_all_fields_by_key(\n table: str,\n fieldlist: Sequence[str],\n keyname: str,\n delims: Tuple[str, str] = (\"\", \"\")) -> str:\n \"\"\"Returns SQL:\n SELECT [all fields in the fieldlist] WHERE [keyname] = ?\n \"\"\"\n return (\n \"SELECT \" +\n \",\".join([delimit(x, delims) for x in fieldlist]) +\n \" FROM \" + delimit(table, delims) +\n \" WHERE \" + delimit(keyname, delims) + \"=?\"\n )", "response": "Returns SQL SELECT all fields in the fieldlist where keyname = keyname."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nreturn an INSERT statement.", "response": "def get_sql_insert(table: str,\n fieldlist: Sequence[str],\n delims: Tuple[str, str] = (\"\", \"\")) -> str:\n \"\"\"Returns ?-marked SQL for an INSERT statement.\"\"\"\n return (\n \"INSERT INTO \" + delimit(table, delims) +\n \" (\" +\n \",\".join([delimit(x, delims) for x in fieldlist]) +\n \") VALUES (\" +\n \",\".join([\"?\"] * len(fieldlist)) +\n \")\"\n )"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nreturn an INSERT - or - UPDATE statement.", "response": "def get_sql_insert_or_update(table: str,\n fieldlist: Sequence[str],\n delims: Tuple[str, str] = (\"\", \"\")) -> str:\n \"\"\"Returns ?-marked SQL for an INSERT-or-if-duplicate-key-UPDATE statement.\n \"\"\"\n # http://stackoverflow.com/questions/4205181\n return \"\"\"\n INSERT INTO {table} ({fields})\n VALUES ({placeholders})\n ON DUPLICATE KEY UPDATE {updatelist}\n \"\"\".format(\n table=delimit(table, delims),\n fields=\",\".join([delimit(x, delims) for x in fieldlist]),\n placeholders=\",\".join([\"?\"] * len(fieldlist)),\n updatelist=\",\".join(\n [\"{field}=VALUES({field})\".format(field=delimit(x, delims))\n for x in fieldlist]\n ),\n )"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef get_sql_insert_without_first_field(\n table: str,\n fieldlist: Sequence[str],\n delims: Tuple[str, str] = (\"\", \"\")) -> str:\n \"\"\"Returns ?-marked SQL for an INSERT statement, ignoring the first field\n (typically, the PK).\"\"\"\n return get_sql_insert(table, fieldlist[1:], delims)", "response": "Returns an INSERT statement ignoring the first field in the fieldlist."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nreturns an UPDATE statement to update all fields except the first field.", "response": "def get_sql_update_by_first_field(table: str,\n fieldlist: Sequence[str],\n delims: Tuple[str, str] = (\"\", \"\")) -> str:\n \"\"\"Returns SQL for an UPDATE statement, to update all fields except the\n first field (PK) using the PK as the key.\"\"\"\n return (\n \"UPDATE \" + delimit(table, delims) +\n \" SET \" +\n \",\".join([delimit(x, delims) + \"=?\" for x in fieldlist[1:]]) +\n \" WHERE \" + delimit(fieldlist[0], delims) + \"=?\"\n )"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nformat a DateTime object as something MySQL will actually accept.", "response": "def datetime2literal_rnc(d: datetime.datetime, c: Optional[Dict]) -> str:\n \"\"\"Format a DateTime object as something MySQL will actually accept.\"\"\"\n # dt = d.strftime(\"%Y-%m-%d %H:%M:%S\")\n # ... can fail with e.g.\n # ValueError: year=1850 is before 1900; the datetime strftime() methods\n # require year >= 1900\n # http://stackoverflow.com/questions/10263956\n dt = d.isoformat(\" \")\n # noinspection PyArgumentList\n return _mysql.string_literal(dt, c)"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nconvert a full datatype e. g. INT VARCHAR NVARCHAR NVARBINARY or NVARCHAR to a MySQL equivalent.", "response": "def full_datatype_to_mysql(d: str) -> str:\n \"\"\"Converts a full datatype, e.g. INT, VARCHAR(10), VARCHAR(MAX), to a\n MySQL equivalent.\"\"\"\n d = d.upper()\n (s, length) = split_long_sqltype(d)\n if d in [\"VARCHAR(MAX)\", \"NVARCHAR(MAX)\"]:\n # http://wiki.ispirer.com/sqlways/mysql/data-types/longtext\n return \"LONGTEXT\"\n elif d in [\"VARBINARY(MAX)\"] or s in [\"IMAGE\"]:\n # http://wiki.ispirer.com/sqlways/mysql/data-types/varbinary\n return \"LONGBLOB\"\n else:\n return d"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef debug_object(obj: T) -> str:\n pairs = []\n for k, v in vars(obj).items():\n pairs.append(u\"{}={}\".format(k, v))\n return u\", \".join(pairs)", "response": "Prints key - value pairs for an object s dictionary."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef dump_database_object(obj: T, fieldlist: Iterable[str]) -> None:\n log.info(_LINE_EQUALS)\n log.info(\"DUMP OF: {}\", obj)\n for f in fieldlist:\n log.info(u\"{f}: {v}\", f=f, v=getattr(obj, f))\n log.info(_LINE_EQUALS)", "response": "Prints key - value pairs for an object s dictionary."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef assign_from_list(obj: T,\n fieldlist: Sequence[str],\n valuelist: Sequence[any]) -> None:\n \"\"\"Within \"obj\", assigns the values from the value list to the fields in\n the fieldlist.\"\"\"\n if len(fieldlist) != len(valuelist):\n raise AssertionError(\"assign_from_list: fieldlist and valuelist of \"\n \"different length\")\n for i in range(len(valuelist)):\n setattr(obj, fieldlist[i], valuelist[i])", "response": "Within obj assigns the values from the value list to the fields in\n the fieldlist."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef create_object_from_list(cls: Type,\n fieldlist: Sequence[str],\n valuelist: Sequence[Any],\n *args, **kwargs) -> T:\n \"\"\"\n Create an object by instantiating ``cls(*args, **kwargs)`` and assigning the\n values in ``valuelist`` to the fields in ``fieldlist``.\n\n If ``construct_with_pk`` is ``True``, initialize with\n ``cls(valuelist[0], *args, **kwargs)``\n and assign the values in ``valuelist[1:]`` to ``fieldlist[1:]``.\n\n Note: in Python 3, we could define as\n\n .. code-block:: none\n\n ...(... valuelist, *args, construct_with_pk=False, **kwargs):\n\n but not in Python 2, and this is meant to be back-compatible.\n \"\"\"\n construct_with_pk = kwargs.pop('construct_with_pk', False)\n # print(\"construct_with_pk: {}\".format(construct_with_pk))\n # print(\"args: {}\".format(args))\n # print(\"kwargs: {}\".format(kwargs))\n if construct_with_pk:\n obj = cls(valuelist[0], *args, **kwargs)\n assign_from_list(obj, fieldlist[1:], valuelist[1:])\n else:\n obj = cls(*args, **kwargs)\n assign_from_list(obj, fieldlist, valuelist)\n return obj", "response": "Create an object from a list of values."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef blank_object(obj: T, fieldlist: Sequence[str]) -> None:\n for f in fieldlist:\n setattr(obj, f, None)", "response": "Within obj sets all fields in the fieldlist to None."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef debug_query_result(rows: Sequence[Any]) -> None:\n log.info(\"Retrieved {} rows\", len(rows))\n for i in range(len(rows)):\n log.info(\"Row {}: {}\", i, rows[i])", "response": "Writes a query result to the log."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nconnect via PyMySQL and creates a database.", "response": "def create_database_mysql(database: str,\n user: str,\n password: str,\n server: str = \"localhost\",\n port: int = 3306,\n charset: str = \"utf8\",\n collate: str = \"utf8_general_ci\",\n use_unicode: bool = True) -> bool:\n \"\"\"Connects via PyMySQL/MySQLdb and creates a database.\"\"\"\n con = mysql.connect(\n host=server,\n port=port,\n user=user,\n passwd=password,\n charset=charset,\n use_unicode=use_unicode\n )\n sql = (\"CREATE DATABASE IF NOT EXISTS {} DEFAULT CHARACTER SET {} \"\n \"DEFAULT COLLATE {}\").format(\n database,\n charset,\n collate\n )\n cursor = con.cursor()\n debug_sql(sql)\n cursor.execute(sql)\n log.info(\"Created database {}\", database)\n return True"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nconnects via PyMySQL and creates a master user.", "response": "def add_master_user_mysql(database: str,\n root_user: str,\n root_password: str,\n new_user: str,\n new_password: str,\n server: str = \"localhost\",\n port: int = 3306,\n charset: str = \"utf8\",\n use_unicode: bool = True,\n localhost_only: bool = True) -> None:\n \"\"\"Connects via PyMySQL/MySQLdb and creates a database superuser.\"\"\"\n con = mysql.connect(\n host=server,\n port=port,\n user=root_user,\n passwd=root_password,\n charset=charset,\n use_unicode=use_unicode\n )\n wherefrom = \"localhost\" if localhost_only else \"%\"\n sql = (\"GRANT ALL PRIVILEGES ON {}.* TO '{}'@'{}' \"\n \"IDENTIFIED BY '{}'\").format(\n database,\n new_user,\n wherefrom,\n new_password\n )\n cursor = con.cursor()\n debug_sql(sql)\n cursor.execute(sql)\n log.info(\"Added master user {} to database {}\", new_user, database)"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef describe_table(cls,\n db: DATABASE_SUPPORTER_FWD_REF,\n table: str) -> List[List[Any]]:\n \"\"\"Returns details on a specific table.\"\"\"\n raise RuntimeError(_MSG_NO_FLAVOUR)", "response": "Returns details on a specific table."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef fetch_column_names(cls,\n db: DATABASE_SUPPORTER_FWD_REF,\n table: str) -> List[str]:\n \"\"\"Returns all column names for a table.\"\"\"\n raise RuntimeError(_MSG_NO_FLAVOUR)", "response": "Returns all column names for a table."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef get_datatype(cls,\n db: DATABASE_SUPPORTER_FWD_REF,\n table: str,\n column: str) -> str:\n \"\"\"Returns database SQL datatype for a column: e.g. VARCHAR.\"\"\"\n raise RuntimeError(_MSG_NO_FLAVOUR)", "response": "Returns the SQL datatype for a column."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nchecking if the database has read - only access.", "response": "def is_read_only(cls,\n db: DATABASE_SUPPORTER_FWD_REF,\n logger: logging.Logger = None) -> bool:\n \"\"\"Do we have read-only access?\"\"\"\n\n def convert_enums(row_):\n # All these columns are of type enum('N', 'Y');\n # https://dev.mysql.com/doc/refman/5.0/en/enum.html\n return [True if x == 'Y' else (False if x == 'N' else None)\n for x in row_]\n\n # 1. Check per-database privileges.\n # We don't check SELECT privileges. We're just trying to ensure\n # nothing dangerous is present - for ANY database.\n # If we get an exception\n try:\n sql = \"\"\"\n SELECT db,\n /* must not have: */\n Insert_priv, Update_priv, Delete_priv,\n Create_priv, Drop_priv, Index_priv, Alter_priv,\n Lock_tables_priv, Create_view_priv,\n Create_routine_priv, Alter_routine_priv,\n Execute_priv, Event_priv, Trigger_priv\n FROM mysql.db\n WHERE\n CONCAT(user, '@', host) = CURRENT_USER()\n \"\"\"\n rows = db.fetchall(sql)\n for row in rows:\n dbname = row[0]\n prohibited = convert_enums(row[1:])\n if any(prohibited):\n if logger:\n logger.debug(\n \"MySQL.is_read_only(): FAIL: database privileges \"\n \"wrong: dbname={}, prohibited={}\".format(\n dbname, prohibited\n )\n )\n return False\n except mysql.OperationalError:\n # Probably: error 1142, \"SELECT command denied to user 'xxx'@'yyy'\n # for table 'db'\". This would be OK.\n pass\n\n # 2. Global privileges, e.g. as held by root\n try:\n sql = \"\"\"\n SELECT /* must not have: */\n Insert_priv, Update_priv, Delete_priv,\n Create_priv, Drop_priv,\n Reload_priv, Shutdown_priv,\n Process_priv, File_priv, Grant_priv,\n Index_priv, Alter_priv,\n Show_db_priv, Super_priv,\n Lock_tables_priv, Execute_priv,\n Repl_slave_priv, Repl_client_priv,\n Create_view_priv,\n Create_routine_priv, Alter_routine_priv,\n Create_user_priv,\n Event_priv, Trigger_priv,\n Create_tablespace_priv\n FROM mysql.user\n WHERE\n CONCAT(user, '@', host) = CURRENT_USER()\n \"\"\"\n rows = db.fetchall(sql)\n if not rows or len(rows) > 1:\n return False\n prohibited = convert_enums(rows[0])\n if any(prohibited):\n if logger:\n logger.debug(\n \"MySQL.is_read_only(): FAIL: GLOBAL privileges \"\n \"wrong: prohibited={}\".format(prohibited))\n return False\n except mysql.OperationalError:\n # Probably: error 1142, \"SELECT command denied to user 'xxx'@'yyy'\n # for table 'user'\". This would be OK.\n pass\n\n return True"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef connect(self,\n engine: str = None,\n interface: str = None,\n host: str = None,\n port: int = None,\n database: str = None,\n driver: str = None,\n dsn: str = None,\n odbc_connection_string: str = None,\n user: str = None,\n password: str = None,\n autocommit: bool = True,\n charset: str = \"utf8\",\n use_unicode: bool = True) -> bool:\n \"\"\"\n engine: access, mysql, sqlserver\n interface: mysql, odbc, jdbc\n \"\"\"\n\n # Catch all exceptions, so the error-catcher never shows a password.\n # Note also that higher-level things may catch exceptions, so use the\n # logger as well.\n try:\n return self._connect(\n engine=engine, interface=interface,\n host=host, port=port, database=database,\n driver=driver, dsn=dsn,\n odbc_connection_string=odbc_connection_string,\n user=user, password=password,\n autocommit=autocommit, charset=charset,\n use_unicode=use_unicode)\n except Exception as e:\n self.reraise_connection_exception(e)", "response": "Connect to the database and return a boolean indicating if the connection was successful."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nping a MySQL database connection reconnecting if necessary.", "response": "def ping(self) -> None:\n \"\"\"Pings a database connection, reconnecting if necessary.\"\"\"\n if self.db is None or self.db_pythonlib not in [PYTHONLIB_MYSQLDB,\n PYTHONLIB_PYMYSQL]:\n return\n try:\n self.db.ping(True) # test connection; reconnect upon failure\n # ... should auto-reconnect; however, it seems to fail the first\n # time, then work the next time.\n # Exception (the first time) is:\n # :\n # (2006, 'MySQL server has gone away')\n # http://mail.python.org/pipermail/python-list/2008-February/\n # 474598.html\n except mysql.OperationalError: # loss of connection\n self.db = None\n self.connect_to_database_mysql(\n self._database, self._user, self._password, self._server,\n self._port, self._charset, self._use_unicode)"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nconnect to a MySQL database via ODBC.", "response": "def connect_to_database_odbc_mysql(self,\n database: str,\n user: str,\n password: str,\n server: str = \"localhost\",\n port: int = 3306,\n driver: str = \"{MySQL ODBC 5.1 Driver}\",\n autocommit: bool = True) -> None:\n \"\"\"Connects to a MySQL database via ODBC.\"\"\"\n self.connect(engine=ENGINE_MYSQL, interface=INTERFACE_ODBC,\n database=database, user=user, password=password,\n host=server, port=port, driver=driver,\n autocommit=autocommit)"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef connect_to_database_odbc_sqlserver(self,\n odbc_connection_string: str = None,\n dsn: str = None,\n database: str = None,\n user: str = None,\n password: str = None,\n server: str = \"localhost\",\n driver: str = \"{SQL Server}\",\n autocommit: bool = True) -> None:\n \"\"\"Connects to an SQL Server database via ODBC.\"\"\"\n self.connect(engine=ENGINE_SQLSERVER, interface=INTERFACE_ODBC,\n odbc_connection_string=odbc_connection_string,\n dsn=dsn,\n database=database, user=user, password=password,\n host=server, driver=driver,\n autocommit=autocommit)", "response": "Connects to an SQL Server database via ODBC."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef connect_to_database_odbc_access(self,\n dsn: str,\n autocommit: bool = True) -> None:\n \"\"\"Connects to an Access database via ODBC, with the DSN\n prespecified.\"\"\"\n self.connect(engine=ENGINE_ACCESS, interface=INTERFACE_ODBC,\n dsn=dsn, autocommit=autocommit)", "response": "Connects to an Access database via ODBC with the DSN\n prespecified."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\ntranslates ?-placeholder SQL to appropriate dialect. For example, MySQLdb uses %s rather than ?.", "response": "def localize_sql(self, sql: str) -> str:\n \"\"\"Translates ?-placeholder SQL to appropriate dialect.\n\n For example, MySQLdb uses %s rather than ?.\n \"\"\"\n # pyodbc seems happy with ? now (pyodbc.paramstyle is 'qmark');\n # using ? is much simpler, because we may want to use % with LIKE\n # fields or (in my case) with date formatting strings for\n # STR_TO_DATE().\n # If you get this wrong, you may see \"not all arguments converted\n # during string formatting\";\n # http://stackoverflow.com/questions/9337134\n if self.db_pythonlib in [PYTHONLIB_PYMYSQL, PYTHONLIB_MYSQLDB]:\n # These engines use %, so we need to convert ? to %, without\n # breaking literal % values.\n sql = _PERCENT_REGEX.sub(\"%%\", sql)\n # ... replace all % with %% first\n sql = _QUERY_VALUE_REGEX.sub(\"%s\", sql)\n # ... replace all ? with %s in the SQL\n # Otherwise: engine uses ?, so we don't have to fiddle.\n return sql"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef insert_record(self,\n table: str,\n fields: Sequence[str],\n values: Sequence[Any],\n update_on_duplicate_key: bool = False) -> int:\n \"\"\"Inserts a record into database, table \"table\", using the list of\n fieldnames and the list of values. Returns the new PK (or None).\"\"\"\n self.ensure_db_open()\n if len(fields) != len(values):\n raise AssertionError(\"Field/value mismatch\")\n if update_on_duplicate_key:\n sql = get_sql_insert_or_update(table, fields, self.get_delims())\n else:\n sql = get_sql_insert(table, fields, self.get_delims())\n sql = self.localize_sql(sql)\n log.debug(\"About to insert_record with SQL template: \" + sql)\n try:\n cursor = self.db.cursor()\n debug_sql(sql, values)\n cursor.execute(sql, values)\n # ... binds the placeholders (?, %s) to values in the process\n new_pk = get_pk_of_last_insert(cursor)\n log.debug(\"Record inserted.\")\n return new_pk\n except: # nopep8\n log.exception(\"insert_record: Failed to insert record.\")\n raise", "response": "Inserts a record into the database table using the list of fieldnames and the list of values. Returns the new PK or None."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef insert_record_by_fieldspecs_with_values(\n self,\n table: str,\n fieldspeclist: FIELDSPECLIST_TYPE) -> int:\n \"\"\"Inserts a record into the database using a list of fieldspecs having\n their value stored under the 'value' key.\n \"\"\"\n fields = []\n values = []\n for fs in fieldspeclist:\n fields.append(fs[\"name\"])\n values.append(fs[\"value\"])\n return self.insert_record(table, fields, values)", "response": "Inserts a record into the database using a list of fieldspecs having\n their value stored under the value key."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef insert_record_by_dict(self,\n table: str,\n valuedict: Dict[str, Any]) -> Optional[int]:\n \"\"\"Inserts a record into database, table \"table\", using a dictionary\n containing field/value mappings. Returns the new PK (or None).\"\"\"\n if not valuedict:\n return None\n n = len(valuedict)\n fields = []\n args = []\n for f, v in valuedict.items():\n fields.append(self.delimit(f))\n args.append(v)\n query = \"\"\"\n INSERT INTO {table}\n ({fields})\n VALUES ({placeholders})\n \"\"\".format(\n table=table,\n fields=\",\".join(fields),\n placeholders=\",\".join([\"?\"]*n)\n )\n query = self.localize_sql(query)\n log.debug(\"About to insert_record_by_dict with SQL template: \" + query)\n try:\n cursor = self.db.cursor()\n debug_sql(query, args)\n cursor.execute(query, args)\n new_pk = get_pk_of_last_insert(cursor)\n log.debug(\"Record inserted.\")\n return new_pk\n except: # nopep8\n log.exception(\"insert_record_by_dict: Failed to insert record.\")\n raise", "response": "Inserts a record into the database table using a dictionary containing field and value mappings. Returns the new PK or None."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\ninsert a record into the database table using the list of AttributeNames fieldnames and the list of records. Returns the number of rows affected.", "response": "def insert_multiple_records(self,\n table: str,\n fields: Sequence[str],\n records: Sequence[Sequence[Any]]) -> int:\n \"\"\"Inserts a record into database, table \"table\", using the list of\n fieldnames and the list of records (each a list of values).\n Returns number of rows affected.\"\"\"\n self.ensure_db_open()\n sql = self.localize_sql(get_sql_insert(table, fields,\n self.get_delims()))\n log.debug(\"About to insert multiple records with SQL template: \" + sql)\n try:\n cursor = self.db.cursor()\n debug_sql(sql, records)\n cursor.executemany(sql, records)\n # ... binds the placeholders (?, %s) to values in the process\n # http://www.python.org/dev/peps/pep-0249/\n log.debug(\"Records inserted.\")\n return cursor.rowcount\n except: # nopep8\n log.exception(\"insert_multiple_records: Failed to insert records.\")\n raise"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nexecutes SQL on a supplied cursor. Returns number of rows affected.", "response": "def db_exec_with_cursor(self, cursor, sql: str, *args) -> int:\n \"\"\"Executes SQL on a supplied cursor, with \"?\" placeholders,\n substituting in the arguments. Returns number of rows affected.\"\"\"\n sql = self.localize_sql(sql)\n try:\n debug_sql(sql, args)\n cursor.execute(sql, args)\n return cursor.rowcount\n except: # nopep8\n log.exception(\"db_exec_with_cursor: SQL was: \" + sql)\n raise"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef db_exec(self, sql: str, *args) -> int:\n self.ensure_db_open()\n cursor = self.db.cursor()\n return self.db_exec_with_cursor(cursor, sql, *args)", "response": "Executes SQL with? placeholders for arguments for arguments."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef db_exec_and_commit(self, sql: str, *args) -> int:\n rowcount = self.db_exec(sql, *args)\n self.commit()\n return rowcount", "response": "Execute SQL and commit."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nexecutes SQL without modification. Returns rowcount.", "response": "def db_exec_literal(self, sql: str) -> int:\n \"\"\"Executes SQL without modification. Returns rowcount.\"\"\"\n self.ensure_db_open()\n cursor = self.db.cursor()\n debug_sql(sql)\n try:\n cursor.execute(sql)\n return cursor.rowcount\n except: # nopep8\n log.exception(\"db_exec_literal: SQL was: \" + sql)\n raise"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef fetchvalue(self, sql: str, *args) -> Optional[Any]:\n row = self.fetchone(sql, *args)\n if row is None:\n return None\n return row[0]", "response": "Executes SQL ; returns the first value of the first row or None."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nexecutes SQL ; returns the first row or None.", "response": "def fetchone(self, sql: str, *args) -> Optional[Sequence[Any]]:\n \"\"\"Executes SQL; returns the first row, or None.\"\"\"\n self.ensure_db_open()\n cursor = self.db.cursor()\n self.db_exec_with_cursor(cursor, sql, *args)\n try:\n return cursor.fetchone()\n except: # nopep8\n log.exception(\"fetchone: SQL was: \" + sql)\n raise"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef fetchall(self, sql: str, *args) -> Sequence[Sequence[Any]]:\n self.ensure_db_open()\n cursor = self.db.cursor()\n self.db_exec_with_cursor(cursor, sql, *args)\n try:\n rows = cursor.fetchall()\n return rows\n except: # nopep8\n log.exception(\"fetchall: SQL was: \" + sql)\n raise", "response": "Executes SQL ; returns all rows or []."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef gen_fetchall(self, sql: str, *args) -> Iterator[Sequence[Any]]:\n self.ensure_db_open()\n cursor = self.db.cursor()\n self.db_exec_with_cursor(cursor, sql, *args)\n try:\n row = cursor.fetchone()\n while row is not None:\n yield row\n row = cursor.fetchone()\n except: # nopep8\n log.exception(\"gen_fetchall: SQL was: \" + sql)\n raise", "response": "fetchall - Executes a SQL query and returns an iterator of the result set rows."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nexecutes SQL ; returns rows and fieldnames.", "response": "def fetchall_with_fieldnames(self, sql: str, *args) \\\n -> Tuple[Sequence[Sequence[Any]], Sequence[str]]:\n \"\"\"Executes SQL; returns (rows, fieldnames).\"\"\"\n self.ensure_db_open()\n cursor = self.db.cursor()\n self.db_exec_with_cursor(cursor, sql, *args)\n try:\n rows = cursor.fetchall()\n fieldnames = [i[0] for i in cursor.description]\n return rows, fieldnames\n except: # nopep8\n log.exception(\"fetchall_with_fieldnames: SQL was: \" + sql)\n raise"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nexecute SQL ; returns list of dictionaries where each dict contains fieldname / value pairs.", "response": "def fetchall_as_dictlist(self, sql: str, *args) -> List[Dict[str, Any]]:\n \"\"\"Executes SQL; returns list of dictionaries, where each dict contains\n fieldname/value pairs.\"\"\"\n self.ensure_db_open()\n cursor = self.db.cursor()\n self.db_exec_with_cursor(cursor, sql, *args)\n try:\n rows = cursor.fetchall()\n fieldnames = [i[0] for i in cursor.description]\n dictlist = []\n for r in rows:\n dictlist.append(dict(zip(fieldnames, r)))\n return dictlist\n except: # nopep8\n log.exception(\"fetchall_as_dictlist: SQL was: \" + sql)\n raise"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nexecute SQL ; returns list of first values of each row.", "response": "def fetchallfirstvalues(self, sql: str, *args) -> List[Any]:\n \"\"\"Executes SQL; returns list of first values of each row.\"\"\"\n rows = self.fetchall(sql, *args)\n return [row[0] for row in rows]"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef fetch_fieldnames(self, sql: str, *args) -> List[str]:\n self.ensure_db_open()\n cursor = self.db.cursor()\n self.db_exec_with_cursor(cursor, sql, *args)\n try:\n return [i[0] for i in cursor.description]\n except: # nopep8\n log.exception(\"fetch_fieldnames: SQL was: \" + sql)\n raise", "response": "Executes SQL ; returns just the output fieldnames."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\ncount rows in a table given a set of WHERE criteria and returns a count.", "response": "def count_where(self, table: str, wheredict: Dict[str, Any] = None) -> int:\n \"\"\"Counts rows in a table, given a set of WHERE criteria (ANDed),\n returning a count.\"\"\"\n sql = \"SELECT COUNT(*) FROM \" + self.delimit(table)\n if wheredict is not None:\n sql += \" WHERE \" + \" AND \".join([\n self.delimit(k) + \"=?\"\n for k in wheredict.keys()\n ])\n whereargs = wheredict.values()\n count = self.fetchone(sql, *whereargs)[0]\n else:\n count = self.fetchone(sql)[0]\n return count"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nchecking for the existence of a record by a single field.", "response": "def does_row_exist(self, table: str, field: str, value: Any) -> bool:\n \"\"\"Checks for the existence of a record by a single field (typically a\n PK).\"\"\"\n sql = (\"SELECT COUNT(*) FROM \" + self.delimit(table) +\n \" WHERE \" + self.delimit(field) + \"=?\")\n row = self.fetchone(sql, value)\n return True if row[0] >= 1 else False"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\ndeleting all records where field is value.", "response": "def delete_by_field(self, table: str, field: str, value: Any) -> int:\n \"\"\"Deletes all records where \"field\" is \"value\".\"\"\"\n sql = (\"DELETE FROM \" + self.delimit(table) +\n \" WHERE \" + self.delimit(field) + \"=?\")\n return self.db_exec(sql, value)"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef fetch_object_from_db_by_other_field(self,\n obj: Any,\n table: str,\n fieldlist: Sequence[str],\n keyname: str,\n keyvalue: Any) -> bool:\n \"\"\"Fetches object from database table by a field specified by\n keyname/keyvalue. Writes back to object. Returns True/False for\n success/failure.\"\"\"\n row = self.fetchone(\n get_sql_select_all_fields_by_key(table, fieldlist, keyname,\n self.get_delims()),\n keyvalue\n )\n if row is None:\n blank_object(obj, fieldlist)\n return False\n assign_from_list(obj, fieldlist, row)\n return True", "response": "Fetches object from database table by a field specified by keyname and keyvalue. Writes back to object. Returns True if success or False if failure."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nfetches all objects from a table returning an array of objects of class cls.", "response": "def fetch_all_objects_from_db(self,\n cls: Type[T],\n table: str,\n fieldlist: Sequence[str],\n construct_with_pk: bool,\n *args) -> List[T]:\n \"\"\"Fetches all objects from a table, returning an array of objects of\n class cls.\"\"\"\n return self.fetch_all_objects_from_db_where(\n cls, table, fieldlist, construct_with_pk, None, *args)"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef fetch_all_objects_from_db_by_pklist(self,\n cls: Type,\n table: str,\n fieldlist: Sequence[str],\n pklist: Sequence[Any],\n construct_with_pk: bool,\n *args) -> List[T]:\n \"\"\"Fetches all objects from a table, given a list of PKs.\"\"\"\n objarray = []\n for pk in pklist:\n if construct_with_pk:\n obj = cls(pk, *args) # should do its own fetching\n else:\n obj = cls(*args)\n self.fetch_object_from_db_by_pk(obj, table, fieldlist, pk)\n objarray.append(obj)\n return objarray", "response": "Fetches all objects from a table given a list of PKs."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nfetches all objects from a table given a set of WHERE criteria returning an array of objects of class cls.", "response": "def fetch_all_objects_from_db_where(self,\n cls: Type[T],\n table: str,\n fieldlist: Sequence[str],\n construct_with_pk: bool,\n wheredict: Optional[Dict[str, Any]],\n *args) -> List[T]:\n \"\"\"\n Fetches all objects from a table, given a set of WHERE criteria\n (ANDed), returning an array of objects of class cls.\n As usual here, the first field in the fieldlist must be the PK.\n \"\"\"\n sql = (\n \"SELECT \" + \",\".join([self.delimit(x) for x in fieldlist]) +\n \" FROM \" + self.delimit(table)\n )\n whereargs = []\n if wheredict is not None:\n sql += \" WHERE \" + \" AND \".join([\n self.delimit(k) + \"=?\"\n for k in wheredict.keys()\n ])\n whereargs = wheredict.values()\n rows = self.fetchall(sql, *whereargs)\n objects = []\n for row in rows:\n objects.append(\n create_object_from_list(cls, fieldlist, row, *args,\n construct_with_pk=construct_with_pk))\n return objects"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef insert_object_into_db_pk_known(self,\n obj: Any,\n table: str,\n fieldlist: Sequence[str]) -> None:\n \"\"\"Inserts object into database table, with PK (first field) already\n known.\"\"\"\n pkvalue = getattr(obj, fieldlist[0])\n if pkvalue is None:\n raise AssertionError(\"insert_object_intoto_db_pk_known called \"\n \"without PK\")\n valuelist = []\n for f in fieldlist:\n valuelist.append(getattr(obj, f))\n self.db_exec(\n get_sql_insert(table, fieldlist, self.get_delims()),\n *valuelist\n )", "response": "Inserts object into database table with PK already in fieldlist."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef insert_object_into_db_pk_unknown(self,\n obj: Any,\n table: str,\n fieldlist: Sequence[str]) -> None:\n \"\"\"Inserts object into database table, with PK (first field) initially\n unknown (and subsequently set in the object from the database).\"\"\"\n self.ensure_db_open()\n valuelist = []\n for f in fieldlist[1:]:\n valuelist.append(getattr(obj, f))\n cursor = self.db.cursor()\n self.db_exec_with_cursor(\n cursor,\n get_sql_insert_without_first_field(table, fieldlist,\n self.get_delims()),\n *valuelist\n )\n pkvalue = get_pk_of_last_insert(cursor)\n setattr(obj, fieldlist[0], pkvalue)", "response": "Inserts object into database table with PK initially\n unknown and subsequently set in the object from the database."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef update_object_in_db(self,\n obj: Any,\n table: str,\n fieldlist: Sequence[str]) -> None:\n \"\"\"Updates an object in the database (saves it to the database, where\n it exists there already).\"\"\"\n self.ensure_db_open()\n pkvalue = getattr(obj, fieldlist[0])\n valuelist = []\n # Non-PK fields first\n for f in fieldlist[1:]:\n valuelist.append(getattr(obj, f))\n # Then PK\n valuelist.append(pkvalue)\n cursor = self.db.cursor()\n self.db_exec_with_cursor(\n cursor,\n get_sql_update_by_first_field(table, fieldlist, self.get_delims()),\n *valuelist\n )", "response": "Updates an object in the database."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nsaving an object to the database inserting or updating as necessary.", "response": "def save_object_to_db(self,\n obj: Any,\n table: str,\n fieldlist: Sequence[str],\n is_new_record: bool) -> None:\n \"\"\"Saves a object to the database, inserting or updating as\n necessary.\"\"\"\n if is_new_record:\n pkvalue = getattr(obj, fieldlist[0])\n if pkvalue is None:\n self.insert_object_into_db_pk_unknown(obj, table, fieldlist)\n else:\n self.insert_object_into_db_pk_known(obj, table, fieldlist)\n else:\n self.update_object_in_db(obj, table, fieldlist)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\ndoing an index exist? ( Specific to MySQL.", "response": "def index_exists(self, table: str, indexname: str) -> bool:\n \"\"\"Does an index exist? (Specific to MySQL.)\"\"\"\n # MySQL:\n sql = (\"SELECT COUNT(*) FROM information_schema.statistics\"\n \" WHERE table_name=? AND index_name=?\")\n row = self.fetchone(sql, table, indexname)\n return True if row[0] >= 1 else False"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef create_index(self,\n table: str,\n field: str,\n nchars: int = None,\n indexname: str = None,\n unique: bool = False) -> Optional[int]:\n \"\"\"Creates an index (default name _idx_FIELDNAME), unless it exists\n already.\"\"\"\n limit = \"\"\n if nchars is not None:\n limit = \"({})\".format(nchars)\n if indexname is None:\n indexname = \"_idx_{}\".format(field)\n if self.index_exists(table, indexname):\n return None\n uniquestr = \"UNIQUE\" if unique else \"\"\n sql = (\n \"CREATE {unique} INDEX {indexname} \"\n \"ON {table} ({field}{limit})\".format(\n unique=uniquestr,\n indexname=indexname,\n table=table,\n field=field,\n limit=limit,\n )\n )\n return self.db_exec(sql)", "response": "Creates an index for the specified key in the specified table."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef create_index_from_fieldspec(self,\n table: str,\n fieldspec: FIELDSPEC_TYPE,\n indexname: str = None) -> None:\n \"\"\"Calls create_index based on a fieldspec, if the fieldspec has\n indexed = True.\"\"\"\n if \"indexed\" in fieldspec and fieldspec[\"indexed\"]:\n if \"index_nchar\" in fieldspec:\n nchar = fieldspec[\"index_nchar\"]\n else:\n nchar = None\n self.create_index(table, fieldspec[\"name\"], nchar,\n indexname=indexname)", "response": "Calls create_index based on a fieldspec."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef create_fulltext_index(self,\n table: str,\n field: str,\n indexname: str = None) -> Optional[int]:\n \"\"\"Creates a FULLTEXT index (default name _idxft_FIELDNAME), unless it\n exists already. See:\n\n http://dev.mysql.com/doc/refman/5.6/en/innodb-fulltext-index.html\n http://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html\n \"\"\"\n if indexname is None:\n indexname = \"_idxft_{}\".format(field)\n if self.index_exists(table, indexname):\n return None\n sql = \"CREATE FULLTEXT INDEX {} ON {} ({})\".format(indexname, table,\n field)\n return self.db_exec(sql)", "response": "Creates a FULLTEXT index for the given table and field."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nreturn the SQL fragment to define a field.", "response": "def fielddefsql_from_fieldspec(fieldspec: FIELDSPEC_TYPE) -> str:\n \"\"\"Returns SQL fragment to define a field.\"\"\"\n sql = fieldspec[\"name\"] + \" \" + fieldspec[\"sqltype\"]\n if \"notnull\" in fieldspec and fieldspec[\"notnull\"]:\n sql += \" NOT NULL\"\n if \"autoincrement\" in fieldspec and fieldspec[\"autoincrement\"]:\n sql += \" AUTO_INCREMENT\"\n if \"pk\" in fieldspec and fieldspec[\"pk\"]:\n sql += \" PRIMARY KEY\"\n else:\n if \"unique\" in fieldspec and fieldspec[\"unique\"]:\n sql += \" UNIQUE\"\n if \"comment\" in fieldspec:\n sql += \" COMMENT \" + sql_quote_string(fieldspec[\"comment\"])\n return sql"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nreturn list of field - defining SQL fragments.", "response": "def fielddefsql_from_fieldspeclist(\n self, fieldspeclist: FIELDSPECLIST_TYPE) -> str:\n \"\"\"Returns list of field-defining SQL fragments.\"\"\"\n return \",\".join([\n self.fielddefsql_from_fieldspec(x)\n for x in fieldspeclist\n ])"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nreturns a subset of the fieldspecs matching the fieldnames list.", "response": "def fieldspec_subset_by_name(\n fieldspeclist: FIELDSPECLIST_TYPE,\n fieldnames: Container[str]) -> FIELDSPECLIST_TYPE:\n \"\"\"Returns a subset of the fieldspecs matching the fieldnames list.\"\"\"\n result = []\n for x in fieldspeclist:\n if x[\"name\"] in fieldnames:\n result.append(x)\n return result"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\ndoing the table exist?", "response": "def table_exists(self, tablename: str) -> bool:\n \"\"\"Does the table exist?\"\"\"\n # information_schema is ANSI standard\n sql = \"\"\"\n SELECT COUNT(*)\n FROM information_schema.tables\n WHERE table_name=?\n AND table_schema={}\n \"\"\".format(self.get_current_schema_expr())\n row = self.fetchone(sql, tablename)\n return True if row[0] >= 1 else False"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\ndo the column exist?", "response": "def column_exists(self, tablename: str, column: str) -> bool:\n \"\"\"Does the column exist?\"\"\"\n sql = \"\"\"\n SELECT COUNT(*)\n FROM information_schema.columns\n WHERE table_name=?\n AND column_name=?\n AND table_schema={}\n \"\"\".format(self.get_current_schema_expr())\n row = self.fetchone(sql, tablename, column)\n return True if row[0] >= 1 else False"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\ndropping a table. Use caution!", "response": "def drop_table(self, tablename: str) -> int:\n \"\"\"Drops a table. Use caution!\"\"\"\n sql = \"DROP TABLE IF EXISTS {}\".format(tablename)\n log.info(\"Dropping table \" + tablename + \" (ignore any warning here)\")\n return self.db_exec_literal(sql)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nmaking a table if it doesn t already exist.", "response": "def make_table(self,\n tablename: str,\n fieldspeclist: FIELDSPECLIST_TYPE,\n dynamic: bool = False,\n compressed: bool = False) -> Optional[int]:\n \"\"\"Makes a table, if it doesn't already exist.\"\"\"\n if self.table_exists(tablename):\n log.info(\"Skipping creation of table \" + tablename +\n \" (already exists)\")\n return None\n if not self.is_mysql():\n dynamic = False\n compressed = False\n # http://dev.mysql.com/doc/refman/5.5/en/innodb-compression-usage.html\n sql = \"\"\"\n CREATE TABLE IF NOT EXISTS {tablename}\n ({fieldspecs})\n {dynamic}\n {compressed}\n \"\"\".format(\n tablename=tablename,\n fieldspecs=self.fielddefsql_from_fieldspeclist(fieldspeclist),\n dynamic=\"ROW_FORMAT=DYNAMIC\" if dynamic else \"\",\n compressed=\"ROW_FORMAT=COMPRESSED\" if compressed else \"\",\n )\n log.info(\"Creating table \" + tablename)\n return self.db_exec_literal(sql)"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nrenames a table. MySQL - specific.", "response": "def rename_table(self, from_table: str, to_table: str) -> Optional[int]:\n \"\"\"Renames a table. MySQL-specific.\"\"\"\n if not self.table_exists(from_table):\n log.info(\"Skipping renaming of table \" + from_table +\n \" (doesn't exist)\")\n return None\n if self.table_exists(to_table):\n raise RuntimeError(\"Can't rename table {} to {}: destination \"\n \"already exists!\".format(from_table, to_table))\n log.info(\"Renaming table {} to {}\", from_table, to_table)\n sql = \"RENAME TABLE {} TO {}\".format(from_table, to_table)\n return self.db_exec_literal(sql)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef add_column(self, tablename: str, fieldspec: FIELDSPEC_TYPE) -> int:\n sql = \"ALTER TABLE {} ADD COLUMN {}\".format(\n tablename, self.fielddefsql_from_fieldspec(fieldspec))\n log.info(sql)\n return self.db_exec_literal(sql)", "response": "Adds a column to an existing table."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef drop_column(self, tablename: str, fieldname: str) -> int:\n sql = \"ALTER TABLE {} DROP COLUMN {}\".format(tablename, fieldname)\n log.info(sql)\n return self.db_exec_literal(sql)", "response": "Drops a column from an existing table."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef modify_column_if_table_exists(self,\n tablename: str,\n fieldname: str,\n newdef: str) -> Optional[int]:\n \"\"\"Alters a column's definition without renaming it.\"\"\"\n if not self.table_exists(tablename):\n return None\n sql = \"ALTER TABLE {t} MODIFY COLUMN {field} {newdef}\".format(\n t=tablename,\n field=fieldname,\n newdef=newdef\n )\n log.info(sql)\n return self.db_exec_literal(sql)", "response": "Alters a column s definition without renaming it."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef change_column_if_table_exists(self,\n tablename: str,\n oldfieldname: str,\n newfieldname: str,\n newdef: str) -> Optional[int]:\n \"\"\"Renames a column and alters its definition.\"\"\"\n if not self.table_exists(tablename):\n return None\n if not self.column_exists(tablename, oldfieldname):\n return None\n sql = \"ALTER TABLE {t} CHANGE COLUMN {old} {new} {newdef}\".format(\n t=tablename,\n old=oldfieldname,\n new=newfieldname,\n newdef=newdef,\n )\n log.info(sql)\n return self.db_exec_literal(sql)", "response": "Renames a column and alters its definition."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef create_or_update_table(self,\n tablename: str,\n fieldspeclist: FIELDSPECLIST_TYPE,\n drop_superfluous_columns: bool = False,\n dynamic: bool = False,\n compressed: bool = False) -> None:\n \"\"\"\n - Make table, if it doesn't exist.\n - Add fields that aren't there.\n - Warn about superfluous fields, but don't delete them, unless\n ``drop_superfluous_columns == True``.\n - Make indexes, if requested.\n \"\"\"\n\n # 1. Make table, if it doesn't exist\n self.make_table(tablename, fieldspeclist, dynamic=dynamic,\n compressed=compressed)\n\n # 2. Are all the fields there?\n # ... copes fine with fieldnames coming back in Unicode and being\n # compared to str\n fields_in_db = set(self.fetch_column_names(tablename))\n desired_fieldnames = set(\n self.fieldnames_from_fieldspeclist(fieldspeclist))\n missing_fieldnames = desired_fieldnames - fields_in_db\n missing_fieldspecs = self.fieldspec_subset_by_name(fieldspeclist,\n missing_fieldnames)\n for f in missing_fieldspecs:\n self.add_column(tablename, f)\n\n # 3. Anything superfluous?\n superfluous_fieldnames = fields_in_db - desired_fieldnames\n for f in superfluous_fieldnames:\n if drop_superfluous_columns:\n log.warning(\"... dropping superfluous field: \" + f)\n self.drop_column(tablename, f)\n else:\n log.warning(\"... superfluous field (ignored): \" + f)\n\n # 4. Make indexes, if some have been requested:\n for fs in fieldspeclist:\n self.create_index_from_fieldspec(tablename, fs)", "response": "Create or update a new table."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nreturns details on a specific table.", "response": "def describe_table(self, table: str) -> List[List[Any]]:\n \"\"\"Returns details on a specific table.\"\"\"\n return self.flavour.describe_table(self, table)"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nreturns all column names for a table.", "response": "def fetch_column_names(self, table: str) -> List[str]:\n \"\"\"Returns all column names for a table.\"\"\"\n return self.flavour.fetch_column_names(self, table)"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef get_datatype(self, table: str, column: str) -> str:\n return self.flavour.get_datatype(self, table, column).upper()", "response": "Returns database SQL datatype for a column. e. g. VARCHAR."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nreturn database SQL datatype for a column e. g. VARCHAR ( 50 ).", "response": "def get_column_type(self, table: str, column: str) -> str:\n \"\"\"Returns database SQL datatype for a column, e.g. VARCHAR(50).\"\"\"\n return self.flavour.get_column_type(self, table, column).upper()"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef get_comment(self, table: str, column: str) -> str:\n return self.flavour.get_comment(self, table, column)", "response": "Returns database SQL comment for a column."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nexecuting SQL and writes the result to the log.", "response": "def debug_query(self, sql: str, *args) -> None:\n \"\"\"Executes SQL and writes the result to the log.\"\"\"\n rows = self.fetchall(sql, *args)\n debug_query_result(rows)"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\ndeletes all records from a table. Use caution!", "response": "def wipe_table(self, table: str) -> int:\n \"\"\"Delete all records from a table. Use caution!\"\"\"\n sql = \"DELETE FROM \" + self.delimit(table)\n return self.db_exec(sql)"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef create_or_replace_primary_key(self,\n table: str,\n fieldnames: Sequence[str]) -> int:\n \"\"\"Make a primary key, or replace it if it exists.\"\"\"\n # *** create_or_replace_primary_key: Uses code specific to MySQL\n sql = \"\"\"\n SELECT COUNT(*)\n FROM information_schema.table_constraints\n WHERE table_name=?\n AND table_schema={}\n AND constraint_name='PRIMARY'\n \"\"\".format(self.get_current_schema_expr())\n # http://forums.mysql.com/read.php?10,114742,114748#msg-114748\n row = self.fetchone(sql, table)\n has_pk_already = True if row[0] >= 1 else False\n drop_pk_if_exists = \" DROP PRIMARY KEY,\" if has_pk_already else \"\"\n fieldlist = \",\".join([self.delimit(f) for f in fieldnames])\n sql = (\"ALTER TABLE \" + self.delimit(table) +\n drop_pk_if_exists +\n \" ADD PRIMARY KEY(\" + fieldlist + \")\")\n # http://stackoverflow.com/questions/8859353\n return self.db_exec(sql)", "response": "Create a primary key or replace it if it exists."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nreturning the noise covariance of the data.", "response": "def noise_covariance(fit, dof=2, **kw):\n \"\"\"\n Covariance taking into account the 'noise covariance' of the data.\n This is technically more realistic for continuously sampled data.\n From Faber, 1993\n \"\"\"\n ev = fit.eigenvalues\n\n measurement_noise = ev[-1]/(fit.n-dof)\n return 4*ev*measurement_noise"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef angular_errors(hyp_axes):\n # Not quite sure why this is sqrt but it is empirically correct\n ax = N.sqrt(hyp_axes)\n return tuple(N.arctan2(ax[-1],ax[:-1]))", "response": "Returns a tuple of minimum and maximum angular errors corresponding to 1st and 2nd axes of PCA distribution."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef statistical_axes(fit, **kw):\n method = kw.pop('method', 'noise')\n confidence_level = kw.pop('confidence_level', 0.95)\n dof = kw.pop('dof',2)\n\n nominal = fit.eigenvalues\n\n if method == 'sampling':\n cov = sampling_covariance(fit,**kw)\n elif method == 'noise':\n cov = noise_covariance(fit,**kw)\n\n if kw.pop('chisq', False):\n # Model the incorrect behaviour of using the\n # Chi2 distribution instead of the Fisher\n # distribution (which is a measure of the\n # ratio between the two).\n z = chi2.ppf(confidence_level,dof)\n else:\n z = fisher_statistic(fit.n,confidence_level,dof=dof)\n\n # Apply two fisher F parameters (one along each axis)\n # Since we apply to each axis without division,\n # it is as if we are applying N.sqrt(2*F) to the entire\n # distribution, aligning us with (Francq, 2014)\n err = z*N.sqrt(cov)\n return apply_error_scaling(nominal, err, n=fit.n, **kw)", "response": "Returns a Hyperbolic error using a statistical process."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef create_view(operations, operation):\n operations.execute(\"CREATE VIEW %s AS %s\" % (\n operation.target.name,\n operation.target.sqltext\n ))", "response": "Implements ``CREATE VIEW``.\n\n Args:\n operations: instance of ``alembic.operations.base.Operations``\n operation: instance of :class:`.ReversibleOp`\n\n Returns:\n ``None``"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nimplement ``CREATE FUNCTION``. Args: operations: instance of ``alembic.operations.base.Operations`` operation: instance of :class:`.ReversibleOp` Returns: ``None``", "response": "def create_sp(operations, operation):\n \"\"\"\n Implements ``CREATE FUNCTION``.\n\n Args:\n operations: instance of ``alembic.operations.base.Operations``\n operation: instance of :class:`.ReversibleOp`\n\n Returns:\n ``None``\n \"\"\"\n operations.execute(\n \"CREATE FUNCTION %s %s\" % (\n operation.target.name, operation.target.sqltext\n )\n )"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nreturn a Python object from an Alembic migration module.", "response": "def _get_object_from_version(cls, operations, ident):\n \"\"\"\n Returns a Python object from an Alembic migration module (script).\n\n Args:\n operations: instance of ``alembic.operations.base.Operations``\n ident: string of the format ``version.objname``\n\n Returns:\n the object whose name is ``objname`` within the Alembic migration\n script identified by ``version``\n \"\"\"\n version, objname = ident.split(\".\")\n\n module_ = operations.get_context().script.get_revision(version).module\n obj = getattr(module_, objname)\n return obj"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef iso_string_to_python_datetime(\n isostring: str) -> Optional[datetime.datetime]:\n \"\"\"\n Takes an ISO-8601 string and returns a ``datetime``.\n \"\"\"\n if not isostring:\n return None # if you parse() an empty string, you get today's date\n return dateutil.parser.parse(isostring)", "response": "Takes an ISO - 8601 string and returns a datetime. datetime object."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nconvert a Python datetime to a string literal compatible with SQLite", "response": "def python_utc_datetime_to_sqlite_strftime_string(\n value: datetime.datetime) -> str:\n \"\"\"\n Converts a Python datetime to a string literal compatible with SQLite,\n including the millisecond field.\n \"\"\"\n millisec_str = str(round(value.microsecond / 1000)).zfill(3)\n return value.strftime(\"%Y-%m-%d %H:%M:%S\") + \".\" + millisec_str"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nconverting a Python datetime object that has a timezone to an ISO - 8601 string.", "response": "def python_localized_datetime_to_human_iso(value: datetime.datetime) -> str:\n \"\"\"\n Converts a Python ``datetime`` that has a timezone to an ISO-8601 string\n with ``:`` between the hours and minutes of the timezone.\n\n Example:\n >>> import datetime\n >>> import pytz\n >>> x = datetime.datetime.now(pytz.utc)\n >>> python_localized_datetime_to_human_iso(x)\n '2017-08-21T20:47:18.952971+00:00'\n \"\"\"\n s = value.strftime(\"%Y-%m-%dT%H:%M:%S.%f%z\")\n return s[:29] + \":\" + s[29:]"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nconverts a MySQL lookup into a SQL string.", "response": "def isodt_lookup_mysql(lookup, compiler, connection,\n operator) -> Tuple[str, Any]:\n \"\"\"\n For a comparison \"LHS *op* RHS\", transforms the LHS from a column\n containing an ISO-8601 date/time into an SQL ``DATETIME``,\n for MySQL.\n \"\"\"\n lhs, lhs_params = compiler.compile(lookup.lhs)\n rhs, rhs_params = lookup.process_rhs(compiler, connection)\n params = lhs_params + rhs_params\n return '{lhs} {op} {rhs}'.format(\n lhs=iso_string_to_sql_utcdatetime_mysql(lhs),\n op=operator,\n rhs=rhs,\n ), params"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\ntake an instance and calculates the arguments to pass to the constructor.", "response": "def deconstruct(self) -> Tuple[str, str, List[Any], Dict[str, Any]]:\n \"\"\"\n Takes an instance and calculates the arguments to pass to ``__init__``\n to reconstruct it.\n \"\"\"\n name, path, args, kwargs = super().deconstruct()\n del kwargs['max_length']\n return name, path, args, kwargs"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nconverting database value to Python value.", "response": "def from_db_value(self, value, expression, connection, context):\n \"\"\"\n Convert database value to Python value.\n Called when data is loaded from the database.\n \"\"\"\n # log.debug(\"from_db_value: {}, {}\", value, type(value))\n if value is None:\n return value\n if value == '':\n return None\n return iso_string_to_python_datetime(value)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef to_python(self, value: Optional[str]) -> Optional[Any]:\n # https://docs.djangoproject.com/en/1.8/howto/custom-model-fields/\n # log.debug(\"to_python: {}, {}\", value, type(value))\n if isinstance(value, datetime.datetime):\n return value\n if value is None:\n return value\n if value == '':\n return None\n return iso_string_to_python_datetime(value)", "response": "Converts value to a Python value."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nconvert Python value to database value for QUERYING.", "response": "def get_prep_value(self, value):\n \"\"\"\n Convert Python value to database value for QUERYING.\n We query with UTC, so this function converts datetime values to UTC.\n\n Calls to this function are followed by calls to ``get_db_prep_value()``,\n which is for backend-specific conversions.\n \"\"\"\n log.debug(\"get_prep_value: {}, {}\", value, type(value))\n if not value:\n return ''\n # For underlying (database) string types, e.g. VARCHAR, this\n # function must always return a string type.\n # https://docs.djangoproject.com/en/1.8/howto/custom-model-fields/\n # Convert to UTC\n return value.astimezone(timezone.utc)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef get_db_prep_value(self, value, connection, prepared=False):\n log.debug(\"get_db_prep_value: {}, {}\", value, type(value))\n value = super().get_db_prep_value(value, connection, prepared)\n if value is None:\n return value\n # log.debug(\"connection.settings_dict['ENGINE']: {}\",\n # connection.settings_dict['ENGINE'])\n if connection.settings_dict['ENGINE'] == 'django.db.backends.sqlite3':\n return python_utc_datetime_to_sqlite_strftime_string(value)\n return value", "response": "This method is used to convert the value to the database value for QUERYING."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef get_db_prep_save(self, value, connection, prepared=False):\n log.debug(\"get_db_prep_save: {}, {}\", value, type(value))\n if not value:\n return ''\n # For underlying (database) string types, e.g. VARCHAR, this\n # function must always return a string type.\n # https://docs.djangoproject.com/en/1.8/howto/custom-model-fields/\n return python_localized_datetime_to_human_iso(value)", "response": "Convert Python value to database value for SAVING."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef list_file_extensions(path: str, reportevery: int = 1) -> List[str]:\n extensions = set()\n count = 0\n for root, dirs, files in os.walk(path):\n count += 1\n if count % reportevery == 0:\n log.debug(\"Walking directory {}: {!r}\", count, root)\n for file in files:\n filename, ext = os.path.splitext(file)\n extensions.add(ext)\n return sorted(list(extensions))", "response": "Returns a sorted list of every file extension found in a directory and its subdirectories."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef main() -> None:\n main_only_quicksetup_rootlogger(level=logging.DEBUG)\n parser = argparse.ArgumentParser()\n parser.add_argument(\"directory\", nargs=\"?\", default=os.getcwd())\n parser.add_argument(\"--reportevery\", default=10000)\n args = parser.parse_args()\n log.info(\"Extensions in directory {!r}:\", args.directory)\n print(\"\\n\".join(repr(x) for x in\n list_file_extensions(args.directory,\n reportevery=args.reportevery)))", "response": "Command - line processor."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef are_debian_packages_installed(packages: List[str]) -> Dict[str, bool]:\n assert len(packages) >= 1\n require_executable(DPKG_QUERY)\n args = [\n DPKG_QUERY,\n \"-W\", # --show\n # \"-f='${Package} ${Status} ${Version}\\n'\",\n \"-f=${Package} ${Status}\\n\", # --showformat\n ] + packages\n completed_process = subprocess.run(args,\n stdout=subprocess.PIPE,\n stderr=subprocess.PIPE,\n check=False)\n encoding = sys.getdefaultencoding()\n stdout = completed_process.stdout.decode(encoding)\n stderr = completed_process.stderr.decode(encoding)\n present = OrderedDict()\n for line in stdout.split(\"\\n\"):\n if line: # e.g. \"autoconf install ok installed\"\n words = line.split()\n assert len(words) >= 2\n package = words[0]\n present[package] = \"installed\" in words[1:]\n for line in stderr.split(\"\\n\"):\n if line: # e.g. \"dpkg-query: no packages found matching XXX\"\n words = line.split()\n assert len(words) >= 2\n package = words[-1]\n present[package] = False\n log.debug(\"Debian package presence: {}\", present)\n return present", "response": "Check which of a list of Debian packages are installed via dpkg - query."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef require_debian_packages(packages: List[str]) -> None:\n present = are_debian_packages_installed(packages)\n missing_packages = [k for k, v in present.items() if not v]\n if missing_packages:\n missing_packages.sort()\n msg = (\n \"Debian packages are missing, as follows. Suggest:\\n\\n\"\n \"sudo apt install {}\".format(\" \".join(missing_packages))\n )\n log.critical(msg)\n raise ValueError(msg)", "response": "Ensures that specific packages are installed under Debian."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef validate_pair(ob: Any) -> bool:\n try:\n if len(ob) != 2:\n log.warning(\"Unexpected result: {!r}\", ob)\n raise ValueError()\n except ValueError:\n return False\n return True", "response": "Validate that the object is a valid pair of objects."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\ntakes a command (either a single command or list of arguments) and return the environment created after running that command. Note that the command must be a batch (``.bat``) file or ``.cmd`` file, or the changes to the environment will not be captured. If ``initial_env`` is supplied, it is used as the initial environment passed to the child process. (Otherwise, this process's ``os.environ()`` will be used by default.) From https://stackoverflow.com/questions/1214496/how-to-get-environment-from-a-subprocess-in-python, with decoding bug fixed for Python 3. PURPOSE: under Windows, ``VCVARSALL.BAT`` sets up a lot of environment variables to compile for a specific target architecture. We want to be able to read them, not to replicate its work. METHOD: create a composite command that executes the specified command, then echoes an unusual string tag, then prints the environment via ``SET``; capture the output, work out what came from ``SET``. Args: env_cmd: command, or list of command arguments initial_env: optional dictionary containing initial environment Returns: dict: environment created after running the command", "response": "def windows_get_environment_from_batch_command(\n env_cmd: Union[str, List[str]],\n initial_env: Dict[str, str] = None) -> Dict[str, str]:\n \"\"\"\n Take a command (either a single command or list of arguments) and return\n the environment created after running that command. Note that the command\n must be a batch (``.bat``) file or ``.cmd`` file, or the changes to the\n environment will not be captured.\n\n If ``initial_env`` is supplied, it is used as the initial environment\n passed to the child process. (Otherwise, this process's ``os.environ()``\n will be used by default.)\n\n From https://stackoverflow.com/questions/1214496/how-to-get-environment-from-a-subprocess-in-python,\n with decoding bug fixed for Python 3.\n\n PURPOSE: under Windows, ``VCVARSALL.BAT`` sets up a lot of environment\n variables to compile for a specific target architecture. We want to be able\n to read them, not to replicate its work.\n \n METHOD: create a composite command that executes the specified command, \n then echoes an unusual string tag, then prints the environment via ``SET``;\n capture the output, work out what came from ``SET``.\n\n Args:\n env_cmd: command, or list of command arguments\n initial_env: optional dictionary containing initial environment\n\n Returns:\n dict: environment created after running the command\n \"\"\" # noqa\n if not isinstance(env_cmd, (list, tuple)):\n env_cmd = [env_cmd]\n # construct the command that will alter the environment\n env_cmd = subprocess.list2cmdline(env_cmd)\n # create a tag so we can tell in the output when the proc is done\n tag = '+/!+/!+/! Finished command to set/print env +/!+/!+/!' # RNC\n # construct a cmd.exe command to do accomplish this\n cmd = 'cmd.exe /s /c \"{env_cmd} && echo \"{tag}\" && set\"'.format(\n env_cmd=env_cmd, tag=tag)\n # launch the process\n log.info(\"Fetching environment using command: {}\", env_cmd)\n log.debug(\"Full command: {}\", cmd)\n proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, env=initial_env)\n # parse the output sent to stdout\n encoding = sys.getdefaultencoding()\n\n def gen_lines() -> Generator[str, None, None]: # RNC: fix decode problem\n for line in proc.stdout:\n yield line.decode(encoding)\n\n # define a way to handle each KEY=VALUE line\n def handle_line(line: str) -> Tuple[str, str]: # RNC: as function\n # noinspection PyTypeChecker\n parts = line.rstrip().split('=', 1)\n # split(\"=\", 1) means \"split at '=' and do at most 1 split\"\n if len(parts) < 2:\n return parts[0], \"\"\n return parts[0], parts[1]\n\n lines = gen_lines() # RNC\n # consume whatever output occurs until the tag is reached\n consume(itertools.takewhile(lambda l: tag not in l, lines))\n # ... RNC: note that itertools.takewhile() generates values not matching\n # the condition, but then consumes the condition value itself. So the\n # tag's already gone. Example:\n #\n # def gen():\n # mylist = [1, 2, 3, 4, 5]\n # for x in mylist:\n # yield x\n #\n # g = gen()\n # list(itertools.takewhile(lambda x: x != 3, g)) # [1, 2]\n # next(g) # 4, not 3\n #\n # parse key/values into pairs\n pairs = map(handle_line, lines)\n # make sure the pairs are valid (this also eliminates the tag)\n valid_pairs = filter(validate_pair, pairs)\n # construct a dictionary of the pairs\n result = dict(valid_pairs) # consumes generator\n # let the process finish\n proc.communicate()\n log.debug(\"Fetched environment:\\n\" + pformat(result))\n return result"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nchecking if target exists in x outside quotes.", "response": "def contains_unquoted_target(x: str,\n quote: str = '\"', target: str = '&') -> bool:\n \"\"\"\n Checks if ``target`` exists in ``x`` outside quotes (as defined by\n ``quote``). Principal use: from\n :func:`contains_unquoted_ampersand_dangerous_to_windows`.\n \"\"\"\n in_quote = False\n for c in x:\n if c == quote:\n in_quote = not in_quote\n elif c == target:\n if not in_quote:\n return True\n return False"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nreturning a message that is wrapped in a message.", "response": "def gut_message(message: Message) -> Message:\n \"\"\"\n Remove body from a message, and wrap in a message/external-body.\n \"\"\"\n wrapper = Message()\n wrapper.add_header('Content-Type', 'message/external-body',\n access_type='x-spam-deleted',\n expiration=time.strftime(\"%a, %d %b %Y %H:%M:%S %z\"),\n size=str(len(message.get_payload())))\n\n message.set_payload('')\n wrapper.set_payload([message])\n\n return wrapper"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\ncleaning a message by all its binary parts and return the message itself for convenience.", "response": "def clean_message(message: Message, topmost: bool = False) -> Message:\n \"\"\"\n Clean a message of all its binary parts.\n\n This guts all binary attachments, and returns the message itself for\n convenience.\n\n \"\"\"\n if message.is_multipart():\n # Don't recurse in already-deleted attachments\n if message.get_content_type() != 'message/external-body':\n parts = message.get_payload()\n parts[:] = map(clean_message, parts)\n elif message_is_binary(message):\n # Don't gut if this is the topmost message\n if not topmost:\n message = gut_message(message)\n\n return message"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nhashing a plaintext password using bcrypt.", "response": "def hash_password(plaintextpw: str,\n log_rounds: int = BCRYPT_DEFAULT_LOG_ROUNDS) -> str:\n \"\"\"\n Makes a hashed password (using a new salt) using ``bcrypt``.\n\n The hashed password includes the salt at its start, so no need to store a\n separate salt.\n \"\"\"\n salt = bcrypt.gensalt(log_rounds) # optional parameter governs complexity\n hashedpw = bcrypt.hashpw(plaintextpw, salt)\n return hashedpw"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef is_password_valid(plaintextpw: str, storedhash: str) -> bool:\n # Upon CamCOPS from MySQL 5.5.34 (Ubuntu) to 5.1.71 (CentOS 6.5), the\n # VARCHAR was retrieved as Unicode. We needed to convert that to a str.\n # For Python 3 compatibility, we just str-convert everything, avoiding the\n # unicode keyword, which no longer exists.\n if storedhash is None:\n storedhash = \"\"\n storedhash = str(storedhash)\n if plaintextpw is None:\n plaintextpw = \"\"\n plaintextpw = str(plaintextpw)\n try:\n h = bcrypt.hashpw(plaintextpw, storedhash)\n except ValueError: # e.g. ValueError: invalid salt\n return False\n return h == storedhash", "response": "Checks if a plaintext password matches a stored hash."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef create_sys_dsn(driver: str, **kw) -> bool:\n attributes = [] # type: List[str]\n for attr in kw.keys():\n attributes.append(\"%s=%s\" % (attr, kw[attr]))\n return bool(\n ctypes.windll.ODBCCP32.SQLConfigDataSource(0, ODBC_ADD_SYS_DSN,\n driver,\n nul.join(attributes))\n )", "response": "Create a system ODBC data source name."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef create_user_dsn(driver: str, **kw) -> bool:\n attributes = [] # type: List[str]\n for attr in kw.keys():\n attributes.append(\"%s=%s\" % (attr, kw[attr]))\n return bool(\n ctypes.windll.ODBCCP32.SQLConfigDataSource(0, ODBC_ADD_DSN, driver,\n nul.join(attributes))\n )", "response": "Create a user ODBC data source name."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nregister a Microsoft Access database with Microsoft SQL Server.", "response": "def register_access_db(fullfilename: str, dsn: str, description: str) -> bool:\n \"\"\"\n (Windows only.)\n Registers a Microsoft Access database with ODBC.\n\n Args:\n fullfilename: filename of the existing database\n dsn: ODBC data source name to create\n description: description of the database\n\n Returns:\n bool: was the DSN created?\n\n \"\"\"\n directory = os.path.dirname(fullfilename)\n return create_sys_dsn(\n access_driver,\n SERVER=\"\",\n DESCRIPTION=description,\n DSN=dsn,\n DBQ=fullfilename,\n DefaultDir=directory\n )"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef create_and_register_access97_db(filename: str,\n dsn: str,\n description: str) -> bool:\n \"\"\"\n (Windows only.)\n Creates a Microsoft Access 97 database and registers it with ODBC.\n\n Args:\n filename: filename of the database to create\n dsn: ODBC data source name to create\n description: description of the database\n\n Returns:\n bool: was the DSN created?\n \"\"\"\n fullfilename = os.path.abspath(filename)\n create_string = fullfilename + \" General\"\n # ... filename, space, sort order (\"General\" for English)\n return (create_user_dsn(access_driver, CREATE_DB3=create_string) and\n register_access_db(filename, dsn, description))", "response": "Creates a Microsoft Access 97 database and registers it with ODBC."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef create_and_register_access2000_db(filename: str,\n dsn: str,\n description: str) -> bool:\n \"\"\"\n (Windows only.)\n Creates a Microsoft Access 2000 database and registers it with ODBC.\n\n Args:\n filename: filename of the database to create\n dsn: ODBC data source name to create\n description: description of the database\n\n Returns:\n bool: was the DSN created?\n \"\"\"\n fullfilename = os.path.abspath(filename)\n create_string = fullfilename + \" General\"\n # ... filename, space, sort order (\"General\" for English)\n return (create_user_dsn(access_driver, CREATE_DB4=create_string) and\n register_access_db(filename, dsn, description))", "response": "Creates a Microsoft Access 2000 database and registers it with ODBC."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\ncreates a Microsoft Access database and registers it with ODBC.", "response": "def create_and_register_access_db(filename: str,\n dsn: str,\n description: str) -> bool:\n \"\"\"\n (Windows only.)\n Creates a Microsoft Access database and registers it with ODBC.\n\n Args:\n filename: filename of the database to create\n dsn: ODBC data source name to create\n description: description of the database\n\n Returns:\n bool: was the DSN created?\n \"\"\"\n fullfilename = os.path.abspath(filename)\n create_string = fullfilename + \" General\"\n # ... filename, space, sort order (\"General\" for English)\n return (create_user_dsn(access_driver, CREATE_DB=create_string) and\n register_access_db(filename, dsn, description))"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _finalized(self):\n return \\\n len(self._finalized_dependencies) >= self.dependency_count \\\n and len(self._enqueued_dependencies) >= self.dependency_count", "response": "Return true if all dependencies have been finalized and that they ll have sent at least one batch of data."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef version(self):\n try:\n f = self.func.__call__.__code__\n except AttributeError:\n f = self.func.__code__\n\n h = md5()\n h.update(f.co_code)\n h.update(str(f.co_names).encode())\n\n try:\n closure = self.func.__closure__\n except AttributeError:\n return h.hexdigest()\n\n if closure is None or self.closure_fingerprint is None:\n return h.hexdigest()\n\n d = dict(\n (name, cell.cell_contents)\n for name, cell in zip(f.co_freevars, closure))\n h.update(self.closure_fingerprint(d).encode())\n\n return h.hexdigest()", "response": "Compute the version identifier for this functional node using the functional node s local names and closure fingerprint."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\ncreate a table from an SQLAlchemy ORM class.", "response": "def create_table_from_orm_class(engine: Engine,\n ormclass: DeclarativeMeta,\n without_constraints: bool = False) -> None:\n \"\"\"\n From an SQLAlchemy ORM class, creates the database table via the specified\n engine, using a ``CREATE TABLE`` SQL (DDL) statement.\n\n Args:\n engine: SQLAlchemy :class:`Engine` object\n ormclass: SQLAlchemy ORM class\n without_constraints: don't add foreign key constraints\n \"\"\"\n table = ormclass.__table__ # type: Table\n log.info(\"Creating table {} on engine {}{}\",\n table.name,\n get_safe_url_from_engine(engine),\n \" (omitting constraints)\" if without_constraints else \"\")\n # https://stackoverflow.com/questions/19175311/how-to-create-only-one-table-with-sqlalchemy # noqa\n if without_constraints:\n include_foreign_key_constraints = []\n else:\n include_foreign_key_constraints = None # the default\n creator = CreateTable(\n table,\n include_foreign_key_constraints=include_foreign_key_constraints\n )\n creator.execute(bind=engine)"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef mapper_init(self):\n self.counties = CachedCountyLookup(precision=GEOHASH_PRECISION)\n self.extractor = WordExtractor()", "response": "Initialize the counties and extractor objects"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef run(self):\n logging.info(\"Starting GeoJSON MongoDB loading process.\")\n mongo = dict(uri=self.mongo, db=self.db, collection=self.collection)\n self.load(self.source, **mongo)\n logging.info(\"Finished loading {0} into MongoDB\".format(self.source))", "response": "Load the state and county GeoJSON files into MongoDB DB"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef load(self, geojson, uri=None, db=None, collection=None):\n logging.info(\"Mongo URI: {0}\".format(uri))\n logging.info(\"Mongo DB: {0}\".format(db))\n logging.info(\"Mongo Collection: {0}\".format(collection))\n logging.info(\"Geojson File to be loaded: {0}\".format(geojson))\n mongo = MongoGeo(db=db, collection=collection, uri=uri)\n GeoJSONLoader().load(geojson, mongo.insert)", "response": "Load a geojson file into a mongodb instance"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nreturning formatted version string.", "response": "def get_version():\n \"\"\"\n Return formatted version string.\n\n Returns:\n str: string with project version or empty string.\n \"\"\"\n\n if all([VERSION, UPDATED, any([isinstance(UPDATED, date), isinstance(UPDATED, datetime), ]), ]):\n\n return FORMAT_STRING.format(**{\"version\": VERSION, \"updated\": UPDATED, })\n elif VERSION:\n\n return VERSION\n elif UPDATED:\n\n return localize(UPDATED) if any([isinstance(UPDATED, date), isinstance(UPDATED, datetime), ]) else \"\"\n else:\n\n return \"\""} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef process_file(filename: str,\n filetypes: List[str],\n move_to: str,\n delete_if_not_specified_file_type: bool,\n show_zip_output: bool) -> None:\n \"\"\"\n Deals with an OpenXML, including if it is potentially corrupted.\n\n Args:\n filename: filename to process\n filetypes: list of filetypes that we care about, e.g.\n ``['docx', 'pptx', 'xlsx']``.\n move_to: move matching files to this directory\n delete_if_not_specified_file_type: if ``True``, and the file is **not**\n a type specified in ``filetypes``, then delete the file.\n show_zip_output: show the output from the external ``zip`` tool?\n \"\"\"\n # log.critical(\"process_file: start\")\n try:\n reader = CorruptedOpenXmlReader(filename,\n show_zip_output=show_zip_output)\n if reader.file_type in filetypes:\n log.info(\"Found {}: {}\", reader.description, filename)\n if move_to:\n dest_file = os.path.join(move_to, os.path.basename(filename))\n _, ext = os.path.splitext(dest_file)\n if ext != reader.suggested_extension():\n dest_file += reader.suggested_extension()\n reader.move_to(destination_filename=dest_file)\n else:\n log.info(\"Unrecognized or unwanted contents: \" + filename)\n if delete_if_not_specified_file_type:\n log.info(\"Deleting: \" + filename)\n os.remove(filename)\n except Exception as e:\n # Must explicitly catch and report errors, since otherwise they vanish\n # into the ether.\n log.critical(\"Uncaught error in subprocess: {!r}\\n{}\", e,\n traceback.format_exc())\n raise", "response": "Processes an OpenXML file and returns a list of the ether files that can be used to create the ether tree."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef main() -> None:\n parser = ArgumentParser(\n formatter_class=RawDescriptionHelpFormatter,\n description=\"\"\"\nTool to recognize and rescue Microsoft Office OpenXML files, even if they have\ngarbage appended to them. \n\n- Rationale: when you have accidentally deleted files from an NTFS disk, and\n they really matter, you should (a) stop what you're doing; (b) clone the disk\n to an image file using \"dd\" under Linux; (c) perform all subsequent \n operations on the cloned image (in read-only mode). Those steps might \n include:\n - ntfsundelete, to find files that the filesystem is still aware of;\n - scalpel, to find files based on their contents.\n\n- Scalpel is great at finding stuff efficiently, but it works best when files\n can be defined by both a start (header) signature and an end (footer)\n signature. However, the Microsoft Office OpenXML file format has a \n recognizable header, but no standard footer. In these circumstances, Scalpel\n reads up to a certain limit that you specify in its configuration file. (To\n retrieve large Powerpoint files, this limit needs to be substantial, e.g.\n 50 Mb or more, depending on your ways of working with Powerpoint.)\n\n- That means that files emerging from a Scalpel search for DOCX/PPTX/XLSX files\n may be\n - false positives, having nothing to do with Office;\n - clean Office files (the least likely category!);\n - Office files with garbage stuck on the end.\n \n- The OpenXML file format is just a zip file. If you stick too much garbage on\n the end of a zip file, zip readers will see it as corrupt. \n \n- THIS TOOL detects (and optionally moves) potentially corrupted zipfiles based \n on file contents, by unzipping the file and checking for \"inner\" files with\n names like:\n\n File type Contents filename signature (regular expression)\n ----------------------------------------------------------------\n DOCX {DOCX_CONTENTS_REGEX_STR} \n PPTX {PPTX_CONTENTS_REGEX_STR}\n XLSX {XLSX_CONTENTS_REGEX_STR}\n\n- WARNING: it's possible for an OpenXML file to contain more than one of these.\n If so, they may be mis-classified.\n\n- If a file is not immediately readable as a zip, it uses Linux's \"zip -FF\" to \n repair zip files with corrupted ends, and tries again.\n \n- Having found valid-looking files, you can elect to move them elsewhere.\n\n- As an additional and VERY DANGEROUS operation, you can elect to delete files\n that this tool doesn't recognize. (Why? Because a 450Gb disk might produce\n well in excess of 1.7Tb of candidate files; many will be false positives and\n even the true positives will all be expanded to your file size limit, e.g.\n 50 Mb. You may have a problem with available disk space, so running this tool\n regularly allows you to clear up the junk. Use the --run_every option to help \n with this.)\n\n \"\"\".format(\n DOCX_CONTENTS_REGEX_STR=DOCX_CONTENTS_REGEX_STR,\n PPTX_CONTENTS_REGEX_STR=PPTX_CONTENTS_REGEX_STR,\n XLSX_CONTENTS_REGEX_STR=XLSX_CONTENTS_REGEX_STR,\n )\n )\n parser.add_argument(\n \"filename\", nargs=\"+\",\n help=\"File(s) to check. You can also specify directores if you use \"\n \"--recursive\"\n )\n parser.add_argument(\n \"--recursive\", action=\"store_true\",\n help=\"Allow search to descend recursively into any directories \"\n \"encountered.\"\n )\n parser.add_argument(\n \"--skip_files\", nargs=\"*\", default=[],\n help=\"File pattern(s) to skip. You can specify wildcards like '*.txt' \"\n \"(but you will have to enclose that pattern in quotes under \"\n \"UNIX-like operating systems). The basename of each file will be \"\n \"tested against these filenames/patterns. Consider including \"\n \"Scalpel's 'audit.txt'.\"\n )\n parser.add_argument(\n \"--filetypes\", nargs=\"+\", default=FILETYPES,\n help=\"File types to check. Options: {}\".format(FILETYPES)\n )\n parser.add_argument(\n \"--move_to\",\n help=\"If the file is recognized as one of the specified file types, \"\n \"move it to the directory specified here.\"\n )\n parser.add_argument(\n \"--delete_if_not_specified_file_type\", action=\"store_true\",\n help=\"If a file is NOT recognized as one of the specified file types, \"\n \"delete it. VERY DANGEROUS.\"\n )\n parser.add_argument(\n \"--run_repeatedly\", type=int,\n help=\"Run the tool repeatedly with a pause of \"\n \"seconds between runs. (For this to work well with the move/\"\n \"delete options, you should specify one or more DIRECTORIES in \"\n \"the 'filename' arguments, not files, and you will need the \"\n \"--recursive option.)\"\n )\n parser.add_argument(\n \"--nprocesses\", type=int, default=multiprocessing.cpu_count(),\n help=\"Specify the number of processes to run in parallel.\"\n )\n parser.add_argument(\n \"--verbose\", action=\"store_true\",\n help=\"Verbose output\"\n )\n parser.add_argument(\n \"--show_zip_output\", action=\"store_true\",\n help=\"Verbose output from the external 'zip' tool\"\n )\n args = parser.parse_args()\n main_only_quicksetup_rootlogger(\n level=logging.DEBUG if args.verbose else logging.INFO,\n with_process_id=True\n )\n\n # Further argument checks\n if args.move_to:\n if not os.path.isdir(args.move_to):\n raise ValueError(\"Destination directory {!r} is not a \"\n \"directory\".format(args.move_to))\n if not args.filetypes:\n raise ValueError(\"No file type to scan for\")\n filetypes = [ft.lower() for ft in args.filetypes]\n if any(ft not in FILETYPES for ft in filetypes):\n raise ValueError(\"Invalid filetypes; choose from {}\".format(FILETYPES))\n assert shutil.which(\"zip\"), \"Need 'zip' tool!\"\n\n # Repeated scanning loop\n while True:\n log.info(\"Starting scan.\")\n log.info(\"- Looking for filetypes {}\", filetypes)\n log.info(\"- Scanning files/directories {!r}{}\",\n args.filename,\n \" recursively\" if args.recursive else \"\")\n log.info(\"- Skipping files matching {!r}\", args.skip_files)\n log.info(\"- Using {} simultaneous processes\", args.nprocesses)\n if args.move_to:\n log.info(\"- Moving target files to \" + args.move_to)\n if args.delete_if_not_specified_file_type:\n log.info(\"- Deleting non-target files.\")\n\n # Iterate through files\n pool = multiprocessing.Pool(processes=args.nprocesses)\n for filename in gen_filenames(starting_filenames=args.filename,\n recursive=args.recursive):\n src_basename = os.path.basename(filename)\n if any(fnmatch.fnmatch(src_basename, pattern)\n for pattern in args.skip_files):\n log.info(\"Skipping file as ordered: \" + filename)\n continue\n exists, locked = exists_locked(filename)\n if locked or not exists:\n log.info(\"Skipping currently inaccessible file: \" + filename)\n continue\n kwargs = {\n 'filename': filename,\n 'filetypes': filetypes,\n 'move_to': args.move_to,\n 'delete_if_not_specified_file_type':\n args.delete_if_not_specified_file_type,\n 'show_zip_output': args.show_zip_output,\n }\n # log.critical(\"start\")\n pool.apply_async(process_file, [], kwargs)\n # result = pool.apply_async(process_file, [], kwargs)\n # result.get() # will re-raise any child exceptions\n # ... but it waits for the process to complete! That's no help.\n # log.critical(\"next\")\n # ... https://stackoverflow.com/questions/22094852/how-to-catch-exceptions-in-workers-in-multiprocessing # noqa\n pool.close()\n pool.join()\n\n log.info(\"Finished scan.\")\n if args.run_repeatedly is None:\n break\n log.info(\"Sleeping for {} s...\", args.run_repeatedly)\n sleep(args.run_repeatedly)", "response": "This is the main function for the find_recovered_openxml tool. It is used to find the actual set of files that are not part of the NTFS disk and the actual set of files that are not part of the NTFS disk."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nmoving the file to which this class refers to a new location.", "response": "def move_to(self, destination_filename: str,\n alter_if_clash: bool = True) -> None:\n \"\"\"\n Move the file to which this class refers to a new location.\n The function will not overwrite existing files (but offers the option\n to rename files slightly to avoid a clash).\n\n Args:\n destination_filename: filename to move to\n alter_if_clash: if ``True`` (the default), appends numbers to\n the filename if the destination already exists, so that the\n move can proceed.\n \"\"\"\n if not self.src_filename:\n return\n if alter_if_clash:\n counter = 0\n while os.path.exists(destination_filename):\n root, ext = os.path.splitext(destination_filename)\n destination_filename = \"{r}_{c}{e}\".format(\n r=root, c=counter, e=ext)\n counter += 1\n # ... for example, \"/a/b/c.txt\" becomes \"/a/b/c_0.txt\", then\n # \"/a/b/c_1.txt\", and so on.\n else:\n if os.path.exists(destination_filename):\n src = self.rescue_filename or self.src_filename\n log.warning(\"Destination exists; won't move {!r} to {!r}\",\n src, destination_filename)\n return\n if self.rescue_filename:\n shutil.move(self.rescue_filename, destination_filename)\n os.remove(self.src_filename)\n log.info(\"Moved recovered file {!r} to {!r} and deleted corrupted \"\n \"original {!r}\",\n self.rescue_filename,\n destination_filename,\n self.src_filename)\n self.rescue_filename = \"\"\n else:\n shutil.move(self.src_filename, destination_filename)\n log.info(\"Moved {!r} to {!r}\", self.src_filename,\n destination_filename)\n self.src_filename = \"\""} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nchecks if the file should be built or not", "response": "def should_build(self, fpath, meta):\n \"\"\"\n Checks if the file should be built or not\n Only skips layouts which are tagged as INCREMENTAL\n Rebuilds only those files with mtime changed since previous build\n \"\"\"\n if meta.get('layout', self.default_template) in self.inc_layout:\n if self.prev_mtime.get(fpath, 0) == os.path.getmtime(fpath):\n return False\n else:\n return True\n return True"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nreturn a dict of queues in cluster and servers running them", "response": "def clusterQueues(self):\n \"\"\" Return a dict of queues in cluster and servers running them\n \"\"\"\n servers = yield self.getClusterServers()\n\n queues = {}\n\n for sname in servers:\n qs = yield self.get('rhumba.server.%s.queues' % sname)\n uuid = yield self.get('rhumba.server.%s.uuid' % sname)\n \n qs = json.loads(qs)\n\n for q in qs:\n if q not in queues:\n queues[q] = []\n\n queues[q].append({'host': sname, 'uuid': uuid})\n\n defer.returnValue(queues)"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef clusterStatus(self):\n servers = yield self.getClusterServers()\n\n d = {\n 'workers': {},\n 'crons': {},\n 'queues': {}\n }\n\n now = time.time()\n\n reverse_map = {}\n\n for sname in servers:\n last = yield self.get('rhumba.server.%s.heartbeat' % sname)\n status = yield self.get('rhumba.server.%s.status' % sname)\n uuid = yield self.get('rhumba.server.%s.uuid' % sname)\n\n reverse_map[uuid] = sname\n\n if not last:\n last = 0\n\n last = float(last)\n\n if (status == 'ready') and (now - last > 5):\n status = 'offline'\n\n if not sname in d['workers']:\n d['workers'][sname] = []\n\n d['workers'][sname].append({\n 'lastseen': last,\n 'status': status,\n 'id': uuid\n })\n\n # Crons\n crons = yield self.keys('rhumba\\.crons\\.*')\n\n for key in crons:\n segments = key.split('.')\n\n queue = segments[2]\n if queue not in d['crons']:\n d['crons'][queue] = {'methods': {}}\n\n if len(segments)==4:\n last = yield self.get(key)\n d['crons'][queue]['methods'][segments[3]] = float(last)\n else:\n uid = yield self.get(key)\n d['crons'][queue]['master'] = '%s:%s' % (uid, reverse_map[uid])\n\n # Queues\n queue_keys = yield self.keys('rhumba.qstats.*')\n\n for key in queue_keys:\n qname = key.split('.')[2]\n if qname not in d['queues']:\n qlen = yield self.queueSize(qname)\n\n stats = yield self.getQueueMessageStats(qname)\n\n d['queues'][qname] = {\n 'waiting': qlen,\n 'messages': stats\n }\n\n defer.returnValue(d)", "response": "Returns a dict of cluster nodes and their status information"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef get(self, index, id, fields=None, doc_type=EsConst.ALL_VALUES, **query_params):\n if fields:\n query_params[EsConst.FIELDS] = fields\n\n path = self._es_parser.make_path(index, doc_type, id)\n result = yield self._perform_request(HttpMethod.GET, path, params=query_params)\n returnValue(result)", "response": "Retrieve specific record by id"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef exists(self, index, doc_type, id, **query_params):\n self._es_parser.is_not_empty_params(index, doc_type, id)\n path = self._es_parser.make_path(index, doc_type, id)\n result = yield self._perform_request(HttpMethod.HEAD,\n path,\n params=query_params)\n returnValue(result)", "response": "Check if the document exists in the elastic search"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nget the _source field of the document with the specified id", "response": "def get_source(self, index, doc_type, id, **query_params):\n \"\"\"\n Get the _source of the document\n ``_\n :param index: the index name\n :param doc_type: the document type\n :param id: the id of the doc type\n :arg _source: True or false to return the _source field or not, or a\n list of fields to return\n :arg _source_exclude: A list of fields to exclude from the returned\n _source field\n :arg _source_include: A list of fields to extract and return from the\n _source field\n :arg parent: The ID of the parent document\n :arg preference: Specify the node or shard the operation should be\n performed on (default: random)\n :arg realtime: Specify whether to perform the operation in realtime or\n search mode\n :arg refresh: Refresh the shard containing the document before\n performing the operation\n :arg routing: Specific routing value\n :arg version: Explicit version number for concurrency control\n :arg version_type: Specific version type, valid choices are: 'internal',\n 'external', 'external_gte', 'force'\n \"\"\"\n self._es_parser.is_not_empty_params(index, doc_type, id)\n path = self._es_parser.make_path(index, doc_type, id, EsMethods.SOURCE)\n result = yield self._perform_request(HttpMethod.GET,\n path,\n params=query_params)\n returnValue(result)"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef mget(self, body, index=None, doc_type=None, **query_params):\n self._es_parser.is_not_empty_params(body)\n\n path = self._es_parser.make_path(index,\n doc_type,\n EsMethods.MULTIPLE_GET)\n\n result = yield self._perform_request(HttpMethod.GET,\n path,\n body=body,\n params=query_params)\n returnValue(result)", "response": "Get multiple document from the same index and doc_type by ids\n "} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nupdate a document in an index", "response": "def update(self, index, doc_type, id, body=None, **query_params):\n \"\"\"\n Update a document with the body param or list of ids\n ``_\n\n :param index: the name of the index\n :param doc_type: the document type\n :param id: the id of the doc type\n :param body: the data to update in the index\n :arg consistency: Explicit write consistency setting for the operation,\n valid choices are: 'one', 'quorum', 'all'\n :arg fields: A comma-separated list of fields to return in the response\n :arg lang: The script language (default: groovy)\n :arg parent: ID of the parent document. Is is only used for routing and\n when for the upsert request\n :arg refresh: Refresh the index after performing the operation\n :arg retry_on_conflict: Specify how many times should the operation be\n retried when a conflict occurs (default: 0)\n :arg routing: Specific routing value\n :arg script: The URL-encoded script definition (instead of using request\n body)\n :arg script_id: The id of a stored script\n :arg scripted_upsert: True if the script referenced in script or\n script_id should be called to perform inserts - defaults to false\n :arg timeout: Explicit operation timeout\n :arg timestamp: Explicit timestamp for the document\n :arg ttl: Expiration time for the document\n :arg version: Explicit version number for concurrency control\n :arg version_type: Specific version type, valid choices are: 'internal',\n 'force'\n \"\"\"\n self._es_parser.is_not_empty_params(index, doc_type, id)\n path = self._es_parser.make_path(index, doc_type, id, EsMethods.UPDATE)\n result = yield self._perform_request(HttpMethod.POST, path, body=body, params=query_params)\n returnValue(result)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nmaking a search on the elastic search index", "response": "def search(self, index=None, doc_type=None, body=None, **query_params):\n \"\"\"\n Make a search query on the elastic search\n ``_\n\n :param index: the index name to query\n :param doc_type: he doc type to search in\n :param body: the query\n :param query_params: params\n :arg _source: True or false to return the _source field or not, or a\n list of fields to return\n :arg _source_exclude: A list of fields to exclude from the returned\n _source field\n :arg _source_include: A list of fields to extract and return from the\n _source field\n :arg allow_no_indices: Whether to ignore if a wildcard indices\n expression resolves into no concrete indices. (This includes `_all`\n string or when no indices have been specified)\n :arg analyze_wildcard: Specify whether wildcard and prefix queries\n should be analyzed (default: false)\n :arg analyzer: The analyzer to use for the query string\n :arg default_operator: The default operator for query string query (AND\n or OR), default 'OR', valid choices are: 'AND', 'OR'\n :arg df: The field to use as default where no field prefix is given in\n the query string\n :arg expand_wildcards: Whether to expand wildcard expression to concrete\n indices that are open, closed or both., default 'open', valid\n choices are: 'open', 'closed', 'none', 'all'\n :arg explain: Specify whether to return detailed information about score\n computation as part of a hit\n :arg fielddata_fields: A comma-separated list of fields to return as the\n field data representation of a field for each hit\n :arg fields: A comma-separated list of fields to return as part of a hit\n :arg from\\_: Starting offset (default: 0)\n :arg ignore_unavailable: Whether specified concrete indices should be\n ignored when unavailable (missing or closed)\n :arg lenient: Specify whether format-based query failures (such as\n providing text to a numeric field) should be ignored\n :arg lowercase_expanded_terms: Specify whether query terms should be\n lowercased\n :arg preference: Specify the node or shard the operation should be\n performed on (default: random)\n :arg q: Query in the Lucene query string syntax\n :arg request_cache: Specify if request cache should be used for this\n request or not, defaults to index level setting\n :arg routing: A comma-separated list of specific routing values\n :arg scroll: Specify how long a consistent view of the index should be\n maintained for scrolled search\n :arg search_type: Search operation type, valid choices are:\n 'query_then_fetch', 'dfs_query_then_fetch'\n :arg size: Number of hits to return (default: 10)\n :arg sort: A comma-separated list of : pairs\n :arg stats: Specific 'tag' of the request for logging and statistical\n purposes\n :arg suggest_field: Specify which field to use for suggestions\n :arg suggest_mode: Specify suggest mode, default 'missing', valid\n choices are: 'missing', 'popular', 'always'\n :arg suggest_size: How many suggestions to return in response\n :arg suggest_text: The source text for which the suggestions should be\n returned\n :arg terminate_after: The maximum number of documents to collect for\n each shard, upon reaching which the query execution will terminate\n early.\n :arg timeout: Explicit operation timeout\n :arg track_scores: Whether to calculate and return scores even if they\n are not used for sorting\n :arg version: Specify whether to return document version as part of a\n hit\n \"\"\"\n path = self._es_parser.make_path(index, doc_type, EsMethods.SEARCH)\n result = yield self._perform_request(HttpMethod.POST, path, body=body, params=query_params)\n returnValue(result)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef explain(self, index, doc_type, id, body=None, **query_params):\n self._es_parser.is_not_empty_params(index, doc_type, id)\n\n path = self._es_parser.make_path(index,\n doc_type,\n id,\n EsMethods.EXPLAIN)\n\n result = yield self._perform_request(HttpMethod.GET,\n path,\n body,\n params=query_params)\n returnValue(result)", "response": "The explain API computes a score explanation for a specific document and a specific document."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef delete(self, index, doc_type, id, **query_params):\n path = self._es_parser.make_path(index, doc_type, id)\n result = yield self._perform_request(HttpMethod.DELETE, path, params=query_params)\n returnValue(result)", "response": "Delete specific record by id"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef index(self, index, doc_type, body, id=None, **query_params):\n self._es_parser.is_not_empty_params(index, doc_type, body)\n\n method = HttpMethod.POST if id in NULL_VALUES else HttpMethod.PUT\n path = self._es_parser.make_path(index, doc_type, id)\n result = yield self._perform_request(method, path, body, params=query_params)\n returnValue(result)", "response": "Adds or updates a typed JSON document in a specific index."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef create(self, index, doc_type, body, id=None, **query_params):\n query_params['op_type'] = 'create'\n result = yield self.index(index, doc_type, body, id=id, params=query_params)\n returnValue(result)", "response": "Adds a typed JSON document in a specific index making it searchable."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef scan(self, index, doc_type, query=None, scroll='5m', preserve_order=False, size=10, **kwargs):\n if not preserve_order:\n kwargs['search_type'] = 'scan'\n # initial search\n results = yield self.search(index=index,\n doc_type=doc_type,\n body=query,\n size=size,\n scroll=scroll,\n **kwargs)\n\n returnValue(Scroller(self, results, scroll, size))", "response": "Scans the index for the language of the entry in the index."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nexecutes a query and get the number of matches for that query.", "response": "def count(self, index=None, doc_type=None, body=None, **query_params):\n \"\"\"\n Execute a query and get the number of matches for that query.\n ``_\n :param index: A comma-separated list of indices to restrict the results\n :param doc_type: A comma-separated list of types to restrict the results\n :param body: A query to restrict the results specified with the Query DSL\n (optional)\n :arg allow_no_indices: Whether to ignore if a wildcard indices\n expression resolves into no concrete indices. (This includes `_all`\n string or when no indices have been specified)\n :arg analyze_wildcard: Specify whether wildcard and prefix queries\n should be analyzed (default: false)\n :arg analyzer: The analyzer to use for the query string\n :arg default_operator: The default operator for query string query (AND\n or OR), default 'OR', valid choices are: 'AND', 'OR'\n :arg df: The field to use as default where no field prefix is given in\n the query string\n :arg expand_wildcards: Whether to expand wildcard expression to concrete\n indices that are open, closed or both., default 'open', valid\n choices are: 'open', 'closed', 'none', 'all'\n :arg ignore_unavailable: Whether specified concrete indices should be\n ignored when unavailable (missing or closed)\n :arg lenient: Specify whether format-based query failures (such as\n providing text to a numeric field) should be ignored\n :arg lowercase_expanded_terms: Specify whether query terms should be\n lowercased\n :arg min_score: Include only documents with a specific `_score` value in\n the result\n :arg preference: Specify the node or shard the operation should be\n performed on (default: random)\n :arg q: Query in the Lucene query string syntax\n :arg routing: Specific routing value\n \"\"\"\n if doc_type and not index:\n index = EsConst.ALL_VALUES\n\n path = self._es_parser.make_path(index, doc_type, EsMethods.COUNT)\n result = yield self._perform_request(HttpMethod.GET, path, body, params=query_params)\n returnValue(result)"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef bulk(self, body, index=None, doc_type=None, **query_params):\n self._es_parser.is_not_empty_params(body)\n path = self._es_parser.make_path(index, doc_type, EsMethods.BULK)\n result = yield self._perform_request(HttpMethod.POST,\n path,\n self._bulk_body(body),\n params=query_params)\n returnValue(result)", "response": "Perform bulk operation on a set of items."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nexecute multiple search requests within the same API.", "response": "def msearch(self, body, index=None, doc_type=None, **query_params):\n \"\"\"\n Execute several search requests within the same API.\n ``_\n :arg body: The request definitions (metadata-search request definition\n pairs), separated by newlines\n :arg index: A comma-separated list of index names to use as default\n :arg doc_type: A comma-separated list of document types to use as\n default\n :arg search_type: Search operation type, valid choices are:\n 'query_then_fetch', 'query_and_fetch', 'dfs_query_then_fetch',\n 'dfs_query_and_fetch'\n \"\"\"\n self._es_parser.is_not_empty_params(body)\n path = self._es_parser.make_path(index,\n doc_type,\n EsMethods.MULTIPLE_SEARCH)\n\n result = yield self._perform_request(HttpMethod.GET,\n path,\n self._bulk_body(body),\n params=query_params)\n returnValue(result)"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\ncloses all http connections.", "response": "def close(self):\n \"\"\"\n close all http connections.\n returns a deferred that fires once they're all closed.\n \"\"\"\n\n def validate_client(client):\n \"\"\"\n Validate that the connection is for the current client\n :param client:\n :return:\n \"\"\"\n host, port = client.addr\n parsed_url = urlparse(self._hostname)\n return host == parsed_url.hostname and port == parsed_url.port\n\n # read https://github.com/twisted/treq/issues/86\n # to understand the following...\n def _check_fds(_):\n fds = set(reactor.getReaders() + reactor.getReaders())\n if not [fd for fd in fds if isinstance(fd, Client) and validate_client(fd)]:\n return\n\n return deferLater(reactor, 0, _check_fds, None)\n\n pool = self._async_http_client_params[\"pool\"]\n return pool.closeCachedConnections().addBoth(_check_fds)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nremoving invalid coordinates from a sequence of coordinates.", "response": "def clean_coordinates(coords, silent=False):\n \"\"\"\n Removes invalid coordinates (this is generally caused\n by importing coordinates outside of the DEM)\n \"\"\"\n l1 = len(coords)\n coords = coords[~N.isnan(coords).any(axis=1)] # remove NaN values\n l2 = len(coords)\n if not silent:\n msg = \"{0} coordinates\".format(l2)\n if l2 < l1:\n msg += \" ({0} removed as invalid)\".format(l1-l2)\n print(msg)\n return coords"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef offset_mask(mask):\n def axis_data(axis):\n \"\"\"Gets the bounds of a masked area along a certain axis\"\"\"\n x = mask.sum(axis)\n trimmed_front = N.trim_zeros(x,\"f\")\n offset = len(x)-len(trimmed_front)\n size = len(N.trim_zeros(trimmed_front,\"b\"))\n return offset,size\n\n xo,xs = axis_data(0)\n yo,ys = axis_data(1)\n\n array = mask[yo:yo+ys,xo:xo+xs]\n offset = (yo,xo)\n return offset, array", "response": "Returns a mask shrunk to the minimum bounding rectangle of the nonzero portion of the previous mask and its offset from the original mask."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef extract_line(geom, dem, **kwargs):\n kwargs.setdefault('masked', True)\n\n coords_in = coords_array(geom)\n\n # Transform geometry into pixels\n f = lambda *x: ~dem.transform * x\n px = transform(f,geom)\n\n # Subdivide geometry segments if option is given\n interval = kwargs.pop('subdivide', 1)\n if interval is not None:\n px = subdivide(px, interval=interval)\n\n # Transform pixels back to geometry\n # to capture subdivisions\n f = lambda *x: dem.transform * (x[0],x[1])\n geom = transform(f,px)\n\n # Get min and max coords for windowing\n # Does not deal with edge cases where points\n # are outside of footprint of DEM\n\n coords_px = coords_array(px)\n mins = N.floor(coords_px.min(axis=0))\n maxs = N.ceil(coords_px.max(axis=0))\n\n window = tuple((int(mn),int(mx))\n for mn,mx in zip(mins[::-1],maxs[::-1]))\n\n aff = Affine.translation(*(-mins))\n\n f = lambda *x: aff * x\n px_to_extract = transform(f,px)\n\n band = dem.read(1, window=window, **kwargs)\n extracted = bilinear(band, px_to_extract)\n coords = coords_array(extracted)\n\n coords[:,:2] = coords_array(geom)\n return coords", "response": "Extract a linear feature from a rasterio geospatial dataset."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nfork a new node process and returns a Deferred that resolves when the child process finishes.", "response": "def fork(executable, args=(), env={}, path=None, timeout=3600):\n \"\"\"fork\n Provides a deferred wrapper function with a timeout function\n\n :param executable: Executable\n :type executable: str.\n :param args: Tupple of arguments\n :type args: tupple.\n :param env: Environment dictionary\n :type env: dict.\n :param timeout: Kill the child process if timeout is exceeded\n :type timeout: int.\n \"\"\"\n d = defer.Deferred()\n p = ProcessProtocol(d, timeout)\n reactor.spawnProcess(p, executable, (executable,)+tuple(args), env, path)\n return d"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef make_grammar(dialect: str) -> SqlGrammar:\n if dialect == SqlaDialectName.MYSQL:\n return mysql_grammar\n elif dialect == SqlaDialectName.MSSQL:\n return mssql_grammar\n else:\n raise AssertionError(\"Invalid SQL dialect: {}\".format(repr(dialect)))", "response": "Returns an instance of the appropriate SqlGrammar class for the given SQL dialect."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef number_to_dp(number: Optional[float],\n dp: int,\n default: Optional[str] = \"\",\n en_dash_for_minus: bool = True) -> str:\n \"\"\"\n Format number to ``dp`` decimal places, optionally using a UTF-8 en dash\n for minus signs.\n \"\"\"\n if number is None:\n return default\n if number == float(\"inf\"):\n return u\"\u221e\"\n if number == float(\"-inf\"):\n s = u\"-\u221e\"\n else:\n s = u\"{:.{precision}f}\".format(number, precision=dp)\n if en_dash_for_minus:\n s = s.replace(\"-\", u\"\u2013\") # hyphen becomes en dash for minus sign\n return s", "response": "Format number to dp decimal places."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef debug_form_contents(form: cgi.FieldStorage,\n to_stderr: bool = True,\n to_logger: bool = False) -> None:\n \"\"\"\n Writes the keys and values of a CGI form to ``stderr``.\n \"\"\"\n for k in form.keys():\n text = \"{0} = {1}\".format(k, form.getvalue(k))\n if to_stderr:\n sys.stderr.write(text)\n if to_logger:\n log.info(text)", "response": "Writes the keys and values of a CGI form to stderr."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef cgi_method_is_post(environ: Dict[str, str]) -> bool:\n method = environ.get(\"REQUEST_METHOD\", None)\n if not method:\n return False\n return method.upper() == \"POST\"", "response": "Determines if the CGI method is POST."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef get_cgi_parameter_str(form: cgi.FieldStorage,\n key: str,\n default: str = None) -> str:\n \"\"\"\n Extracts a string parameter from a CGI form.\n Note: ``key`` is CASE-SENSITIVE.\n \"\"\"\n paramlist = form.getlist(key)\n if len(paramlist) == 0:\n return default\n return paramlist[0]", "response": "Extracts a string parameter from a CGI form."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nextracts a string parameter from a CGI form or None if the key doesn t exist.", "response": "def get_cgi_parameter_str_or_none(form: cgi.FieldStorage,\n key: str) -> Optional[str]:\n \"\"\"\n Extracts a string parameter from a CGI form, or ``None`` if the key doesn't\n exist or the string is zero-length.\n \"\"\"\n s = get_cgi_parameter_str(form, key)\n if s is None or len(s) == 0:\n return None\n return s"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nextract a list of values all with the same key from a CGI form.", "response": "def get_cgi_parameter_list(form: cgi.FieldStorage, key: str) -> List[str]:\n \"\"\"\n Extracts a list of values, all with the same key, from a CGI form.\n \"\"\"\n return form.getlist(key)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef get_cgi_parameter_bool(form: cgi.FieldStorage, key: str) -> bool:\n return is_1(get_cgi_parameter_str(form, key))", "response": "Extracts a boolean parameter from a CGI form."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nextracting a boolean parameter from a CGI form or returns default if the key is not found.", "response": "def get_cgi_parameter_bool_or_default(form: cgi.FieldStorage,\n key: str,\n default: bool = None) -> Optional[bool]:\n \"\"\"\n Extracts a boolean parameter from a CGI form (``\"1\"`` = ``True``,\n other string = ``False``, absent/zero-length string = default value).\n \"\"\"\n s = get_cgi_parameter_str(form, key)\n if s is None or len(s) == 0:\n return default\n return is_1(s)"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef get_cgi_parameter_bool_or_none(form: cgi.FieldStorage,\n key: str) -> Optional[bool]:\n \"\"\"\n Extracts a boolean parameter from a CGI form (``\"1\"`` = ``True``,\n other string = False, absent/zero-length string = ``None``).\n \"\"\"\n return get_cgi_parameter_bool_or_default(form, key, default=None)", "response": "Extracts a boolean parameter from a CGI form or returns None if the key is not found."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef get_cgi_parameter_int(form: cgi.FieldStorage, key: str) -> Optional[int]:\n return get_int_or_none(get_cgi_parameter_str(form, key))", "response": "Extracts an integer parameter from a CGI form or None if the key is absent or the string value is not convertible to int."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nextract a float parameter from a CGI form or None if the key is absent.", "response": "def get_cgi_parameter_float(form: cgi.FieldStorage,\n key: str) -> Optional[float]:\n \"\"\"\n Extracts a float parameter from a CGI form, or None if the key is\n absent or the string value is not convertible to ``float``.\n \"\"\"\n return get_float_or_none(get_cgi_parameter_str(form, key))"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef get_cgi_parameter_datetime(form: cgi.FieldStorage,\n key: str) -> Optional[datetime.datetime]:\n \"\"\"\n Extracts a date/time parameter from a CGI form. Applies the LOCAL\n timezone if none specified.\n \"\"\"\n try:\n s = get_cgi_parameter_str(form, key)\n if not s:\n # if you dateutil.parser.parse() an empty string,\n # you get today's date\n return None\n d = dateutil.parser.parse(s)\n if d.tzinfo is None: # as it will be\n d = d.replace(tzinfo=dateutil.tz.tzlocal())\n return d\n except ValueError:\n return None", "response": "Extracts a date or time parameter from a CGI form. Applies the LOCAL\nCTYPE timezone if none specified."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef get_cgi_parameter_file(form: cgi.FieldStorage,\n key: str) -> Optional[bytes]:\n \"\"\"\n Extracts a file's contents from a \"file\" input in a CGI form, or None\n if no such file was uploaded.\n \"\"\"\n (filename, filecontents) = get_cgi_parameter_filename_and_file(form, key)\n return filecontents", "response": "Extracts a file s contents from a file input in a CGI form."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nreturns a filename and contents of a file input in a CGI form.", "response": "def get_cgi_parameter_filename_and_file(form: cgi.FieldStorage, key: str) \\\n -> Tuple[Optional[str], Optional[bytes]]:\n \"\"\"\n Extracts a file's name and contents from a \"file\" input in a CGI form.\n Returns ``(name, contents)``, or ``(None, None)`` if no such file was\n uploaded.\n \"\"\"\n if not (key in form):\n log.warning('get_cgi_parameter_file: form has no key {}', key)\n return None, None\n fileitem = form[key] # a nested FieldStorage instance; see\n # http://docs.python.org/2/library/cgi.html#using-the-cgi-module\n if isinstance(fileitem, cgi.MiniFieldStorage):\n log.warning('get_cgi_parameter_file: MiniFieldStorage found - did you '\n 'forget to set enctype=\"multipart/form-data\" in '\n 'your form?')\n return None, None\n if not isinstance(fileitem, cgi.FieldStorage):\n log.warning('get_cgi_parameter_file: no FieldStorage instance with '\n 'key {} found', key)\n return None, None\n if fileitem.filename and fileitem.file: # can check \"file\" or \"filename\"\n return fileitem.filename, fileitem.file.read()\n # as per\n # http://upsilon.cc/~zack/teaching/0607/techweb/02-python-cgi.pdf\n # Alternative:\n # return get_cgi_parameter_str(form, key) # contents of the file\n # Otherwise, information about problems:\n if not fileitem.file:\n log.warning('get_cgi_parameter_file: fileitem has no file')\n elif not fileitem.filename:\n log.warning('get_cgi_parameter_file: fileitem has no filename')\n else:\n log.warning('get_cgi_parameter_file: unknown failure reason')\n return None, None"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef cgi_parameter_exists(form: cgi.FieldStorage, key: str) -> bool:\n s = get_cgi_parameter_str(form, key)\n return s is not None", "response": "Does a CGI form contain the key?"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nreturns an environment variable s value CGI - escaped or None.", "response": "def getenv_escaped(key: str, default: str = None) -> Optional[str]:\n \"\"\"\n Returns an environment variable's value, CGI-escaped, or ``None``.\n \"\"\"\n value = os.getenv(key, default)\n # noinspection PyDeprecation\n return cgi.escape(value) if value is not None else None"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nreturning a CGI - escaped version of the value read from an INI file using .", "response": "def getconfigvar_escaped(config: configparser.ConfigParser,\n section: str,\n key: str) -> Optional[str]:\n \"\"\"\n Returns a CGI-escaped version of the value read from an INI file using\n :class:`ConfigParser`, or ``None``.\n \"\"\"\n value = config.get(section, key)\n # noinspection PyDeprecation\n return cgi.escape(value) if value is not None else None"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef get_cgi_fieldstorage_from_wsgi_env(\n env: Dict[str, str],\n include_query_string: bool = True) -> cgi.FieldStorage:\n \"\"\"\n Returns a :class:`cgi.FieldStorage` object from the WSGI environment.\n \"\"\"\n # http://stackoverflow.com/questions/530526/accessing-post-data-from-wsgi\n post_env = env.copy()\n if not include_query_string:\n post_env['QUERY_STRING'] = ''\n form = cgi.FieldStorage(\n fp=env['wsgi.input'],\n environ=post_env,\n keep_blank_values=True\n )\n return form", "response": "Returns a cgi. FieldStorage object from the WSGI environment."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nconvert a PNG blob into a local URL encapsulating the PNG.", "response": "def get_png_data_url(blob: Optional[bytes]) -> str:\n \"\"\"\n Converts a PNG blob into a local URL encapsulating the PNG.\n \"\"\"\n return BASE64_PNG_URL_PREFIX + base64.b64encode(blob).decode('ascii')"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nconverts a PNG blob to an HTML IMG tag with embedded data.", "response": "def get_png_img_html(blob: Union[bytes, memoryview],\n extra_html_class: str = None) -> str:\n \"\"\"\n Converts a PNG blob to an HTML IMG tag with embedded data.\n \"\"\"\n return \"\"\"\"\"\".format(\n 'class=\"{}\" '.format(extra_html_class) if extra_html_class else \"\",\n get_png_data_url(blob)\n )"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef pdf_result(pdf_binary: bytes,\n extraheaders: TYPE_WSGI_RESPONSE_HEADERS = None,\n filename: str = None) -> WSGI_TUPLE_TYPE:\n \"\"\"\n Returns ``(contenttype, extraheaders, data)`` tuple for a PDF.\n \"\"\"\n extraheaders = extraheaders or []\n if filename:\n extraheaders.append(\n ('content-disposition', 'inline; filename=\"{}\"'.format(filename))\n )\n contenttype = 'application/pdf'\n if filename:\n contenttype += '; filename=\"{}\"'.format(filename)\n # log.debug(\"type(pdf_binary): {}\", type(pdf_binary))\n return contenttype, extraheaders, pdf_binary", "response": "Returns a WSGI response object for a PDF."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef zip_result(zip_binary: bytes,\n extraheaders: TYPE_WSGI_RESPONSE_HEADERS = None,\n filename: str = None) -> WSGI_TUPLE_TYPE:\n \"\"\"\n Returns ``(contenttype, extraheaders, data)`` tuple for a ZIP.\n \"\"\"\n extraheaders = extraheaders or []\n if filename:\n extraheaders.append(\n ('content-disposition', 'inline; filename=\"{}\"'.format(filename))\n )\n contenttype = 'application/zip'\n if filename:\n contenttype += '; filename=\"{}\"'.format(filename)\n return contenttype, extraheaders, zip_binary", "response": "Returns a WSGI response body for a ZIP file."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nreturn a WSGI response tuple for UTF - 8 HTML.", "response": "def html_result(html: str,\n extraheaders: TYPE_WSGI_RESPONSE_HEADERS = None) \\\n -> WSGI_TUPLE_TYPE:\n \"\"\"\n Returns ``(contenttype, extraheaders, data)`` tuple for UTF-8 HTML.\n \"\"\"\n extraheaders = extraheaders or []\n return 'text/html; charset=utf-8', extraheaders, html.encode(\"utf-8\")"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nreturn a WSGI response object for the XML.", "response": "def xml_result(xml: str,\n extraheaders: TYPE_WSGI_RESPONSE_HEADERS = None) \\\n -> WSGI_TUPLE_TYPE:\n \"\"\"\n Returns ``(contenttype, extraheaders, data)`` tuple for UTF-8 XML.\n \"\"\"\n extraheaders = extraheaders or []\n return 'text/xml; charset=utf-8', extraheaders, xml.encode(\"utf-8\")"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nreturns WSGI response body for UTF - 8 text.", "response": "def text_result(text: str,\n extraheaders: TYPE_WSGI_RESPONSE_HEADERS = None,\n filename: str = None) -> WSGI_TUPLE_TYPE:\n \"\"\"\n Returns ``(contenttype, extraheaders, data)`` tuple for UTF-8 text.\n \"\"\"\n extraheaders = extraheaders or []\n if filename:\n extraheaders.append(\n ('content-disposition', 'inline; filename=\"{}\"'.format(filename))\n )\n contenttype = 'text/plain; charset=utf-8'\n if filename:\n contenttype += '; filename=\"{}\"'.format(filename)\n return contenttype, extraheaders, text.encode(\"utf-8\")"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nprints the result of a CGI - script to stdout.", "response": "def print_result_for_plain_cgi_script_from_tuple(\n contenttype_headers_content: WSGI_TUPLE_TYPE,\n status: str = '200 OK') -> None:\n \"\"\"\n Writes HTTP result to stdout.\n\n Args:\n contenttype_headers_content:\n the tuple ``(contenttype, extraheaders, data)``\n status:\n HTTP status message (default ``\"200 OK``)\n \"\"\"\n contenttype, headers, content = contenttype_headers_content\n print_result_for_plain_cgi_script(contenttype, headers, content, status)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef print_result_for_plain_cgi_script(contenttype: str,\n headers: TYPE_WSGI_RESPONSE_HEADERS,\n content: bytes,\n status: str = '200 OK') -> None:\n \"\"\"\n Writes HTTP request result to stdout.\n \"\"\"\n headers = [\n (\"Status\", status),\n (\"Content-Type\", contenttype),\n (\"Content-Length\", str(len(content))),\n ] + headers\n sys.stdout.write(\"\\n\".join([h[0] + \": \" + h[1] for h in headers]) + \"\\n\\n\")\n sys.stdout.write(content)", "response": "Writes HTTP request result to stdout."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef wsgi_simple_responder(\n result: Union[str, bytes],\n handler: Callable[[Union[str, bytes]], WSGI_TUPLE_TYPE],\n start_response: TYPE_WSGI_START_RESPONSE,\n status: str = '200 OK',\n extraheaders: TYPE_WSGI_RESPONSE_HEADERS = None) \\\n -> TYPE_WSGI_APP_RESULT:\n \"\"\"\n Simple WSGI app.\n\n Args:\n result: the data to be processed by ``handler``\n handler: a function returning a ``(contenttype, extraheaders, data)``\n tuple, e.g. ``text_result``, ``html_result``\n start_response: standard WSGI ``start_response`` function\n status: status code (default ``\"200 OK\"``)\n extraheaders: optional extra HTTP headers\n\n Returns:\n WSGI application result\n\n \"\"\"\n extraheaders = extraheaders or []\n (contenttype, extraheaders2, output) = handler(result)\n response_headers = [('Content-Type', contenttype),\n ('Content-Length', str(len(output)))]\n response_headers.extend(extraheaders)\n if extraheaders2 is not None:\n response_headers.extend(extraheaders2)\n # noinspection PyArgumentList\n start_response(status, response_headers)\n return [output]", "response": "Simple WSGI response handler."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef webify(v: Any, preserve_newlines: bool = True) -> str:\n nl = \"
\" if preserve_newlines else \" \"\n if v is None:\n return \"\"\n if not isinstance(v, str):\n v = str(v)\n # noinspection PyDeprecation\n return cgi.escape(v).replace(\"\\n\", nl).replace(\"\\\\n\", nl)", "response": "Converts a value into an HTML - safe string."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nreturning a HTML - emboldens content unless blank.", "response": "def bold_if_not_blank(x: Optional[str]) -> str:\n \"\"\"\n HTML-emboldens content, unless blank.\n \"\"\"\n if x is None:\n return u\"{}\".format(x)\n return u\"{}\".format(x)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef make_urls_hyperlinks(text: str) -> str:\n find_url = r'''\n (?x)( # verbose identify URLs within text\n (http|ftp|gopher) # make sure we find a resource type\n :// # ...needs to be followed by colon-slash-slash\n (\\w+[:.]?){2,} # at least two domain groups, e.g. (gnosis.)(cx)\n (/?| # could be just the domain name (maybe w/ slash)\n [^ \\n\\r\"]+ # or stuff then space, newline, tab, quote\n [\\w/]) # resource name ends in alphanumeric or slash\n (?=[\\s\\.,>)'\"\\]]) # assert: followed by white or clause ending\n ) # end of match group\n '''\n replace_url = r'\\1'\n find_email = re.compile(r'([.\\w\\-]+@(\\w[\\w\\-]+\\.)+[\\w\\-]+)')\n # '.' doesn't need escaping inside square brackets\n # https://stackoverflow.com/questions/10397968/escape-dot-in-a-regex-range\n replace_email = r'\\1'\n text = re.sub(find_url, replace_url, text)\n text = re.sub(find_email, replace_email, text)\n return text", "response": "Makes hyperlinks to the text that appears to contain URLs."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef html_table_from_query(rows: Iterable[Iterable[Optional[str]]],\n descriptions: Iterable[Optional[str]]) -> str:\n \"\"\"\n Converts rows from an SQL query result to an HTML table.\n Suitable for processing output from the defunct function\n ``rnc_db.fetchall_with_fieldnames(sql)``.\n \"\"\"\n html = u\"\\n\"\n\n # Header row\n html += u\"\"\n for x in descriptions:\n if x is None:\n x = u\"\"\n html += u\"\".format(webify(x))\n html += u\"\\n\"\n\n # Data rows\n for row in rows:\n html += u\"\"\n for x in row:\n if x is None:\n x = u\"\"\n html += u\"\".format(webify(x))\n html += u\"\\n\"\n\n html += u\"
{}
{}
\\n\"\n return html", "response": "Converts rows from an SQL query result to an HTML table."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nunderlines a heading for RST files.", "response": "def rst_underline(heading: str, underline_char: str) -> str:\n \"\"\"\n Underlines a heading for RST files.\n\n Args:\n heading: text to underline\n underline_char: character to use\n\n Returns:\n underlined heading, over two lines (without a final terminating\n newline)\n \"\"\"\n assert \"\\n\" not in heading\n assert len(underline_char) == 1\n return heading + \"\\n\" + (underline_char * len(heading))"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nwriting the contents to a file if permitted.", "response": "def write_if_allowed(filename: str,\n content: str,\n overwrite: bool = False,\n mock: bool = False) -> None:\n \"\"\"\n Writes the contents to a file, if permitted.\n\n Args:\n filename: filename to write\n content: contents to write\n overwrite: permit overwrites?\n mock: pretend to write, but don't\n\n Raises:\n RuntimeError: if file exists but overwriting not permitted\n \"\"\"\n # Check we're allowed\n if not overwrite and exists(filename):\n fail(\"File exists, not overwriting: {!r}\".format(filename))\n\n # Make the directory, if necessary\n directory = dirname(filename)\n if not mock:\n mkdir_p(directory)\n\n # Write the file\n log.info(\"Writing to {!r}\", filename)\n if mock:\n log.warning(\"Skipping writes as in mock mode\")\n else:\n with open(filename, \"wt\") as outfile:\n outfile.write(content)"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nreturns the filename of the target RST file relative to a specified index file. Used to make the index refer to the RST.", "response": "def rst_filename_rel_autodoc_index(self, index_filename: str) -> str:\n \"\"\"\n Returns the filename of the target RST file, relative to a specified\n index file. Used to make the index refer to the RST.\n \"\"\"\n index_dir = dirname(abspath(expanduser(index_filename)))\n return relpath(self.target_rst_filename, start=index_dir)"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef python_module_name(self) -> str:\n if not self.is_python:\n return \"\"\n filepath = self.source_filename_rel_python_root\n dirs_and_base = splitext(filepath)[0]\n dir_and_file_parts = dirs_and_base.split(sep)\n return \".\".join(dir_and_file_parts)", "response": "Returns the name of the Python module that this instance refers to."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nreturn the code type annotation for Pygments ; e. g. python for Python cpp for C ++ etc.", "response": "def pygments_language(self) -> str:\n \"\"\"\n Returns the code type annotation for Pygments; e.g. ``python`` for\n Python, ``cpp`` for C++, etc.\n \"\"\"\n extension = splitext(self.source_filename)[1]\n if extension in self.pygments_language_override:\n return self.pygments_language_override[extension]\n try:\n lexer = get_lexer_for_filename(self.source_filename) # type: Lexer\n return lexer.name\n except ClassNotFound:\n log.warning(\"Don't know Pygments code type for extension {!r}\",\n self.source_extension)\n return CODE_TYPE_NONE"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef rst_content(self,\n prefix: str = \"\",\n suffix: str = \"\",\n heading_underline_char: str = \"=\",\n method: AutodocMethod = None) -> str:\n \"\"\"\n Returns the text contents of an RST file that will automatically\n document our source file.\n\n Args:\n prefix: prefix, e.g. RST copyright comment\n suffix: suffix, after the part we're creating\n heading_underline_char: RST character to use to underline the\n heading\n method: optional method to override ``self.method``; see\n constructor\n\n Returns:\n the RST contents\n \"\"\"\n spacer = \" \"\n # Choose our final method\n if method is None:\n method = self.method\n is_python = self.is_python\n if method == AutodocMethod.BEST:\n if is_python:\n method = AutodocMethod.AUTOMODULE\n else:\n method = AutodocMethod.CONTENTS\n elif method == AutodocMethod.AUTOMODULE:\n if not is_python:\n method = AutodocMethod.CONTENTS\n\n # Write the instruction\n if method == AutodocMethod.AUTOMODULE:\n if self.source_rst_title_style_python:\n title = self.python_module_name\n else:\n title = self.source_filename_rel_project_root\n instruction = \".. automodule:: {modulename}\\n :members:\".format(\n modulename=self.python_module_name\n )\n elif method == AutodocMethod.CONTENTS:\n title = self.source_filename_rel_project_root\n # Using \".. include::\" with options like \":code: python\" doesn't\n # work properly; everything comes out as Python.\n # Instead, see http://www.sphinx-doc.org/en/1.4.9/markup/code.html;\n # we need \".. literalinclude::\" with \":language: LANGUAGE\".\n\n instruction = (\n \".. literalinclude:: {filename}\\n\"\n \"{spacer}:language: {language}\".format(\n filename=self.source_filename_rel_rst_file,\n spacer=spacer,\n language=self.pygments_language\n )\n )\n else:\n raise ValueError(\"Bad method!\")\n\n # Create the whole file\n content = \"\"\"\n.. {filename}\n \n{AUTOGENERATED_COMMENT}\n\n{prefix}\n\n{underlined_title}\n\n{instruction}\n\n{suffix}\n \"\"\".format(\n filename=self.rst_filename_rel_project_root,\n AUTOGENERATED_COMMENT=AUTOGENERATED_COMMENT,\n prefix=prefix,\n underlined_title=rst_underline(\n title, underline_char=heading_underline_char),\n instruction=instruction,\n suffix=suffix,\n ).strip() + \"\\n\"\n return content", "response": "Returns the contents of an RST file that will automatically create a new object."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nwrite the RST file to our destination RST filename.", "response": "def write_rst(self,\n prefix: str = \"\",\n suffix: str = \"\",\n heading_underline_char: str = \"=\",\n method: AutodocMethod = None,\n overwrite: bool = False,\n mock: bool = False) -> None:\n \"\"\"\n Writes the RST file to our destination RST filename, making any\n necessary directories.\n\n Args:\n prefix: as for :func:`rst_content`\n suffix: as for :func:`rst_content`\n heading_underline_char: as for :func:`rst_content`\n method: as for :func:`rst_content`\n overwrite: overwrite the file if it exists already?\n mock: pretend to write, but don't\n \"\"\"\n content = self.rst_content(\n prefix=prefix,\n suffix=suffix,\n heading_underline_char=heading_underline_char,\n method=method\n )\n write_if_allowed(self.target_rst_filename, content,\n overwrite=overwrite, mock=mock)"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nadds source files to the index.", "response": "def add_source_files(\n self,\n source_filenames_or_globs: Union[str, List[str]],\n method: AutodocMethod = None,\n recursive: bool = None,\n source_rst_title_style_python: bool = None,\n pygments_language_override: Dict[str, str] = None) -> None:\n \"\"\"\n Adds source files to the index.\n\n Args:\n source_filenames_or_globs: string containing a filename or a\n glob, describing the file(s) to be added, or a list of such\n strings\n method: optional method to override ``self.method``\n recursive: use :func:`glob.glob` in recursive mode? (If ``None``,\n the default, uses the version from the constructor.)\n source_rst_title_style_python: optional to override\n ``self.source_rst_title_style_python``\n pygments_language_override: optional to override\n ``self.pygments_language_override``\n \"\"\"\n if not source_filenames_or_globs:\n return\n\n if method is None:\n # Use the default\n method = self.method\n if recursive is None:\n recursive = self.recursive\n if source_rst_title_style_python is None:\n source_rst_title_style_python = self.source_rst_title_style_python\n if pygments_language_override is None:\n pygments_language_override = self.pygments_language_override\n\n # Get a sorted list of filenames\n final_filenames = self.get_sorted_source_files(\n source_filenames_or_globs,\n recursive=recursive\n )\n\n # Process that sorted list\n for source_filename in final_filenames:\n self.files_to_index.append(FileToAutodocument(\n source_filename=source_filename,\n project_root_dir=self.project_root_dir,\n python_package_root_dir=self.python_package_root_dir,\n target_rst_filename=self.specific_file_rst_filename(\n source_filename\n ),\n method=method,\n source_rst_title_style_python=source_rst_title_style_python,\n pygments_language_override=pygments_language_override,\n ))"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nreturn a sorted list of filenames to process from a filename glob string or a list of filenames or globs.", "response": "def get_sorted_source_files(\n self,\n source_filenames_or_globs: Union[str, List[str]],\n recursive: bool = True) -> List[str]:\n \"\"\"\n Returns a sorted list of filenames to process, from a filename,\n a glob string, or a list of filenames/globs.\n\n Args:\n source_filenames_or_globs: filename/glob, or list of them\n recursive: use :func:`glob.glob` in recursive mode?\n\n Returns:\n sorted list of files to process\n \"\"\"\n if isinstance(source_filenames_or_globs, str):\n source_filenames_or_globs = [source_filenames_or_globs]\n final_filenames = [] # type: List[str]\n for sfg in source_filenames_or_globs:\n sfg_expanded = expanduser(sfg)\n log.debug(\"Looking for: {!r}\", sfg_expanded)\n for filename in glob.glob(sfg_expanded, recursive=recursive):\n log.debug(\"Trying: {!r}\", filename)\n if self.should_exclude(filename):\n log.info(\"Skipping file {!r}\", filename)\n continue\n final_filenames.append(filename)\n final_filenames.sort()\n return final_filenames"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nreturn True if the filename matches the globtext.", "response": "def filename_matches_glob(filename: str, globtext: str) -> bool:\n \"\"\"\n The ``glob.glob`` function doesn't do exclusion very well. We don't\n want to have to specify root directories for exclusion patterns. We\n don't want to have to trawl a massive set of files to find exclusion\n files. So let's implement a glob match.\n\n Args:\n filename: filename\n globtext: glob\n\n Returns:\n does the filename match the glob?\n\n See also:\n\n - https://stackoverflow.com/questions/20638040/glob-exclude-pattern\n\n \"\"\"\n # Quick check on basename-only matching\n if fnmatch(filename, globtext):\n log.debug(\"{!r} matches {!r}\", filename, globtext)\n return True\n bname = basename(filename)\n if fnmatch(bname, globtext):\n log.debug(\"{!r} matches {!r}\", bname, globtext)\n return True\n # Directory matching: is actually accomplished by the code above!\n # Otherwise:\n return False"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef should_exclude(self, filename) -> bool:\n for skip_glob in self.skip_globs:\n if self.filename_matches_glob(filename, skip_glob):\n return True\n return False", "response": "Returns True if this file should be excluded."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nreturn the RST filename corresponding to a source filename.", "response": "def specific_file_rst_filename(self, source_filename: str) -> str:\n \"\"\"\n Gets the RST filename corresponding to a source filename.\n See the help for the constructor for more details.\n\n Args:\n source_filename: source filename within current project\n\n Returns:\n RST filename\n\n Note in particular: the way we structure the directories means that we\n won't get clashes between files with idential names in two different\n directories. However, we must also incorporate the original source\n filename, in particular for C++ where ``thing.h`` and ``thing.cpp``\n must not generate the same RST filename. So we just add ``.rst``.\n \"\"\"\n highest_code_to_target = relative_filename_within_dir(\n source_filename, self.highest_code_dir)\n bname = basename(source_filename)\n result = join(self.autodoc_rst_root_dir,\n dirname(highest_code_to_target),\n bname + EXT_RST)\n log.debug(\"Source {!r} -> RST {!r}\", source_filename, result)\n return result"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nwrite both the individual RST files and the index.", "response": "def write_index_and_rst_files(self, overwrite: bool = False,\n mock: bool = False) -> None:\n \"\"\"\n Writes both the individual RST files and the index.\n\n Args:\n overwrite: allow existing files to be overwritten?\n mock: pretend to write, but don't\n \"\"\"\n for f in self.files_to_index:\n if isinstance(f, FileToAutodocument):\n f.write_rst(\n prefix=self.rst_prefix,\n suffix=self.rst_suffix,\n heading_underline_char=self.source_rst_heading_underline_char, # noqa\n overwrite=overwrite,\n mock=mock,\n )\n elif isinstance(f, AutodocIndex):\n f.write_index_and_rst_files(overwrite=overwrite, mock=mock)\n else:\n fail(\"Unknown thing in files_to_index: {!r}\".format(f))\n self.write_index(overwrite=overwrite, mock=mock)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef index_filename_rel_other_index(self, other: str) -> str:\n return relpath(self.index_filename, start=dirname(other))", "response": "Returns the filename of this index relative to the director of another index."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nreturning the contents of the index RST file.", "response": "def index_content(self) -> str:\n \"\"\"\n Returns the contents of the index RST file.\n \"\"\"\n # Build the toctree command\n index_filename = self.index_filename\n spacer = \" \"\n toctree_lines = [\n \".. toctree::\",\n spacer + \":maxdepth: {}\".format(self.toctree_maxdepth),\n \"\"\n ]\n for f in self.files_to_index:\n if isinstance(f, FileToAutodocument):\n rst_filename = spacer + f.rst_filename_rel_autodoc_index(\n index_filename)\n elif isinstance(f, AutodocIndex):\n rst_filename = (\n spacer + f.index_filename_rel_other_index(index_filename)\n )\n else:\n fail(\"Unknown thing in files_to_index: {!r}\".format(f))\n rst_filename = \"\" # won't get here; for the type checker\n toctree_lines.append(rst_filename)\n toctree = \"\\n\".join(toctree_lines)\n\n # Create the whole file\n content = \"\"\"\n.. {filename}\n\n{AUTOGENERATED_COMMENT}\n\n{prefix}\n\n{underlined_title}\n\n{introductory_rst}\n\n{toctree}\n\n{suffix}\n \"\"\".format(\n filename=self.index_filename_rel_project_root,\n AUTOGENERATED_COMMENT=AUTOGENERATED_COMMENT,\n prefix=self.rst_prefix,\n underlined_title=rst_underline(\n self.title, underline_char=self.index_heading_underline_char),\n introductory_rst=self.introductory_rst,\n toctree=toctree,\n suffix=self.rst_suffix,\n ).strip() + \"\\n\"\n return content"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nwrite the index file if permitted.", "response": "def write_index(self, overwrite: bool = False, mock: bool = False) -> None:\n \"\"\"\n Writes the index file, if permitted.\n\n Args:\n overwrite: allow existing files to be overwritten?\n mock: pretend to write, but don't\n \"\"\"\n write_if_allowed(self.index_filename, self.index_content(),\n overwrite=overwrite, mock=mock)"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nparse the hosts parameter and return the full host and authentication if exists", "response": "def parse_host(hosts):\n \"\"\"\n Parsing the hosts parameter,\n * currently support only one host\n :param hosts: the hosts json to parse\n :return: the full host and the authentication if exists\n \"\"\"\n hosts = EsParser._normalize_hosts(hosts)\n host = hosts[0]\n\n host_name = host[HostParsing.HOST]\n host_port = host[HostParsing.PORT]\n\n auth = None\n if HostParsing.HTTP_AUTH in host:\n http_auth = host[HostParsing.HTTP_AUTH]\n user_pass = http_auth.split(':')\n auth = (user_pass[0], user_pass[1])\n\n full_host = \"{host}:{port}\".format(host=host_name, port=host_port)\n if not host_name.startswith((HostParsing.HTTP + ':', HostParsing.HTTPS + ':')):\n scheme = HostParsing.HTTPS if host.get(HostParsing.USE_SSL) else HostParsing.HTTP\n full_host = \"{scheme}://{full_host}\".format(full_host=full_host, scheme=scheme)\n\n return full_host, auth"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nupdates the host ssl params if needed.", "response": "def _update_ssl_params(host):\n \"\"\"\n Update the host ssl params (port or scheme) if needed.\n :param host:\n :return:\n \"\"\"\n if host[HostParsing.HOST] \\\n and EsParser._is_secure_connection_type(host):\n host[HostParsing.PORT] = EsParser.SSL_DEFAULT_PORT\n host[HostParsing.USE_SSL] = True\n parsed_url = urlparse(EsParser._fix_host_prefix(host[HostParsing.HOST]))\n host[HostParsing.HOST] = parsed_url.hostname\n host[HostParsing.SCHEME] = HostParsing.HTTPS\n return host"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nparse a host string into a dictionary host", "response": "def _parse_string_host(host_str):\n \"\"\"\n Parse host string into a dictionary host\n :param host_str:\n :return:\n \"\"\"\n host_str = EsParser._fix_host_prefix(host_str)\n parsed_url = urlparse(host_str)\n host = {HostParsing.HOST: parsed_url.hostname}\n if parsed_url.port:\n host[HostParsing.PORT] = parsed_url.port\n if parsed_url.scheme == HostParsing.HTTPS:\n host[HostParsing.PORT] = parsed_url.port or EsParser.SSL_DEFAULT_PORT\n host[HostParsing.USE_SSL] = True\n host[HostParsing.SCHEME] = HostParsing.HTTPS\n elif parsed_url.scheme:\n host[HostParsing.SCHEME] = parsed_url.scheme\n if parsed_url.username or parsed_url.password:\n host[HostParsing.HTTP_AUTH] = '%s:%s' % (parsed_url.username, parsed_url.password)\n if parsed_url.path and parsed_url.path != '/':\n host[HostParsing.URL_PREFIX] = parsed_url.path\n return host"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\ncreates a path from a list of sub paths.", "response": "def make_path(*sub_paths):\n \"\"\"\n Create a path from a list of sub paths.\n :param sub_paths: a list of sub paths\n :return:\n \"\"\"\n queued_params = [quote(c.encode('utf-8'), '') for c in sub_paths if c not in NULL_VALUES]\n queued_params.insert(0, '')\n return '/'.join(queued_params)"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef prepare_url(hostname, path, params=None):\n url = hostname + path\n\n if params:\n url = url + '?' + urlencode(params)\n\n if not url.startswith(('http:', 'https:')):\n url = \"http://\" + url\n\n return url.encode('utf-8')", "response": "Prepare Elasticsearch request url."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef basic_animation(frames=100, interval=30):\n fig = plt.figure()\n ax = plt.axes(xlim=(0, 10), ylim=(-2, 2))\n line, = ax.plot([], [], lw=2)\n\n x = np.linspace(0, 10, 1000)\n\n def init():\n line.set_data([], [])\n return line,\n\n def animate(i):\n y = np.cos(i * 0.02 * np.pi) * np.sin(x - i * 0.02 * np.pi)\n line.set_data(x, y)\n return line,\n\n return animation.FuncAnimation(fig, animate, init_func=init,\n frames=frames, interval=interval)", "response": "Plot a basic sine wave with oscillating amplitude"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nplot a 3D visualization of the dynamics of the Lorentz system.", "response": "def lorenz_animation(N_trajectories=20, rseed=1, frames=200, interval=30):\n \"\"\"Plot a 3D visualization of the dynamics of the Lorenz system\"\"\"\n from scipy import integrate\n from mpl_toolkits.mplot3d import Axes3D\n from matplotlib.colors import cnames\n\n def lorentz_deriv(coords, t0, sigma=10., beta=8./3, rho=28.0):\n \"\"\"Compute the time-derivative of a Lorentz system.\"\"\"\n x, y, z = coords\n return [sigma * (y - x), x * (rho - z) - y, x * y - beta * z]\n\n # Choose random starting points, uniformly distributed from -15 to 15\n np.random.seed(rseed)\n x0 = -15 + 30 * np.random.random((N_trajectories, 3))\n\n # Solve for the trajectories\n t = np.linspace(0, 2, 500)\n x_t = np.asarray([integrate.odeint(lorentz_deriv, x0i, t)\n for x0i in x0])\n\n # Set up figure & 3D axis for animation\n fig = plt.figure()\n ax = fig.add_axes([0, 0, 1, 1], projection='3d')\n ax.axis('off')\n\n # choose a different color for each trajectory\n colors = plt.cm.jet(np.linspace(0, 1, N_trajectories))\n\n # set up lines and points\n lines = sum([ax.plot([], [], [], '-', c=c)\n for c in colors], [])\n pts = sum([ax.plot([], [], [], 'o', c=c, ms=4)\n for c in colors], [])\n\n # prepare the axes limits\n ax.set_xlim((-25, 25))\n ax.set_ylim((-35, 35))\n ax.set_zlim((5, 55))\n\n # set point-of-view: specified by (altitude degrees, azimuth degrees)\n ax.view_init(30, 0)\n\n # initialization function: plot the background of each frame\n def init():\n for line, pt in zip(lines, pts):\n line.set_data([], [])\n line.set_3d_properties([])\n\n pt.set_data([], [])\n pt.set_3d_properties([])\n return lines + pts\n\n # animation function: called sequentially\n def animate(i):\n # we'll step two time-steps per frame. This leads to nice results.\n i = (2 * i) % x_t.shape[1]\n\n for line, pt, xi in zip(lines, pts, x_t):\n x, y, z = xi[:i + 1].T\n line.set_data(x, y)\n line.set_3d_properties(z)\n\n pt.set_data(x[-1:], y[-1:])\n pt.set_3d_properties(z[-1:])\n\n ax.view_init(30, 0.3 * i)\n fig.canvas.draw()\n return lines + pts\n\n return animation.FuncAnimation(fig, animate, init_func=init,\n frames=frames, interval=interval)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nreturning a string that represents the included frames in the given frame_list.", "response": "def _included_frames(frame_list, frame_format):\n \"\"\"frame_list should be a list of filenames\"\"\"\n return INCLUDED_FRAMES.format(Nframes=len(frame_list),\n frame_dir=os.path.dirname(frame_list[0]),\n frame_format=frame_format)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nreturn the embedded frames in the current image.", "response": "def _embedded_frames(frame_list, frame_format):\n \"\"\"frame_list should be a list of base64-encoded png files\"\"\"\n template = ' frames[{0}] = \"data:image/{1};base64,{2}\"\\n'\n embedded = \"\\n\"\n for i, frame_data in enumerate(frame_list):\n embedded += template.format(i, frame_format,\n frame_data.replace('\\n', '\\\\\\n'))\n return embedded"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nremoving noise from the mean and standard deviations from the time and magnitude.", "response": "def remove_noise(time, magnitude, error, error_limit=3, std_limit=5):\n \"\"\"Points within 'std_limit' standard deviations from the mean and with\n errors greater than 'error_limit' times the error mean are\n considered as noise and thus are eliminated.\n\n \"\"\"\n data, mjd = magnitude, time\n\n data_len = len(mjd)\n error_mean = np.mean(error)\n error_tolerance = error_limit * (error_mean or 1)\n data_mean = np.mean(data)\n data_std = np.std(data)\n\n mjd_out, data_out, error_out = [], [], []\n for i in range(data_len):\n is_not_noise = (\n error[i] < error_tolerance and\n (np.absolute(data[i] - data_mean) / data_std) < std_limit)\n\n if is_not_noise:\n mjd_out.append(mjd[i])\n data_out.append(data[i])\n error_out.append(error[i])\n\n data_out = np.asarray(data_out)\n mjd_out = np.asarray(mjd_out)\n error_out = np.asarray(error_out)\n\n return mjd_out, data_out, error_out"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef align(time, time2, magnitude, magnitude2, error, error2):\n\n error = np.zeros(time.shape) if error is None else error\n error2 = np.zeros(time2.shape) if error2 is None else error2\n\n # this asume that the first series is the short one\n sserie = pd.DataFrame({\"mag\": magnitude, \"error\": error}, index=time)\n lserie = pd.DataFrame({\"mag\": magnitude2, \"error\": error2}, index=time2)\n\n # if the second serie is logest then revert\n if len(time) > len(time2):\n sserie, lserie = lserie, sserie\n\n # make the merge\n merged = sserie.join(lserie, how=\"inner\", rsuffix='2')\n\n # recreate columns\n new_time = merged.index.values\n new_mag, new_mag2 = merged.mag.values, merged.mag2.values\n new_error, new_error2 = merged.error.values, merged.error2.values\n\n if len(time) > len(time2):\n new_mag, new_mag2 = new_mag2, new_mag\n new_error, new_error2 = new_error2, new_error\n\n return new_time, new_mag, new_mag2, new_error, new_error2", "response": "Synchronizes the light - curves in the two different bands."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef load_OGLE3_catalog():\n with bz2.BZ2File(CATALOG_PATH) as bz2fp, warnings.catch_warnings():\n warnings.simplefilter(\"ignore\")\n df = pd.read_table(bz2fp, skiprows=6)\n df.rename(columns={\"# ID\": \"ID\"}, inplace=True)\n return df", "response": "Return the full list of variables stars of OGLE - 3 as a DataFrame\n "} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef fetch_OGLE3(ogle3_id, data_home=None,\n metadata=None, download_if_missing=True):\n \"\"\"Retrieve a lighte curve from OGLE-3 database\n\n Parameters\n ----------\n ogle3_id : str\n The id of the source (see: ``load_OGLE3_catalog()`` for\n available sources.\n data_home : optional, default: None\n Specify another download and cache folder for the datasets. By default\n all feets data is stored in '~/feets' subfolders.\n metadata : bool | None\n If it's True, the row of the dataframe from ``load_OGLE3_catalog()``\n with the metadata of the source are added to the result.\n download_if_missing : optional, True by default\n If False, raise a IOError if the data is not locally available\n instead of trying to download the data from the source site.\n\n Returns\n -------\n\n A Data object.\n\n\n Examples\n --------\n\n .. code-block:: pycon\n\n >>> ds = fetch_OGLE3(\"OGLE-BLG-LPV-232377\")\n >>> ds\n Data(id='OGLE-BLG-LPV-232377', ds_name='OGLE-III', bands=('I', 'V'))\n >>> ds.bands\n ('I', 'V')\n >>> ds.data.I\n LightCurve(time[100], magnitude[100], error[100])\n >>> ds.data.I.magnitude\n array([ 13.816, 13.826, 13.818, 13.812, 13.8 , 13.827, 13.797,\n 13.82 , 13.804, 13.783, 13.823, 13.8 , 13.84 , 13.817,\n 13.802, 13.824, 13.822, 13.81 , 13.844, 13.848, 13.813,\n 13.836, 13.83 , 13.83 , 13.837, 13.811, 13.814, 13.82 ,\n 13.826, 13.822, 13.821, 13.817, 13.813, 13.809, 13.817,\n 13.836, 13.804, 13.801, 13.813, 13.823, 13.818, 13.831,\n 13.833, 13.814, 13.814, 13.812, 13.822, 13.814, 13.818,\n 13.817, 13.8 , 13.804, 13.799, 13.809, 13.815, 13.846,\n 13.796, 13.791, 13.804, 13.853, 13.839, 13.816, 13.825,\n 13.81 , 13.8 , 13.807, 13.819, 13.829, 13.844, 13.84 ,\n 13.842, 13.818, 13.801, 13.804, 13.814, 13.821, 13.821,\n 13.822, 13.82 , 13.803, 13.813, 13.826, 13.855, 13.865,\n 13.854, 13.828, 13.809, 13.828, 13.833, 13.829, 13.816,\n 13.82 , 13.827, 13.834, 13.811, 13.817, 13.808, 13.834,\n 13.814, 13.829])\n\n \"\"\"\n\n # retrieve the data dir for ogle\n store_path = _get_OGLE3_data_home(data_home)\n\n # the data dir for this lightcurve\n file_path = os.path.join(store_path, \"{}.tar\".format(ogle3_id))\n\n # members of the two bands of ogle3\n members = {\"I\": \"./{}.I.dat\".format(ogle3_id),\n \"V\": \"./{}.V.dat\".format(ogle3_id)}\n\n # the url of the lightcurve\n if download_if_missing:\n url = URL.format(ogle3_id)\n base.fetch(url, file_path)\n\n bands = []\n data = {}\n with tarfile.TarFile(file_path) as tfp:\n members_names = tfp.getnames()\n for band_name, member_name in members.items():\n if member_name in members_names:\n member = tfp.getmember(member_name)\n src = tfp.extractfile(member)\n lc = _check_dim(np.loadtxt(src))\n data[band_name] = {\"time\": lc[:, 0],\n \"magnitude\": lc[:, 1],\n \"error\": lc[:, 2]}\n bands.append(band_name)\n if metadata:\n cat = load_OGLE3_catalog()\n metadata = cat[cat.ID == ogle3_id].iloc[0].to_dict()\n del cat\n\n return Data(\n id=ogle3_id, metadata=metadata, ds_name=\"OGLE-III\",\n description=DESCR, bands=bands, data=data)", "response": "Retrieve a lighte curve from OGLE - 3 database."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef sort_by_dependencies(exts, retry=None):\n sorted_ext, features_from_sorted = [], set()\n pending = [(e, 0) for e in exts]\n retry = len(exts) * 100 if retry is None else retry\n while pending:\n ext, cnt = pending.pop(0)\n\n if not isinstance(ext, Extractor) and not issubclass(ext, Extractor):\n msg = \"Only Extractor instances are allowed. Found {}.\"\n raise TypeError(msg.format(type(ext)))\n\n deps = ext.get_dependencies()\n if deps.difference(features_from_sorted):\n if cnt + 1 > retry:\n msg = \"Maximun retry ({}) to sort achieved from extractor {}.\"\n raise RuntimeError(msg.format(retry, type(ext)))\n pending.append((ext, cnt + 1))\n else:\n sorted_ext.append(ext)\n features_from_sorted.update(ext.get_features())\n return tuple(sorted_ext)", "response": "Calculate the Feature Extractor Resolution Order."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef getSignificance(wk1, wk2, nout, ofac):\n expy = exp(-wk2)\n effm = 2.0*(nout)/ofac\n sig = effm*expy\n ind = (sig > 0.01).nonzero()\n sig[ind] = 1.0-(1.0-expy[ind])**effm\n return sig", "response": "returns the probability of the false alarm probabilitywk1 wk2 nout of ac"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nretrieving data from an url and store it into dest.", "response": "def fetch(url, dest, force=False):\n \"\"\"Retrieve data from an url and store it into dest.\n\n Parameters\n ----------\n url: str\n Link to the remote data\n dest: str\n Path where the file must be stored\n force: bool (default=False)\n Overwrite if the file exists\n\n Returns\n -------\n cached: bool\n True if the file already exists\n dest: str\n The same string of the parameter\n\n\n \"\"\"\n\n cached = True\n if force or not os.path.exists(dest):\n cached = False\n r = requests.get(url, stream=True)\n if r.status_code == 200:\n with open(dest, 'wb') as f:\n for chunk in r.iter_content(1024):\n f.write(chunk)\n return cached, dest"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\ngenerate a random light curve with any given random function.", "response": "def create_random(magf, magf_params, errf, errf_params,\n timef=np.linspace, timef_params=None, size=DEFAULT_SIZE,\n id=None, ds_name=DS_NAME, description=DESCRIPTION,\n bands=BANDS, metadata=METADATA):\n \"\"\"Generate a data with any given random function.\n\n Parameters\n ----------\n\n magf : callable\n Function to generate the magnitudes.\n magf_params : dict-like\n Parameters to feed the `magf` function.\n errf : callable\n Function to generate the magnitudes.\n errf_params : dict-like\n Parameters to feed the `errf` function.\n timef : callable, (default=numpy.linspace)\n Function to generate the times.\n timef_params : dict-like or None, (default={\"start\": 0., \"stop\": 1.})\n Parameters to feed the `timef` callable.\n size : int (default=10000)\n Number of obervation of the light curves\n id : object (default=None)\n Id of the created data.\n ds_name : str (default=\"feets-synthetic\")\n Name of the dataset\n description : str (default=\"Lightcurve created with random numbers\")\n Description of the data\n bands : tuple of strings (default=(\"B\", \"V\"))\n The bands to be created\n metadata : dict-like or None (default=None)\n The metadata of the created data\n\n Returns\n -------\n\n data\n A Data object with a random lightcurves.\n\n Examples\n --------\n\n .. code-block:: pycon\n\n >>> from numpy import random\n >>> create_random(\n ... magf=random.normal, magf_params={\"loc\": 0, \"scale\": 1},\n ... errf=random.normal, errf_params={\"loc\": 0, \"scale\": 0.008})\n Data(id=None, ds_name='feets-synthetic', bands=('B', 'V'))\n\n \"\"\"\n timef_params = (\n {\"start\": 0., \"stop\": 1.}\n if timef_params is None else\n timef_params.copy())\n timef_params.update(num=size)\n\n magf_params = magf_params.copy()\n magf_params.update(size=size)\n\n errf_params = errf_params.copy()\n errf_params.update(size=size)\n\n data = {}\n for band in bands:\n data[band] = {\n \"time\": timef(**timef_params),\n \"magnitude\": magf(**magf_params),\n \"error\": errf(**errf_params)}\n return Data(\n id=id, ds_name=ds_name, description=description,\n bands=bands, metadata=metadata, data=data)"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\ngenerating a random data with magnitudes that follows a Gaussian distribution. Also their errors are gaussian.", "response": "def create_normal(mu=0., sigma=1., mu_err=0.,\n sigma_err=1., seed=None, **kwargs):\n \"\"\"Generate a data with magnitudes that follows a Gaussian\n distribution. Also their errors are gaussian.\n\n Parameters\n ----------\n\n mu : float (default=0)\n Mean of the gaussian distribution of magnitudes\n sigma : float (default=1)\n Standar deviation of the gaussian distribution of magnitude errors\n mu_err : float (default=0)\n Mean of the gaussian distribution of magnitudes\n sigma_err : float (default=1)\n Standar deviation of the gaussian distribution of magnitude errorrs\n seed : {None, int, array_like}, optional\n Random seed used to initialize the pseudo-random number generator.\n Can be any integer between 0 and 2**32 - 1 inclusive, an\n array (or other sequence) of such integers, or None (the default).\n If seed is None, then RandomState will try to read data from\n /dev/urandom (or the Windows analogue) if available or seed from\n the clock otherwise.\n kwargs : optional\n extra arguments for create_random.\n\n Returns\n -------\n\n data\n A Data object with a random lightcurves.\n\n Examples\n --------\n\n .. code-block:: pycon\n\n >>> ds = create_normal(0, 1, 0, .0008, seed=42)\n >>> ds\n Data(id=None, ds_name='feets-synthetic', bands=('B', 'V'))\n >>> ds.data.B\n LightCurve(time[10000], magnitude[10000], error[10000])\n >>> ds.data.B.time\n array([ 0.00000000e+00, 1.00010001e-04, 2.00020002e-04, ...,\n 9.99799980e-01, 9.99899990e-01, 1.00000000e+00])\n\n \"\"\"\n\n random = np.random.RandomState(seed)\n return create_random(\n magf=random.normal, magf_params={\"loc\": mu, \"scale\": sigma},\n errf=random.normal, errf_params={\"loc\": mu_err, \"scale\": sigma_err},\n **kwargs)"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\ngenerates a data object with magnitudes that follows a uniform distribution ; the error instead are gaussian.", "response": "def create_uniform(low=0., high=1., mu_err=0., sigma_err=1.,\n seed=None, **kwargs):\n \"\"\"Generate a data with magnitudes that follows a uniform\n distribution; the error instead are gaussian.\n\n Parameters\n ----------\n\n low : float, optional\n Lower boundary of the output interval. All values generated will be\n greater than or equal to low. The default value is 0.\n high : float, optional\n Upper boundary of the output interval. All values generated will be\n less than high. The default value is 1.0.\n mu_err : float (default=0)\n Mean of the gaussian distribution of magnitudes\n sigma_err : float (default=1)\n Standar deviation of the gaussian distribution of magnitude errorrs\n seed : {None, int, array_like}, optional\n Random seed used to initialize the pseudo-random number generator.\n Can be any integer between 0 and 2**32 - 1 inclusive, an\n array (or other sequence) of such integers, or None (the default).\n If seed is None, then RandomState will try to read data from\n /dev/urandom (or the Windows analogue) if available or seed from\n the clock otherwise.\n kwargs : optional\n extra arguments for create_random.\n\n Returns\n -------\n\n data\n A Data object with a random lightcurves.\n\n Examples\n --------\n\n .. code-block:: pycon\n\n >>> ds = synthetic.create_uniform(1, 2, 0, .0008, 42)\n >>> ds\n Data(id=None, ds_name='feets-synthetic', bands=('B', 'V'))\n >>> ds.data.B.magnitude\n array([ 1.37454012, 1.95071431, 1.73199394, ..., 1.94670792,\n 1.39748799, 1.2171404 ])\n\n \"\"\"\n random = np.random.RandomState(seed)\n return create_random(\n magf=random.uniform, magf_params={\"low\": low, \"high\": high},\n errf=random.normal, errf_params={\"loc\": mu_err, \"scale\": sigma_err},\n **kwargs)"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef create_periodic(mu_err=0., sigma_err=1., seed=None, **kwargs):\n\n random = np.random.RandomState(seed)\n\n size = kwargs.get(\"size\", DEFAULT_SIZE)\n\n times, mags, errors = [], [], []\n for b in kwargs.get(\"bands\", BANDS):\n time = 100 * random.rand(size)\n error = random.normal(size=size, loc=mu_err, scale=sigma_err)\n mag = np.sin(2 * np.pi * time) + error * random.randn(size)\n times.append(time)\n errors.append(error)\n mags.append(mag)\n\n times, mags, errors = iter(times), iter(mags), iter(errors)\n\n return create_random(\n magf=lambda **k: next(mags), magf_params={},\n errf=lambda **k: next(errors), errf_params={},\n timef=lambda **k: next(times), timef_params={}, **kwargs)", "response": "Generate a data object with magnitudes with periodic variability of variability error."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\ncompute the expected probability density function for a single periodogram.", "response": "def pdf_single(z, N, normalization, dH=1, dK=3):\n \"\"\"Probability density function for Lomb-Scargle periodogram\n\n Compute the expected probability density function of the periodogram\n for the null hypothesis - i.e. data consisting of Gaussian noise.\n\n Parameters\n ----------\n z : array-like\n the periodogram value\n N : int\n the number of data points from which the periodogram was computed\n normalization : string\n The periodogram normalization. Must be one of\n ['standard', 'model', 'log', 'psd']\n dH, dK : integers (optional)\n The number of parameters in the null hypothesis and the model\n\n Returns\n -------\n pdf : np.ndarray\n The expected probability density function\n\n Notes\n -----\n For normalization='psd', the distribution can only be computed for\n periodograms constructed with errors specified.\n All expressions used here are adapted from Table 1 of Baluev 2008 [1]_.\n\n References\n ----------\n .. [1] Baluev, R.V. MNRAS 385, 1279 (2008)\n \"\"\"\n if dK - dH != 2:\n raise NotImplementedError(\"Degrees of freedom != 2\")\n Nk = N - dK\n\n if normalization == 'psd':\n return np.exp(-z)\n elif normalization == 'standard':\n return 0.5 * Nk * (1 - z) ** (0.5 * Nk - 1)\n elif normalization == 'model':\n return 0.5 * Nk * (1 + z) ** (-0.5 * Nk - 1)\n elif normalization == 'log':\n return 0.5 * Nk * np.exp(-0.5 * Nk * z)\n else:\n raise ValueError(\"normalization='{0}' is not recognized\"\n \"\".format(normalization))"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\ncomputes the expected cumulative distribution for the Lomb - Scargle periodogram.", "response": "def cdf_single(z, N, normalization, dH=1, dK=3):\n \"\"\"Cumulative distribution for the Lomb-Scargle periodogram\n\n Compute the expected cumulative distribution of the periodogram\n for the null hypothesis - i.e. data consisting of Gaussian noise.\n\n Parameters\n ----------\n z : array-like\n the periodogram value\n N : int\n the number of data points from which the periodogram was computed\n normalization : string\n The periodogram normalization. Must be one of\n ['standard', 'model', 'log', 'psd']\n dH, dK : integers (optional)\n The number of parameters in the null hypothesis and the model\n\n Returns\n -------\n cdf : np.ndarray\n The expected cumulative distribution function\n\n Notes\n -----\n For normalization='psd', the distribution can only be computed for\n periodograms constructed with errors specified.\n All expressions used here are adapted from Table 1 of Baluev 2008 [1]_.\n\n References\n ----------\n .. [1] Baluev, R.V. MNRAS 385, 1279 (2008)\n\n \"\"\"\n return 1 - fap_single(z, N, normalization=normalization, dH=dH, dK=dK)"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef fap_simple(Z, fmax, t, y, dy, normalization='standard'):\n N = len(t)\n T = max(t) - min(t)\n N_eff = fmax * T\n p_s = cdf_single(Z, N, normalization=normalization)\n return 1 - p_s ** N_eff", "response": "False Alarm Probability based on estimated number of indep frequencies"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\naliasing - free approximation to false alarm probability", "response": "def fap_baluev(Z, fmax, t, y, dy, normalization='standard'):\n \"\"\"Alias-free approximation to false alarm probability\n\n (Eqn 6 of Baluev 2008)\n \"\"\"\n cdf = cdf_single(Z, len(t), normalization)\n tau = tau_davies(Z, fmax, t, y, dy, normalization=normalization)\n return 1 - cdf * np.exp(-tau)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef false_alarm_probability(Z, fmax, t, y, dy, normalization,\n method='baluev', method_kwds=None):\n \"\"\"Approximate the False Alarm Probability\n\n Parameters\n ----------\n TODO\n\n Returns\n -------\n TODO\n \"\"\"\n if method not in METHODS:\n raise ValueError(\"Unrecognized method: {0}\".format(method))\n method = METHODS[method]\n method_kwds = method_kwds or {}\n\n return method(Z, fmax, t, y, dy, normalization, **method_kwds)", "response": "Approximate the False Alarm Probability\n "} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nloads the MACHO lightcurve of 2 bands from the MACHO survey.", "response": "def load_MACHO(macho_id):\n \"\"\"lightcurve of 2 bands (R, B) from the MACHO survey.\n\n Notes\n -----\n\n The files are gathered from the original FATS project tutorial:\n https://github.com/isadoranun/tsfeat\n\n \"\"\"\n tarfname = \"{}.tar.bz2\".format(macho_id)\n tarpath = os.path.join(DATA_PATH, tarfname)\n\n rpath = \"{}.R.mjd\".format(macho_id)\n bpath = \"{}.B.mjd\".format(macho_id)\n with tarfile.open(tarpath, mode=\"r:bz2\") as tf:\n rlc = np.loadtxt(tf.extractfile(rpath))\n blc = np.loadtxt(tf.extractfile(bpath))\n\n bands = (\"R\", \"B\")\n data = {\n \"R\": {\n \"time\": rlc[:, 0],\n \"magnitude\": rlc[:, 1],\n \"error\": rlc[:, 2]},\n \"B\": {\n \"time\": blc[:, 0],\n \"magnitude\": blc[:, 1],\n \"error\": blc[:, 2]}\n }\n descr = (\"The files are gathered from the original FATS project \"\n \"tutorial: https://github.com/isadoranun/tsfeat\")\n\n return Data(\n id=macho_id, metadata=None, ds_name=\"MACHO\",\n description=descr, bands=bands, data=data)"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef anim_to_html(anim, fps=None, embed_frames=True, default_mode='loop'):\n if fps is None and hasattr(anim, '_interval'):\n # Convert interval in ms to frames per second\n fps = 1000. / anim._interval\n\n plt.close(anim._fig)\n if hasattr(anim, \"_html_representation\"):\n return anim._html_representation\n else:\n # tempfile can't be used here: we need a filename, and this\n # fails on windows. Instead, we use a custom filename generator\n #with tempfile.NamedTemporaryFile(suffix='.html') as f:\n with _NameOnlyTemporaryFile(suffix='.html') as f:\n anim.save(f.name, writer=HTMLWriter(fps=fps,\n embed_frames=embed_frames,\n default_mode=default_mode))\n html = open(f.name).read()\n\n anim._html_representation = html\n return html", "response": "Generate HTML representation of the animation"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef display_animation(anim, **kwargs):\n from IPython.display import HTML\n return HTML(anim_to_html(anim, **kwargs))", "response": "Display the animation with an IPython HTML object"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nindenting the string s with the character c n times.", "response": "def indent(s, c=\" \", n=4):\n \"\"\"Indent the string 's' with the character 'c', 'n' times.\n\n Parameters\n ----------\n\n s : str\n String to indent\n c : str, default space\n String to use as indentation\n n : int, default 4\n Number of chars to indent\n\n \"\"\"\n indentation = c * n\n return \"\\n\".join([indentation + l for l in s.splitlines()])"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef generate_date_tail_boost_queries(\n field, timedeltas_and_boosts, relative_to=None):\n \"\"\"\n Generate a list of RangeQueries usable to boost the scores of more\n recent documents.\n\n Example:\n\n ```\n queries = generate_date_tail_boost_queries(\"publish_date\", {\n timedelta(days=90): 1,\n timedelta(days=30): 2,\n timedelta(days=10): 4,\n })\n s = Search(BoolQuery(must=..., should=queries))\n # ...\n ```\n\n Refs:\n http://elasticsearch-users.115913.n3.nabble.com/Boost-recent-documents-td2126107.html#a2126317\n\n :param field: field name to generate the queries against\n :param timedeltas_and_boosts:\n dictionary of timedelta instances and their boosts. Negative or\n zero boost values will not generate rangequeries.\n :type timedeltas_and_boosts: dict[timedelta, float]\n :param relative_to: Relative to this datetime (may be None for \"now\")\n :return: List of RangeQueries\n \"\"\"\n relative_to = relative_to or datetime.datetime.now()\n times = {}\n for timedelta, boost in timedeltas_and_boosts.items():\n date = (relative_to - timedelta).date()\n times[date] = boost\n times = sorted(times.items(), key=lambda i: i[0])\n queries = []\n\n for (x, time) in enumerate(times):\n kwargs = {\"field\": field, \"boost\": time[1]}\n if x == 0:\n kwargs[\"lte\"] = time[0]\n else:\n kwargs[\"gt\"] = time[0]\n if x < len(times) - 1:\n kwargs[\"lte\"] = times[x + 1][0]\n\n if kwargs[\"boost\"] > 0:\n q = RangeQuery()\n q.add_range(**kwargs)\n queries.append(q)\n\n return queries", "response": "Generates a list of RangeQueries usable to boost the scores of recent documents."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef _clean_dict(in_dict):\n out = {}\n for key, value in iteritems(in_dict):\n if isinstance(value, dict):\n value = _clean_dict(value)\n if value is None:\n continue\n out[key] = value\n return out", "response": "Recursively remove None - valued items from dict.\n "} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nyields batches of count items from the given iterable.", "response": "def batch_iterable(iterable, count):\n \"\"\"\n Yield batches of `count` items from the given iterable.\n\n >>> for x in batch([1, 2, 3, 4, 5, 6, 7], 3):\n >>> print(x)\n [1, 2, 3]\n [4, 5, 6]\n [7]\n\n :param iterable: An iterable\n :type iterable: Iterable\n :param count: Number of items per batch. If <= 0, nothing is yielded.\n :type count: int\n :return: Iterable of lists of items\n :rtype: Iterable[list[object]]\n \"\"\"\n if count <= 0:\n return\n current_batch = []\n for item in iterable:\n if len(current_batch) == count:\n yield current_batch\n current_batch = []\n current_batch.append(item)\n if current_batch:\n yield current_batch"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nreturns an iterator of Django model objects in Elasticsearch order optionally using the given Django queryset.", "response": "def get_objects(self, queryset=None):\n \"\"\"\n Return an iterator of Django model objects in Elasticsearch order,\n optionally using the given Django queryset. If no queryset is\n given, a default queryset (Model.objects.all) is used.\n\n :param queryset: Optional queryset to filter in.\n :return:\n \"\"\"\n if not self:\n return\n if not queryset:\n queryset = self[0].django_model.objects.all()\n\n pks = [res.pk for res in self if res.django_model == queryset.model]\n object_map = dict((text_type(obj.pk), obj)\n for obj in queryset.filter(pk__in=pks))\n result_map = dict((res.pk, res)\n for res in self if res.pk in object_map)\n\n for pk in pks:\n obj = object_map.get(pk)\n if obj:\n obj._es = result_map.get(pk)\n try:\n obj._score = obj._es._meta.score\n except AttributeError:\n obj._score = None\n yield obj"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nresetting the target collection.", "response": "def reset(self):\n \"\"\" Reset target collection (rebuild index).\n \"\"\"\n self.connection.rebuild_index(\n self.index, coll_name=self.target_coll_name)"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef update(self, index, fields, tokenizer=default_tokenizer, cutoff=1):\n counts_by_uid = defaultdict(Counter)\n for word, count in self._gather_words(\n index, fields, tokenizer=tokenizer).items():\n uid = hashlib.sha1(unicodedata.normalize(\n \"NFKD\", word.lower()).encode(\"UTF-8\")).hexdigest()\n counts_by_uid[uid][word] += count\n\n for uid, word_to_count in counts_by_uid.items():\n word = word_to_count.most_common(1)[0][0]\n count = sum(word_to_count.values())\n if count <= cutoff:\n continue\n self.connection.session.post(\n \"/%s/%s/%s/_update\" % (self.target_coll_name,\n self.target_type, uid),\n data={\n \"script\": \"ctx._source.count += count\",\n \"params\": {\"count\": count},\n \"upsert\": {\"word\": word, \"count\": count}\n })", "response": "Update the wordgatherer collection."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef search(self, word, limit=30):\n search = Search(PrefixQuery(\"word\", word), sort={\"count\": \"desc\"})\n for doc in self.connection.search(\n search, indexes=[self.index], count=limit):\n yield (doc[\"word\"], doc[\"count\"])", "response": "Search for a word within the wordgatherer collection."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef validate_nb(nb):\n if nb['nbformat'] != 4:\n return False\n\n language_name = (nb.get('metadata', {})\n .get('kernelspec', {})\n .get('language', '').lower())\n return language_name == 'python'", "response": "Validate that given notebook JSON is importable and that the notebook language is python"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef filter_ast(module_ast):\n def node_predicate(node):\n \"\"\"\n Return true if given node is whitelisted\n \"\"\"\n for an in ALLOWED_NODES:\n if isinstance(node, an):\n return True\n\n # Recurse through Assign node LHS targets when an id is not specified,\n # otherwise check that the id is uppercase\n if isinstance(node, ast.Assign):\n return all([node_predicate(t) for t in node.targets if not hasattr(t, 'id')]) \\\n and all([t.id.isupper() for t in node.targets if hasattr(t, 'id')])\n\n return False\n\n module_ast.body = [n for n in module_ast.body if node_predicate(n)]\n return module_ast", "response": "Filter a given module ast removing non - whitelisted nodes"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef code_from_ipynb(nb, markdown=False):\n code = PREAMBLE\n for cell in nb['cells']:\n if cell['cell_type'] == 'code':\n # transform the input to executable Python\n code += ''.join(cell['source'])\n if cell['cell_type'] == 'markdown':\n code += '\\n# ' + '# '.join(cell['source'])\n # We want a blank newline after each cell's output.\n # And the last line of source doesn't have a newline usually.\n code += '\\n\\n'\n return code", "response": "Get the code for a given notebook"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef _get_paths(self, fullname):\n real_path = os.path.join(*fullname[len(self.package_prefix):].split('.'))\n for base_path in sys.path:\n if base_path == '':\n # Empty string means process's cwd\n base_path = os.getcwd()\n path = os.path.join(base_path, real_path)\n yield path + '.ipynb'\n yield path + '.py'\n yield os.path.join(path, '__init__.ipynb')\n yield os.path.join(path, '__init__.py')", "response": "Generate ordered list of paths we should look for fullname module in\n "} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nfinds a module spec by name and path.", "response": "def find_spec(self, fullname, path, target=None):\n \"\"\"\n Claims modules that are under ipynb.fs\n \"\"\"\n if fullname.startswith(self.package_prefix):\n for path in self._get_paths(fullname):\n if os.path.exists(path):\n return ModuleSpec(\n name=fullname,\n loader=self.loader_class(fullname, path),\n origin=path,\n is_package=(path.endswith('__init__.ipynb') or path.endswith('__init__.py')),\n )"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef coroutine(func):\n @wraps(func)\n def start(*args, **kwargs):\n g = func(*args, **kwargs)\n next(g)\n return g\n\n return start", "response": "Decorator for priming generator - based coroutines.\n "} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\nasync def ticker(delay, to):\n for i in range(to):\n yield i\n await asyncio.sleep(delay)", "response": "Yields numbers from 0 to every delay seconds."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nconverts an iterable into a stream of events.", "response": "def iterable_source(iterable, target):\n \"\"\"Convert an iterable into a stream of events.\n\n Args:\n iterable: A series of items which will be sent to the target one by one.\n target: The target coroutine or sink.\n\n Returns:\n An iterator over any remaining items.\n \"\"\"\n it = iter(iterable)\n for item in it:\n try:\n target.send(item)\n except StopIteration:\n return prepend(item, it)\n return empty_iter()"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef poisson_source(rate, iterable, target):\n if rate <= 0.0:\n raise ValueError(\"poisson_source rate {} is not positive\".format(rate))\n\n it = iter(iterable)\n for item in it:\n duration = random.expovariate(rate)\n sleep(duration)\n try:\n target.send(item)\n except StopIteration:\n return prepend(item, it)\n return empty_iter()", "response": "Send events at random times with uniform probability."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef compose(f, *fs):\n rfs = list(chain([f], fs))\n rfs.reverse()\n\n def composed(*args, **kwargs):\n return reduce(\n lambda result, fn: fn(result),\n rfs[1:],\n rfs[0](*args, **kwargs))\n\n return composed", "response": "Compose functions right to left."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\ncreating a scanning reducer.", "response": "def scanning(reducer, init=UNSET):\n \"\"\"Create a scanning reducer.\"\"\"\n\n reducer2 = reducer\n\n def scanning_transducer(reducer):\n return Scanning(reducer, reducer2, init)\n\n return scanning_transducer"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\ncreate a transducer which takes the first n items", "response": "def taking(n):\n \"\"\"Create a transducer which takes the first n items\"\"\"\n\n if n < 0:\n raise ValueError(\"Cannot take fewer than zero ({}) items\".format(n))\n\n def taking_transducer(reducer):\n return Taking(reducer, n)\n\n return taking_transducer"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef dropping(n):\n\n if n < 0:\n raise ValueError(\"Cannot drop fewer than zero ({}) items\".format(n))\n\n def dropping_transducer(reducer):\n return Dropping(reducer, n)\n\n return dropping_transducer", "response": "Create a transducer which drops the first n items"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef batching(size):\n\n if size < 1:\n raise ValueError(\"batching() size must be at least 1\")\n\n def batching_transducer(reducer):\n return Batching(reducer, size)\n\n return batching_transducer", "response": "Create a transducer which produces non - overlapping batches."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef windowing(size, padding=UNSET, window_type=tuple):\n\n if size < 1:\n raise ValueError(\"windowing() size {} is not at least 1\".format(size))\n\n def windowing_transducer(reducer):\n return Windowing(reducer, size, padding, window_type)\n\n return windowing_transducer", "response": "Create a transducer which produces a moving window over items."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef first(predicate=None):\n\n predicate = true if predicate is None else predicate\n\n def first_transducer(reducer):\n return First(reducer, predicate)\n\n return first_transducer", "response": "Create a transducer which obtains the first item then terminates."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\ncreate a transducer which obtains the last item.", "response": "def last(predicate=None):\n \"\"\"Create a transducer which obtains the last item.\"\"\"\n\n predicate = true if predicate is None else predicate\n\n def last_transducer(reducer):\n return Last(reducer, predicate)\n\n return last_transducer"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef element_at(index):\n\n if index < 0:\n raise IndexError(\"element_at used with illegal index {}\".format(index))\n\n def element_at_transducer(reducer):\n return ElementAt(reducer, index)\n\n return element_at_transducer", "response": "Create a transducer which obtains the item at the specified index."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef compile_sources(files, CompilerRunner_=None,\n destdir=None, cwd=None,\n keep_dir_struct=False,\n per_file_kwargs=None,\n **kwargs):\n \"\"\"\n Compile source code files to object files.\n\n Parameters\n ----------\n files: iterable of path strings\n source files, if cwd is given, the paths are taken as relative.\n CompilerRunner_: CompilerRunner instance (optional)\n could be e.g. pycompilation.FortranCompilerRunner\n Will be inferred from filename extensions if missing.\n destdir: path string\n output directory, if cwd is given, the path is taken as relative\n cwd: path string\n working directory. Specify to have compiler run in other directory.\n also used as root of relative paths.\n keep_dir_struct: bool\n Reproduce directory structure in `destdir`. default: False\n per_file_kwargs: dict\n dict mapping instances in `files` to keyword arguments\n **kwargs: dict\n default keyword arguments to pass to CompilerRunner_\n \"\"\"\n _per_file_kwargs = {}\n\n if per_file_kwargs is not None:\n for k, v in per_file_kwargs.items():\n if isinstance(k, Glob):\n for path in glob.glob(k.pathname):\n _per_file_kwargs[path] = v\n elif isinstance(k, ArbitraryDepthGlob):\n for path in glob_at_depth(k.filename, cwd):\n _per_file_kwargs[path] = v\n else:\n _per_file_kwargs[k] = v\n\n # Set up destination directory\n destdir = destdir or '.'\n if not os.path.isdir(destdir):\n if os.path.exists(destdir):\n raise IOError(\"{} is not a directory\".format(destdir))\n else:\n make_dirs(destdir)\n if cwd is None:\n cwd = '.'\n for f in files:\n copy(f, destdir, only_update=True, dest_is_dir=True)\n\n # Compile files and return list of paths to the objects\n dstpaths = []\n for f in files:\n if keep_dir_struct:\n name, ext = os.path.splitext(f)\n else:\n name, ext = os.path.splitext(os.path.basename(f))\n file_kwargs = kwargs.copy()\n file_kwargs.update(_per_file_kwargs.get(f, {}))\n dstpaths.append(src2obj(\n f, CompilerRunner_, cwd=cwd,\n **file_kwargs\n ))\n return dstpaths", "response": "Compile source code files to object files."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef link(obj_files, out_file=None, shared=False, CompilerRunner_=None,\n cwd=None, cplus=False, fort=False, **kwargs):\n \"\"\"\n Link object files.\n\n Parameters\n ----------\n obj_files: iterable of path strings\n out_file: path string (optional)\n path to executable/shared library, if missing\n it will be deduced from the last item in obj_files.\n shared: bool\n Generate a shared library? default: False\n CompilerRunner_: pycompilation.CompilerRunner subclass (optional)\n If not given the `cplus` and `fort` flags will be inspected\n (fallback is the C compiler)\n cwd: path string\n root of relative paths and working directory for compiler\n cplus: bool\n C++ objects? default: False\n fort: bool\n Fortran objects? default: False\n **kwargs: dict\n keyword arguments passed onto CompilerRunner_\n\n Returns\n -------\n The absolute to the generated shared object / executable\n\n \"\"\"\n if out_file is None:\n out_file, ext = os.path.splitext(os.path.basename(obj_files[-1]))\n if shared:\n out_file += sharedext\n\n if not CompilerRunner_:\n if fort:\n CompilerRunner_, extra_kwargs, vendor = \\\n get_mixed_fort_c_linker(\n vendor=kwargs.get('vendor', None),\n metadir=kwargs.get('metadir', None),\n cplus=cplus,\n cwd=cwd,\n )\n for k, v in extra_kwargs.items():\n expand_collection_in_dict(kwargs, k, v)\n else:\n if cplus:\n CompilerRunner_ = CppCompilerRunner\n else:\n CompilerRunner_ = CCompilerRunner\n\n flags = kwargs.pop('flags', [])\n if shared:\n if '-shared' not in flags:\n flags.append('-shared')\n\n # mimic GNU linker behavior on OS X when using -shared\n # (otherwise likely Undefined symbol errors)\n dl_flag = '-undefined dynamic_lookup'\n if sys.platform == 'darwin' and dl_flag not in flags:\n flags.append(dl_flag)\n run_linker = kwargs.pop('run_linker', True)\n if not run_linker:\n raise ValueError(\"link(..., run_linker=False)!?\")\n\n out_file = get_abspath(out_file, cwd=cwd)\n runner = CompilerRunner_(\n obj_files, out_file, flags,\n cwd=cwd,\n **kwargs)\n runner.run()\n return out_file", "response": "Link objects in obj_files into out_file."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nlinking Python extension module to the shared object file.", "response": "def link_py_so(obj_files, so_file=None, cwd=None, libraries=None,\n cplus=False, fort=False, **kwargs):\n \"\"\"\n Link python extension module (shared object) for importing\n\n Parameters\n ----------\n obj_files: iterable of path strings\n object files to be linked\n so_file: path string\n Name (path) of shared object file to create. If\n not specified it will have the basname of the last object\n file in `obj_files` but with the extension '.so' (Unix) or\n '.dll' (Windows).\n cwd: path string\n root of relative paths and working directory of linker.\n libraries: iterable of strings\n libraries to link against, e.g. ['m']\n cplus: bool\n Any C++ objects? default: False\n fort: bool\n Any Fortran objects? default: False\n kwargs**: dict\n keyword arguments passed onto `link(...)`\n\n Returns\n -------\n Absolute path to the generate shared object\n \"\"\"\n libraries = libraries or []\n\n include_dirs = kwargs.pop('include_dirs', [])\n library_dirs = kwargs.pop('library_dirs', [])\n\n # from distutils/command/build_ext.py:\n if sys.platform == \"win32\":\n warnings.warn(\"Windows not yet supported.\")\n elif sys.platform == 'darwin':\n # Don't use the default code below\n pass\n elif sys.platform[:3] == 'aix':\n # Don't use the default code below\n pass\n else:\n # LIBDIR/INSTSONAME should always points to libpython (dynamic or static)\n pylib = os.path.join(get_config_var('LIBDIR'), get_config_var('INSTSONAME'))\n if os.path.exists(pylib):\n libraries.append(pylib)\n else:\n if get_config_var('Py_ENABLE_SHARED'):\n ABIFLAGS = get_config_var('ABIFLAGS')\n pythonlib = 'python{}.{}{}'.format(\n sys.hexversion >> 24, (sys.hexversion >> 16) & 0xff,\n ABIFLAGS or '')\n libraries += [pythonlib]\n else:\n pass\n\n flags = kwargs.pop('flags', [])\n needed_flags = ('-pthread',)\n for flag in needed_flags:\n if flag not in flags:\n flags.append(flag)\n\n # We want something like: gcc, ['-pthread', ...\n # compilername, flags = cc.split()[0], cc.split()[1:]\n\n # # Grab include_dirs\n # include_dirs += list(filter(lambda x: x.startswith('-I'), flags))\n # flags = list(filter(lambda x: not x.startswith('-I'), flags))\n\n # # Grab library_dirs\n # library_dirs += [x[2:] for x in filter(\n # lambda x: x.startswith('-L'), flags)]\n # flags = list(filter(lambda x: not x.startswith('-L'), flags))\n\n # flags.extend(kwargs.pop('flags', []))\n\n return link(obj_files, shared=True, flags=flags, cwd=cwd,\n cplus=cplus, fort=fort, include_dirs=include_dirs,\n libraries=libraries, library_dirs=library_dirs, **kwargs)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef simple_cythonize(src, destdir=None, cwd=None, logger=None,\n full_module_name=None, only_update=False,\n **cy_kwargs):\n \"\"\"\n Generates a C file from a Cython source file.\n\n Parameters\n ----------\n src: path string\n path to Cython source\n destdir: path string (optional)\n Path to output directory (default: '.')\n cwd: path string (optional)\n Root of relative paths (default: '.')\n logger: logging.Logger\n info level used.\n full_module_name: string\n passed to cy_compile (default: None)\n only_update: bool\n Only cythonize if source is newer. default: False\n **cy_kwargs:\n second argument passed to cy_compile.\n Generates a .cpp file if cplus=True in cy_kwargs, else a .c file.\n \"\"\"\n from Cython.Compiler.Main import (\n default_options, CompilationOptions\n )\n from Cython.Compiler.Main import compile as cy_compile\n\n assert src.lower().endswith('.pyx') or src.lower().endswith('.py')\n cwd = cwd or '.'\n destdir = destdir or '.'\n\n ext = '.cpp' if cy_kwargs.get('cplus', False) else '.c'\n c_name = os.path.splitext(os.path.basename(src))[0] + ext\n\n dstfile = os.path.join(destdir, c_name)\n\n if only_update:\n if not missing_or_other_newer(dstfile, src, cwd=cwd):\n msg = '{0} newer than {1}, did not re-cythonize.'.format(\n dstfile, src)\n if logger:\n logger.info(msg)\n else:\n print(msg)\n return dstfile\n\n if cwd:\n ori_dir = os.getcwd()\n else:\n ori_dir = '.'\n os.chdir(cwd)\n try:\n cy_options = CompilationOptions(default_options)\n cy_options.__dict__.update(cy_kwargs)\n if logger:\n logger.info(\"Cythonizing {0} to {1}\".format(\n src, dstfile))\n cy_result = cy_compile([src], cy_options, full_module_name=full_module_name)\n if cy_result.num_errors > 0:\n raise ValueError(\"Cython compilation failed.\")\n if os.path.abspath(os.path.dirname(\n src)) != os.path.abspath(destdir):\n if os.path.exists(dstfile):\n os.unlink(dstfile)\n shutil.move(os.path.join(os.path.dirname(src), c_name),\n destdir)\n finally:\n os.chdir(ori_dir)\n return dstfile", "response": "A simple cythonize function that takes a source file and returns a path to the destination file."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nreturn a new object file that can be used to compile a source code file.", "response": "def src2obj(srcpath, CompilerRunner_=None, objpath=None,\n only_update=False, cwd=None, out_ext=None, inc_py=False,\n **kwargs):\n \"\"\"\n Compiles a source code file to an object file.\n Files ending with '.pyx' assumed to be cython files and\n are dispatched to pyx2obj.\n\n Parameters\n ----------\n srcpath: path string\n path to source file\n CompilerRunner_: pycompilation.CompilerRunner subclass (optional)\n Default: deduced from extension of srcpath\n objpath: path string (optional)\n path to generated object. defualt: deduced from srcpath\n only_update: bool\n only compile if source is newer than objpath. default: False\n cwd: path string (optional)\n working directory and root of relative paths. default: current dir.\n out_ext: string\n set when objpath is a dir and you want to override defaults\n ('.o'/'.obj' for Unix/Windows).\n inc_py: bool\n add Python include path to include_dirs. default: False\n **kwargs: dict\n keyword arguments passed onto CompilerRunner_ or pyx2obj\n \"\"\"\n name, ext = os.path.splitext(os.path.basename(srcpath))\n if objpath is None:\n if os.path.isabs(srcpath):\n objpath = '.'\n else:\n objpath = os.path.dirname(srcpath)\n objpath = objpath or '.' # avoid objpath == ''\n out_ext = out_ext or objext\n if os.path.isdir(objpath):\n objpath = os.path.join(objpath, name+out_ext)\n\n include_dirs = kwargs.pop('include_dirs', [])\n if inc_py:\n from distutils.sysconfig import get_python_inc\n py_inc_dir = get_python_inc()\n if py_inc_dir not in include_dirs:\n include_dirs.append(py_inc_dir)\n\n if ext.lower() == '.pyx':\n return pyx2obj(srcpath, objpath=objpath,\n include_dirs=include_dirs, cwd=cwd,\n only_update=only_update, **kwargs)\n\n if CompilerRunner_ is None:\n CompilerRunner_, std = extension_mapping[ext.lower()]\n if 'std' not in kwargs:\n kwargs['std'] = std\n\n # src2obj implies not running the linker...\n run_linker = kwargs.pop('run_linker', False)\n if run_linker:\n raise CompilationError(\"src2obj called with run_linker=True\")\n\n if only_update:\n if not missing_or_other_newer(objpath, srcpath, cwd=cwd):\n msg = \"Found {0}, did not recompile.\".format(objpath)\n if kwargs.get('logger', None):\n kwargs['logger'].info(msg)\n else:\n print(msg)\n return objpath\n runner = CompilerRunner_(\n [srcpath], objpath, include_dirs=include_dirs,\n run_linker=run_linker, cwd=cwd, **kwargs)\n runner.run()\n return objpath"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef pyx2obj(pyxpath, objpath=None, interm_c_dir=None, cwd=None,\n logger=None, full_module_name=None, only_update=False,\n metadir=None, include_numpy=False, include_dirs=None,\n cy_kwargs=None, gdb=False, cplus=None, **kwargs):\n \"\"\"\n Convenience function\n\n If cwd is specified, pyxpath and dst are taken to be relative\n If only_update is set to `True` the modification time is checked\n and compilation is only run if the source is newer than the\n destination\n\n Parameters\n ----------\n pyxpath: path string\n path to Cython source file\n objpath: path string (optional)\n path to object file to generate\n interm_c_dir: path string (optional)\n directory to put generated C file.\n cwd: path string (optional)\n working directory and root of relative paths\n logger: logging.Logger (optional)\n passed onto `simple_cythonize` and `src2obj`\n full_module_name: string (optional)\n passed onto `simple_cythonize`\n only_update: bool (optional)\n passed onto `simple_cythonize` and `src2obj`\n metadir: path string (optional)\n passed onto src2obj\n include_numpy: bool (optional)\n Add numpy include directory to include_dirs. default: False\n include_dirs: iterable of path strings (optional)\n Passed onto src2obj and via cy_kwargs['include_path']\n to simple_cythonize.\n cy_kwargs: dict (optional)\n keyword arguments passed onto `simple_cythonize`\n gdb: bool (optional)\n convenience: cy_kwargs['gdb_debug'] is set True if gdb=True,\n default: False\n cplus: bool (optional)\n Indicate whether C++ is used. default: auto-detect using `pyx_is_cplus`\n **kwargs: dict\n keyword arguments passed onto src2obj\n\n Returns\n -------\n Absolute path of generated object file.\n\n \"\"\"\n assert pyxpath.endswith('.pyx')\n cwd = cwd or '.'\n objpath = objpath or '.'\n interm_c_dir = interm_c_dir or os.path.dirname(objpath)\n\n abs_objpath = get_abspath(objpath, cwd=cwd)\n\n if os.path.isdir(abs_objpath):\n pyx_fname = os.path.basename(pyxpath)\n name, ext = os.path.splitext(pyx_fname)\n objpath = os.path.join(objpath, name+objext)\n\n cy_kwargs = cy_kwargs or {}\n cy_kwargs['output_dir'] = cwd\n if cplus is None:\n cplus = pyx_is_cplus(pyxpath)\n cy_kwargs['cplus'] = cplus\n if gdb:\n cy_kwargs['gdb_debug'] = True\n if include_dirs:\n cy_kwargs['include_path'] = include_dirs\n\n interm_c_file = simple_cythonize(\n pyxpath, destdir=interm_c_dir,\n cwd=cwd, logger=logger,\n full_module_name=full_module_name,\n only_update=only_update, **cy_kwargs)\n\n include_dirs = include_dirs or []\n if include_numpy:\n import numpy\n numpy_inc_dir = numpy.get_include()\n if numpy_inc_dir not in include_dirs:\n include_dirs.append(numpy_inc_dir)\n\n flags = kwargs.pop('flags', [])\n needed_flags = ('-fwrapv', '-pthread')\n if not cplus:\n needed_flags += ('-Wstrict-prototypes',) # not really needed..\n for flag in needed_flags:\n if flag not in flags:\n flags.append(flag)\n\n options = kwargs.pop('options', [])\n\n if kwargs.pop('strict_aliasing', False):\n raise CompilationError(\"Cython req. strict aliasing to be disabled.\")\n\n if 'pic' not in options:\n options.append('pic')\n if 'warn' not in options:\n options.append('warn')\n\n # Let's be explicit about standard\n if cplus:\n std = kwargs.pop('std', 'c++98')\n else:\n std = kwargs.pop('std', 'c99')\n\n return src2obj(\n interm_c_file,\n objpath=objpath,\n cwd=cwd,\n only_update=only_update,\n metadir=metadir,\n include_dirs=include_dirs,\n flags=flags,\n std=std,\n options=options,\n logger=logger,\n inc_py=True,\n strict_aliasing=False,\n **kwargs)", "response": "This function generates a C object from a Python source file."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef compile_link_import_py_ext(\n srcs, extname=None, build_dir=None, compile_kwargs=None,\n link_kwargs=None, **kwargs):\n \"\"\"\n Compiles sources in `srcs` to a shared object (python extension)\n which is imported. If shared object is newer than the sources, they\n are not recompiled but instead it is imported.\n\n Parameters\n ----------\n srcs: string\n list of paths to sources\n extname: string\n name of extension (default: None)\n (taken from the last file in `srcs` - without extension)\n build_dir: string\n path to directory in which objects files etc. are generated\n compile_kwargs: dict\n keyword arguments passed to compile_sources\n link_kwargs: dict\n keyword arguments passed to link_py_so\n **kwargs:\n additional keyword arguments overwrites to both compile_kwargs\n and link_kwargs useful for convenience e.g. when passing logger\n\n Returns\n -------\n the imported module\n\n Examples\n --------\n >>> mod = compile_link_import_py_ext(['fft.f90', 'convolution.cpp',\\\n 'fft_wrapper.pyx'], only_update=True) # doctest: +SKIP\n >>> Aprim = mod.fft(A) # doctest: +SKIP\n\n \"\"\"\n\n build_dir = build_dir or '.'\n if extname is None:\n extname = os.path.splitext(os.path.basename(srcs[-1]))[0]\n\n compile_kwargs = compile_kwargs or {}\n compile_kwargs.update(kwargs)\n\n link_kwargs = link_kwargs or {}\n link_kwargs.update(kwargs)\n\n try:\n mod = import_module_from_file(os.path.join(build_dir, extname), srcs)\n except ImportError:\n objs = compile_sources(list(map(get_abspath, srcs)), destdir=build_dir,\n cwd=build_dir, **compile_kwargs)\n so = link_py_so(\n objs, cwd=build_dir, fort=any_fort(srcs), cplus=any_cplus(srcs),\n **link_kwargs)\n mod = import_module_from_file(so)\n return mod", "response": "Compiles sources in srcs to a shared object and links them to a shared object."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef compile_link_import_strings(codes, build_dir=None, **kwargs):\n build_dir = build_dir or tempfile.mkdtemp()\n if not os.path.isdir(build_dir):\n raise OSError(\"Non-existent directory: \", build_dir)\n\n source_files = []\n if kwargs.get('logger', False) is True:\n import logging\n logging.basicConfig(level=logging.DEBUG)\n kwargs['logger'] = logging.getLogger()\n\n only_update = kwargs.get('only_update', True)\n for name, code_ in codes:\n dest = os.path.join(build_dir, name)\n differs = True\n md5_in_mem = md5_of_string(code_.encode('utf-8')).hexdigest()\n if only_update and os.path.exists(dest):\n if os.path.exists(dest+'.md5'):\n md5_on_disk = open(dest+'.md5', 'rt').read()\n else:\n md5_on_disk = md5_of_file(dest).hexdigest()\n differs = md5_on_disk != md5_in_mem\n if not only_update or differs:\n with open(dest, 'wt') as fh:\n fh.write(code_)\n open(dest+'.md5', 'wt').write(md5_in_mem)\n source_files.append(dest)\n\n return compile_link_import_py_ext(\n source_files, build_dir=build_dir, **kwargs)", "response": "Compiles a list of source strings into a temporary directory and dumps and links them into a temporary directory."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nswitch Flask - Pymongo db to a different database", "response": "def switch_db(name):\n \"\"\" Hack to switch Flask-Pymongo db\n :param name: db name\n \"\"\"\n with app.app_context():\n app.extensions['pymongo'][mongo.config_prefix] = mongo.cx, mongo.cx[name]"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef get_row_generator(self, ref, cache=None):\n\n \"\"\"Return a row generator for a reference\"\"\"\n from inspect import isgenerator\n from rowgenerators import get_generator\n\n g = get_generator(ref)\n\n if not g:\n raise GenerateError(\"Cant figure out how to generate rows from {} ref: {}\".format(type(ref), ref))\n else:\n return g", "response": "Return a row generator for a reference"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\ngenerates combinations of key value pairs for each key in properties.", "response": "def create_key_filter(properties: Dict[str, list]) -> List[Tuple]:\n \"\"\"Generate combinations of key, value pairs for each key in properties.\n\n Examples\n --------\n properties = {'ent': ['geo_rev', 'supply_chain'], 'own', 'fi'}\n >> create_key_filter(properties)\n --> [('ent', 'geo_rev'), ('ent', 'supply_chain'), ('own', 'fi')]\n \"\"\"\n\n combinations = (product([k], v) for k, v in properties.items())\n\n return chain.from_iterable(combinations)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef create_indexer(indexes: list):\n\n if len(indexes) == 1:\n index = indexes[0]\n return lambda x: (x[index],)\n else:\n return itemgetter(*indexes)", "response": "Create indexer function to pluck values from list."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef including(self, sequence) -> Generator:\n return (element for element in sequence\n if self.indexer(element) in self.predicates)", "response": "Include the elements matching the filter set."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nexclude the elements matching the filter set.", "response": "def excluding(self, sequence) -> Generator:\n \"\"\"Exclude the sequence elements matching the filter set.\"\"\"\n return (element for element in sequence\n if self.indexer(element) not in self.predicates)"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\nasync def can(self, identity, permission) -> bool:\n assert isinstance(permission, (str, enum.Enum)), permission\n assert permission\n identify = await self.identity_policy.identify(identity)\n # non-registered user still may has some permissions\n access = await self.autz_policy.can(identify, permission)\n return access", "response": "Check user permissions.\n\n :return: ``True`` if the identity is allowed the permission, else return ``False``."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\nasync def check_authorized(self, identity):\n identify = await self.identify(identity)\n if identify is None:\n raise UnauthorizedError()\n return identify", "response": "Checks if the identity is authorized."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\nasync def check_permission(self, identity, permission):\n await self.check_authorized(identity)\n allowed = await self.can(identity, permission)\n if not allowed:\n raise ForbiddenError()", "response": "Checks if the identity has the permission."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef arguments():\n\n DESCRIPTION = \"\"\"\\\n \"\"\"\n\n parser = argparse.ArgumentParser(description=DESCRIPTION, formatter_class=Raw)\n\n parser.add_argument(\"--email\", dest=\"email\", action='store', required=False, default=False,\n help=\"An email address is required for querying Entrez databases.\")\n\n parser.add_argument(\"--api\", dest=\"api_key\", action='store', required=False, default=False,\n help=\"A users ENTREZ API Key. Will speed up download.\")\n\n parser.add_argument(\"--query\", dest=\"query\", action='store', required=True,\n help=\"Query to submit to Entrez.\")\n\n parser.add_argument(\"--host\", dest=\"host\", action='store', required=False, default='localhost',\n help=\"Location of an already running database.\")\n\n parser.add_argument(\"--port\", dest=\"port\", action='store', type=int, required=False, default=27017,\n help=\"Mongo database port.\")\n\n parser.add_argument(\"--db\", dest=\"db\", action='store', required=False, default='sramongo',\n help=\"Name of the database.\")\n\n parser.add_argument(\"--debug\", dest=\"debug\", action='store_true', required=False,\n help=\"Turn on debug output.\")\n\n parser.add_argument(\"--force\", dest=\"force\", action='store_true', required=False,\n help=\"Forces clearing the cache.\")\n\n args = parser.parse_args()\n\n if not (args.email or args.api_key):\n logger.error('You must provide either an `--email` or `--api`.')\n sys.exit()\n\n return args", "response": "Pulls in command line arguments."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nexecute benchmarks on every node specified in arguments.", "response": "def run(self, *nodes):\n \"\"\"Execute benchmarks on every node specified in arguments.\n If none are given, then execute benchmarks on every nodes\n specified in the ``network.nodes`` campaign configuration.\n \"\"\"\n nodes = nodes or self.nodes\n self._prelude(*nodes)\n\n @write_yaml_report\n def _run():\n self._build_installer()\n runner = functools.partial(run_on_host, self.campaign)\n if self.campaign.network.max_concurrent_runs > 1:\n pool = Pool(self.campaign.network.max_concurrent_runs)\n pool.map(runner, nodes)\n else:\n for node in nodes:\n runner(node)\n return nodes\n\n with pushd(self.campaign_path):\n _run()"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nexecuting benchmark on the specified node", "response": "def run(self):\n \"\"\"Execute benchmark on the specified node\n \"\"\"\n with self._scp_bensh_runner():\n self._execute_bensh_runner()\n path = self._retrieve_tarball()\n try:\n self._aggregate_tarball(path)\n finally:\n os.remove(path)"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nget the node pairing attribute of the current tag.", "response": "def node_pairing(self):\n \"\"\"if \"node\" then test current node and next one\n if \"tag\", then create tests for every pair of the current tag.\n \"\"\"\n value = self.attributes['node_pairing']\n if value not in IMB.NODE_PAIRING:\n msg = 'Unexpected {0} value: got \"{1}\" but valid values are {2}'\n msg = msg.format('node_pairing', value, IMB.NODE_PAIRING)\n raise ValueError(msg)\n return value"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef write_makeconfig(_path):\n http_proxy = str(CFG[\"gentoo\"][\"http_proxy\"])\n ftp_proxy = str(CFG[\"gentoo\"][\"ftp_proxy\"])\n rsync_proxy = str(CFG[\"gentoo\"][\"rsync_proxy\"])\n\n path.mkfile_uchroot(local.path('/') / _path)\n with open(_path, 'w') as makeconf:\n lines = '''\nPORTAGE_USERNAME=root\nPORTAGE_GROUPNAME=root\nCFLAGS=\"-O2 -pipe\"\nCXXFLAGS=\"${CFLAGS}\"\nFEATURES=\"nostrip -xattr\"\nCHOST=\"x86_64-pc-linux-gnu\"\nUSE=\"bindist mmx sse sse2\"\nPORTDIR=\"/usr/portage\"\nDISTDIR=\"/mnt/distfiles\"\nPKGDIR=\"${PORTDIR}/packages\"\n'''\n\n makeconf.write(lines)\n\n mounts = CFG[\"container\"][\"mounts\"].value\n tmp_dir = str(CFG[\"tmp_dir\"])\n mounts.append({\"src\": tmp_dir, \"tgt\": \"/mnt/distfiles\"})\n CFG[\"container\"][\"mounts\"] = mounts\n\n if http_proxy is not None:\n http_s = \"http_proxy={0}\".format(http_proxy)\n https_s = \"https_proxy={0}\".format(http_proxy)\n makeconf.write(http_s + \"\\n\")\n makeconf.write(https_s + \"\\n\")\n\n if ftp_proxy is not None:\n fp_s = \"ftp_proxy={0}\".format(ftp_proxy)\n makeconf.write(fp_s + \"\\n\")\n\n if rsync_proxy is not None:\n rp_s = \"RSYNC_PROXY={0}\".format(rsync_proxy)\n makeconf.write(rp_s + \"\\n\")", "response": "Write a valid gentoo make. conf file to the given path."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef write_bashrc(_path):\n cfg_mounts = CFG[\"container\"][\"mounts\"].value\n cfg_prefix = CFG[\"container\"][\"prefixes\"].value\n\n path.mkfile_uchroot(\"/etc/portage/bashrc\")\n mounts = uchroot.mounts(\"mnt\", cfg_mounts)\n p_paths, p_libs = uchroot.env(cfg_prefix)\n paths, libs = uchroot.env(mounts)\n\n paths = paths + p_paths\n libs = libs + p_libs\n\n with open(_path, 'w') as bashrc:\n lines = '''\nexport PATH=\"{0}:${{PATH}}\"\nexport LD_LIBRARY_PATH=\"{1}:${{LD_LIBRARY_PATH}}\"\n'''.format(path.list_to_path(paths), path.list_to_path(libs))\n\n bashrc.write(lines)", "response": "Write a valid gentoo bashrc file to the path."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nwriting a valid gentoo layout file to the path.", "response": "def write_layout(_path):\n \"\"\"\n Write a valid gentoo layout file to :path:.\n\n Args:\n path - The output path of the layout.conf\n \"\"\"\n\n path.mkdir_uchroot(\"/etc/portage/metadata\")\n path.mkfile_uchroot(\"/etc/portage/metadata/layout.conf\")\n with open(_path, 'w') as layoutconf:\n lines = '''masters = gentoo'''\n layoutconf.write(lines)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nwriting a valid gentoo wgetrc file to the path.", "response": "def write_wgetrc(_path):\n \"\"\"\n Write a valid gentoo wgetrc file to :path:.\n\n Args:\n path - The output path of the wgetrc\n \"\"\"\n http_proxy = str(CFG[\"gentoo\"][\"http_proxy\"])\n ftp_proxy = str(CFG[\"gentoo\"][\"ftp_proxy\"])\n\n path.mkfile_uchroot(\"/etc/wgetrc\")\n with open(_path, 'w') as wgetrc:\n if http_proxy is not None:\n http_s = \"http_proxy = {0}\".format(http_proxy)\n https_s = \"https_proxy = {0}\".format(http_proxy)\n wgetrc.write(\"use_proxy = on\\n\")\n wgetrc.write(http_s + \"\\n\")\n wgetrc.write(https_s + \"\\n\")\n\n if ftp_proxy is not None:\n fp_s = \"ftp_proxy={0}\".format(ftp_proxy)\n wgetrc.write(fp_s + \"\\n\")"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef setup_benchbuild():\n LOG.debug(\"Setting up Benchbuild...\")\n\n venv_dir = local.path(\"/benchbuild\")\n prefixes = CFG[\"container\"][\"prefixes\"].value\n prefixes.append(venv_dir)\n CFG[\"container\"][\"prefixes\"] = prefixes\n\n src_dir = str(CFG[\"source_dir\"])\n have_src = src_dir is not None\n if have_src:\n __mount_source(src_dir)\n\n benchbuild = find_benchbuild()\n if benchbuild and not requires_update(benchbuild):\n if have_src:\n __upgrade_from_source(venv_dir, with_deps=False)\n return\n\n setup_virtualenv(venv_dir)\n if have_src:\n __upgrade_from_source(venv_dir)\n else:\n __upgrade_from_pip(venv_dir)", "response": "Setup benchbuild inside a container."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef main(argv=None):\n arguments = cli_common(__doc__, argv=argv)\n es_export = ESExporter(arguments['CAMPAIGN-DIR'], arguments['--es'])\n es_export.export()\n if argv is not None:\n return es_export", "response": "ben - elastic entry point"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nset the name of the object.", "response": "def name(self, value):\n \"\"\"\n Setter for **self.__name** attribute.\n\n :param value: Attribute value.\n :type value: unicode\n \"\"\"\n\n if value is not None:\n assert type(value) is unicode, \"'{0}' attribute: '{1}' type is not 'unicode'!\".format(\"name\", value)\n self.__name = value"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef activated(self, value):\n\n if value is not None:\n assert type(value) is bool, \"'{0}' attribute: '{1}' type is not 'bool'!\".format(\"activated\", value)\n self.component_activated.emit() if value else self.component_deactivated.emit()\n self.__activated = value", "response": "Setter for **self. __activated** attribute."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef initialized(self, value):\n\n if value is not None:\n assert type(value) is bool, \"'{0}' attribute: '{1}' type is not 'bool'!\".format(\"initialized\", value)\n self.component_initialized.emit() if value else self.component_uninitialized.emit()\n self.__initialized = value", "response": "Setter for **self. __initialized** attribute."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef deactivatable(self, value):\n\n if value is not None:\n assert type(value) is bool, \"'{0}' attribute: '{1}' type is not 'bool'!\".format(\"deactivatable\", value)\n self.__deactivatable = value", "response": "Sets the value of the\n ."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef r_cts(self):\n _request = request.args.get(\"request\", None)\n if _request is not None:\n try:\n if _request.lower() == \"getcapabilities\":\n return self._get_capabilities(\n urn=request.args.get(\"urn\", None)\n )\n elif _request.lower() == \"getpassage\":\n return self._get_passage(\n urn=request.args.get(\"urn\", None)\n )\n elif _request.lower() == \"getpassageplus\":\n return self._get_passage_plus(\n urn=request.args.get(\"urn\", None)\n )\n elif _request.lower() == \"getlabel\":\n return self._get_label(\n urn=request.args.get(\"urn\", None)\n )\n elif _request.lower() == \"getfirsturn\":\n return self._get_first_urn(\n urn=request.args.get(\"urn\", None)\n )\n elif _request.lower() == \"getprevnexturn\":\n return self._get_prev_next(\n urn=request.args.get(\"urn\", None)\n )\n elif _request.lower() == \"getvalidreff\":\n return self._get_valid_reff(\n urn=request.args.get(\"urn\", None),\n level=request.args.get(\"level\", 1, type=int)\n )\n except NautilusError as E:\n return self.cts_error(error_name=E.__class__.__name__, message=E.__doc__)\n return self.cts_error(MissingParameter.__name__, message=MissingParameter.__doc__)", "response": "This method handles the main route of the CTS API."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef cts_error(self, error_name, message=None):\n self.nautilus_extension.logger.info(\n \"CTS error thrown {} for {} ({})\".format(\n error_name,\n request.query_string.decode(),\n message)\n )\n return render_template(\n \"cts/Error.xml\",\n errorType=error_name,\n message=message\n ), 404, {\"content-type\": \"application/xml\"}", "response": "Create a CTS Error Response"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _get_capabilities(self, urn=None):\n r = self.resolver.getMetadata(objectId=urn)\n if len(r.parents) > 0:\n r = r.parents[-1]\n r = render_template(\n \"cts/GetCapabilities.xml\",\n filters=\"urn={}\".format(urn),\n inventory=Markup(r.export(Mimetypes.XML.CTS))\n )\n return r, 200, {\"content-type\": \"application/xml\"}", "response": "Provisional route for GetCapabilities request\n "} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _get_first_urn(self, urn):\n urn = URN(urn)\n subreference = None\n textId = urn.upTo(URN.NO_PASSAGE)\n if urn.reference is not None:\n subreference = str(urn.reference)\n firstId = self.resolver.getTextualNode(textId=textId, subreference=subreference).firstId\n r = render_template(\n \"cts/GetFirstUrn.xml\",\n firstId=firstId,\n full_urn=textId,\n request_urn=str(urn)\n )\n return r, 200, {\"content-type\": \"application/xml\"}", "response": "Provisional route for GetFirstUrn request"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef events(self):\n\n\t\twhile True:\n\t\t\tevents = self._selector.select()\n\t\t\tfor nevent in range(len(events) + 1):\n\t\t\t\tself._libinput.libinput_dispatch(self._li)\n\t\t\t\thevent = self._libinput.libinput_get_event(self._li)\n\t\t\t\tif hevent:\n\t\t\t\t\ttype_ = self._libinput.libinput_event_get_type(hevent)\n\t\t\t\t\tself._libinput.libinput_dispatch(self._li)\n\t\t\t\t\tif type_.is_pointer():\n\t\t\t\t\t\tyield PointerEvent(hevent, self._libinput)\n\t\t\t\t\telif type_.is_keyboard():\n\t\t\t\t\t\tyield KeyboardEvent(hevent, self._libinput)\n\t\t\t\t\telif type_.is_touch():\n\t\t\t\t\t\tyield TouchEvent(hevent, self._libinput)\n\t\t\t\t\telif type_.is_gesture():\n\t\t\t\t\t\tyield GestureEvent(hevent, self._libinput)\n\t\t\t\t\telif type_.is_tablet_tool():\n\t\t\t\t\t\tyield TabletToolEvent(hevent, self._libinput)\n\t\t\t\t\telif type_.is_tablet_pad():\n\t\t\t\t\t\tyield TabletPadEvent(hevent, self._libinput)\n\t\t\t\t\telif type_.is_switch():\n\t\t\t\t\t\tyield SwitchEvent(hevent, self._libinput)\n\t\t\t\t\telif type_.is_device():\n\t\t\t\t\t\tyield DeviceNotifyEvent(hevent, self._libinput)", "response": "Yields events from the internal libinput s queue."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nreturns the type of the next available event in the internal queue.", "response": "def next_event_type(self):\n\t\t\"\"\"Return the type of the next event in the internal queue.\n\n\t\tThis method does not pop the event off the queue and the next call\n\t\tto :attr:`events` returns that event.\n\n\t\tReturns:\n\t\t\t~libinput.constant.EventType: The event type of the next available\n\t\t\tevent or :obj:`None` if no event is available.\n\t\t\"\"\"\n\n\t\ttype_ = self._libinput.libinput_next_event_type(self._li)\n\t\tif type_ == 0:\n\t\t\treturn None\n\t\telse:\n\t\t\treturn EventType(type_)"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef add_device(self, path):\n\n\t\thdevice = self._libinput.libinput_path_add_device(\n\t\t\tself._li, path.encode())\n\t\tif hdevice:\n\t\t\treturn Device(hdevice, self._libinput)\n\t\treturn None", "response": "Adds a device to a libinput context."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef assign_seat(self, seat):\n\n\t\trc = self._libinput.libinput_udev_assign_seat(self._li, seat.encode())\n\t\tassert rc == 0, 'Failed to assign {}'.format(seat)", "response": "Assign a seat to this libinput context."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef context(self, outdir, log_prefix):\n try:\n self._outdir = outdir\n self._log_prefix = log_prefix\n yield\n finally:\n self._log_prefix = None\n self._outdir = None", "response": "Setup instance to extract metrics from the proper run directory."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef get_subclass(cls, name):\n for subclass in cls.__subclasses__():\n if subclass.name == name:\n return subclass\n raise NameError(\"Not a valid Benchmark class: \" + name)", "response": "Get a Benchmark class by name"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nattempts to automatically find \"adjacent\" shapes in a SVG layer. In a layer within a new SVG document, draw each detected connection between the center points of the corresponding shapes. Parameters ---------- svg_source : str Input SVG file as a filepath (or file-like object). shape_i_attr : str, optional Attribute of each shape SVG element that uniquely identifies the shape. layer_name : str, optional Name to use for the output layer where detected connections are drawn. .. note:: Any existing layer with the same name will be overwritten. shapes_xpath : str, optional XPath path expression to select shape nodes. By default, all ``svg:path`` and ``svg:polygon`` elements are selected. extend : float, optional Extend ``x``/``y`` coords by the specified number of absolute units from the center point of each shape. Each shape is stretched independently in the ``x`` and ``y`` direction. In each direction, a shape is considered adjacent to all other shapes that are overlapped by the extended shape. Returns ------- StringIO.StringIO File-like object containing SVG document with layer named according to :data:`layer_name` with the detected connections drawn as ``svg:line`` instances.", "response": "def auto_detect_adjacent_shapes(svg_source, shape_i_attr='id',\n layer_name='Connections',\n shapes_xpath='//svg:path | //svg:polygon',\n extend=1.5):\n '''\n Attempt to automatically find \"adjacent\" shapes in a SVG layer.\n\n In a layer within a new SVG document, draw each detected connection between\n the center points of the corresponding shapes.\n\n Parameters\n ----------\n svg_source : str\n Input SVG file as a filepath (or file-like object).\n shape_i_attr : str, optional\n Attribute of each shape SVG element that uniquely identifies the shape.\n layer_name : str, optional\n Name to use for the output layer where detected connections are drawn.\n\n .. note:: Any existing layer with the same name will be overwritten.\n shapes_xpath : str, optional\n XPath path expression to select shape nodes.\n\n By default, all ``svg:path`` and ``svg:polygon`` elements are selected.\n extend : float, optional\n Extend ``x``/``y`` coords by the specified number of absolute units\n from the center point of each shape.\n\n Each shape is stretched independently in the ``x`` and ``y`` direction.\n In each direction, a shape is considered adjacent to all other shapes\n that are overlapped by the extended shape.\n\n Returns\n -------\n StringIO.StringIO\n File-like object containing SVG document with layer named according to\n :data:`layer_name` with the detected connections drawn as ``svg:line``\n instances.\n '''\n # Read SVG polygons into dataframe, one row per polygon vertex.\n df_shapes = svg_shapes_to_df(svg_source, xpath=shapes_xpath)\n df_shapes = compute_shape_centers(df_shapes, shape_i_attr)\n df_shape_connections = extract_adjacent_shapes(df_shapes, shape_i_attr,\n extend=extend)\n\n # Parse input file.\n xml_root = etree.parse(svg_source)\n svg_root = xml_root.xpath('/svg:svg', namespaces=INKSCAPE_NSMAP)[0]\n\n # Get the center coordinate of each shape.\n df_shape_centers = (df_shapes.drop_duplicates(subset=[shape_i_attr])\n [[shape_i_attr] + ['x_center', 'y_center']]\n .set_index(shape_i_attr))\n\n # Get the center coordinate of the shapes corresponding to the two\n # endpoints of each connection.\n df_connection_centers = (df_shape_centers.loc[df_shape_connections.source]\n .reset_index(drop=True)\n .join(df_shape_centers.loc[df_shape_connections\n .target]\n .reset_index(drop=True), lsuffix='_source',\n rsuffix='_target'))\n\n # Remove existing connections layer from source, in-memory XML (source file\n # remains unmodified). A new connections layer will be added below.\n connections_xpath = '//svg:g[@inkscape:label=\"%s\"]' % layer_name\n connections_groups = svg_root.xpath(connections_xpath,\n namespaces=INKSCAPE_NSMAP)\n\n if connections_groups:\n for g in connections_groups:\n g.getparent().remove(g)\n\n # Create in-memory SVG\n svg_output = \\\n draw_lines_svg_layer(df_connection_centers\n .rename(columns={'x_center_source': 'x_source',\n 'y_center_source': 'y_source',\n 'x_center_target': 'x_target',\n 'y_center_target': 'y_target'}),\n layer_name=layer_name)\n\n return svg_output"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nflattening a dictionary into a single dictionary.", "response": "def flatten_dict(dic, parent_key='', sep='.'):\n \"\"\"Flatten sub-keys of a dictionary\n \"\"\"\n items = []\n for key, value in dic.items():\n new_key = parent_key + sep + key if parent_key else key\n if isinstance(value, collections.MutableMapping):\n items.extend(flatten_dict(value, new_key, sep=sep).items())\n elif isinstance(value, list):\n for idx, elt in enumerate(value):\n items.extend(\n flatten_dict(elt, new_key + sep + str(idx), sep=sep).items()\n )\n else:\n items.append((new_key, value))\n return dict(items)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef dict_merge(dct, merge_dct):\n for key in merge_dct.keys():\n if (\n key in dct\n and isinstance(dct[key], dict)\n and isinstance(merge_dct[key], collections.Mapping)\n ):\n dict_merge(dct[key], merge_dct[key])\n else:\n dct[key] = merge_dct[key]", "response": "Recursive dict merge. Inspired by dict. update."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\ntransforming tree of dict and list in read - only data structure.", "response": "def freeze(obj):\n \"\"\"Transform tree of dict and list in read-only\n data structure.\n dict instances are transformed to FrozenDict, lists\n in FrozenList.\n \"\"\"\n if isinstance(obj, collections.Mapping):\n return FrozenDict({freeze(k): freeze(v) for k, v in six.iteritems(obj)})\n elif isinstance(obj, list):\n return FrozenList([freeze(e) for e in obj])\n else:\n return obj"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\ncreates a Configuration from a YAML file.", "response": "def from_file(cls, path):\n \"\"\"Create a ``Configuration`` from a file\n\n :param path: path to YAML file\n :return: new configuration\n :rtype: ``Configuration``\n \"\"\"\n if path == '-':\n return Configuration(yaml.safe_load(sys.stdin))\n if not osp.exists(path) and not osp.isabs(path):\n path = osp.join(osp.dirname(osp.abspath(__file__)), path)\n with open(path, 'r') as istr:\n return Configuration(yaml.safe_load(istr))"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nconverting a hex - formatted number to an anatomical color tuple.", "response": "def hex_color_to_rgba(hex_color, normalize_to=255):\n '''\n Convert a hex-formatted number (i.e., `\"#RGB[A]\"` or `\"#RRGGBB[AA]\"`) to an\n RGBA tuple (i.e., `(, , , )`).\n\n Args:\n\n hex_color (str) : hex-formatted number (e.g., `\"#2fc\"`, `\"#3c2f8611\"`)\n normalize_to (int, float) : Factor to normalize each channel by\n\n Returns:\n\n (tuple) : RGBA tuple (i.e., `(, , , )`), where range of\n each channel in tuple is `[0, normalize_to]`.\n '''\n color_pattern_one_digit = (r'#(?P[\\da-fA-F])(?P[\\da-fA-F])'\n r'(?P[\\da-fA-F])(?P[\\da-fA-F])?')\n color_pattern_two_digit = (r'#(?P[\\da-fA-F]{2})(?P[\\da-fA-F]{2})'\n r'(?P[\\da-fA-F]{2})(?P[\\da-fA-F]{2})?')\n\n # First try to match `#rrggbb[aa]`.\n match = re.match(color_pattern_two_digit, hex_color)\n\n if match:\n channels = match.groupdict()\n channel_scale = 255\n else:\n # Try to match `#rgb[a]`.\n match = re.match(color_pattern_one_digit, hex_color)\n if match:\n channels = match.groupdict()\n channel_scale = 15\n else:\n raise ValueError('Color string must be in format #RGB[A] or '\n '#RRGGBB[AA] (i.e., alpha channel is optional)')\n\n scale = normalize_to / channel_scale\n return tuple(type(normalize_to)(int(channels[k], 16) * scale)\n if channels[k] is not None else None\n for k in 'RGBA')"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nsplits a term into parent and record term components", "response": "def split_term(cls, term):\n \"\"\"\n Split a term in to parent and record term components\n :param term: combined term text\n :return: Tuple of parent and record term\n \"\"\"\n\n if '.' in term:\n parent_term, record_term = term.split('.')\n parent_term, record_term = parent_term.strip(), record_term.strip()\n\n if parent_term == '':\n parent_term = ELIDED_TERM\n\n else:\n parent_term, record_term = ROOT_TERM, term.strip()\n\n return parent_term, record_term"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef split_term_lower(cls, term):\n\n return tuple(e.lower() for e in Term.split_term(term))", "response": "Like split_term but also lowercases both parent and record term\n "} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nreturns a string for the file row and column of the term.", "response": "def file_ref(self):\n \"\"\"Return a string for the file, row and column of the term.\"\"\"\n from metatab.util import slugify\n\n assert self.file_name is None or isinstance(self.file_name, str)\n\n if self.file_name is not None and self.row is not None:\n parts = split(self.file_name);\n return \"{} {}:{} \".format(parts[-1], self.row, self.col)\n elif self.row is not None:\n return \" {}:{} \".format(self.row, self.col)\n else:\n return ''"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nadd a term to this term s children. Also sets the child term s parent.", "response": "def add_child(self, child):\n \"\"\"Add a term to this term's children. Also sets the child term's parent\"\"\"\n assert isinstance(child, Term)\n self.children.append(child)\n child.parent = self\n assert not child.term_is(\"Datafile.Section\")"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\ncreates a new term and add it to this term as a child.", "response": "def new_child(self, term, value, **kwargs):\n \"\"\"Create a new term and add it to this term as a child. Creates grandchildren from the kwargs.\n\n :param term: term name. Just the record term\n :param term: Value to assign to the term\n :param term: Term properties, which create children of the child term.\n\n \"\"\"\n\n tc = self.doc.get_term_class(term.lower())\n\n c = tc(term, str(value) if value is not None else None,\n parent=self, doc=self.doc, section=self.section).new_children(**kwargs)\n\n c.term_value_name = self.doc.decl_terms.get(c.join, {}).get('termvaluename', c.term_value_name)\n\n assert not c.term_is(\"*.Section\")\n self.children.append(c)\n return c"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef remove_child(self, child):\n assert isinstance(child, Term)\n self.children.remove(child)\n self.doc.remove_term(child)", "response": "Removes the term from this term s children."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef new_children(self, **kwargs):\n for k, v in kwargs.items():\n self.new_child(k, v)\n\n return self", "response": "Create new children from kwargs"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef set_ownership(self):\n assert self.section is not None\n\n for t in self.children:\n t.parent = self\n t._section = self.section\n t.doc = self.doc\n t.set_ownership()", "response": "Recursivelt set the parent section and doc for a children"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef find(self, term, value=False):\n if '.' in term:\n parent, term = term.split('.')\n assert parent.lower() == self.record_term_lc, (parent.lower(), self.record_term_lc)\n\n for c in self.children:\n if c.record_term_lc == term.lower():\n if value is False or c.value == value:\n yield c", "response": "Return a list of terms by name."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nfind a term using find_first and set it s value and properties and create a new child term. If it does not exist create a new child term.", "response": "def get_or_new_child(self, term, value=False, **kwargs):\n \"\"\"Find a term, using find_first, and set it's value and properties, if it exists. If\n it does not, create a new term and children. \"\"\"\n\n pt, rt = self.split_term(term)\n\n term = self.record_term + '.' + rt\n\n c = self.find_first(rt)\n\n if c is None:\n tc = self.doc.get_term_class(term.lower())\n c = tc(term, value, parent=self, doc=self.doc, section=self.section).new_children(**kwargs)\n assert not c.term_is(\"Datafile.Section\"), (self, c)\n self.children.append(c)\n\n else:\n if value is not False:\n c.value = value\n\n for k, v in kwargs.items():\n c.get_or_new_child(k, v)\n\n # Check that the term was inserted and can be found.\n assert self.find_first(rt)\n assert self.find_first(rt) == c\n\n return c"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef get_value(self, item, default=None):\n try:\n return self[item].value\n except (AttributeError, KeyError) as e:\n return default", "response": "Get the value of a child"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nreturn the fully qualified term name.", "response": "def qualified_term(self):\n \"\"\"Return the fully qualified term name. The parent will be 'root' if there is no parent term defined. \"\"\"\n\n assert self.parent is not None or self.parent_term_lc == 'root'\n\n if self.parent:\n return self.parent.record_term_lc + '.' + self.record_term_lc\n else:\n return 'root.' + self.record_term_lc"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef term_is(self, v):\n\n if isinstance(v, str):\n\n if '.' not in v:\n v = 'root.' + v\n\n v_p, v_r = self.split_term_lower(v)\n\n if self.join_lc == v.lower():\n return True\n elif v_r == '*' and v_p == self.parent_term_lc:\n return True\n elif v_p == '*' and v_r == self.record_term_lc:\n return True\n elif v_p == '*' and v_r == '*':\n return True\n else:\n return False\n\n else:\n\n return any(self.term_is(e) for e in v)", "response": "Return True if the fully qualified name of the term is the same as the argument."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef arg_props(self):\n\n d = dict(zip([str(e).lower() for e in self.section.property_names], self.args))\n\n # print({ c.record_term_lc:c.value for c in self.children})\n d[self.term_value_name.lower()] = self.value\n\n return d", "response": "Return the value and scalar properties as a dictionary. Returns only the value and scalar properties declared on the same row as a term. Returns only the value and scalar properties declared on the same row as a term."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef all_props(self):\n\n d = self.arg_props\n d.update(self.props)\n\n return d", "response": "Returns a dictionary with the values of all children and place holders for all of the section\n argumemts. It combines arg_props with props"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef _convert_to_dict(cls, term, replace_value_names=True):\n\n from collections import OrderedDict\n\n if not term:\n return None\n\n if term.children:\n\n d = OrderedDict()\n\n for c in term.children:\n\n if c.child_property_type == 'scalar':\n d[c.record_term_lc] = cls._convert_to_dict(c, replace_value_names)\n\n elif c.child_property_type == 'sequence':\n try:\n d[c.record_term_lc].append(cls._convert_to_dict(c, replace_value_names))\n except (KeyError, AttributeError):\n # The c.term property doesn't exist, so add a list\n d[c.record_term_lc] = [cls._convert_to_dict(c, replace_value_names)]\n\n elif c.child_property_type == 'sconcat': # Concat with a space\n\n if c.record_term_lc in d:\n s = d[c.record_term_lc] + ' '\n else:\n s = ''\n\n d[c.record_term_lc] =s + (cls._convert_to_dict(c, replace_value_names) or '')\n\n elif c.child_property_type == 'bconcat': # Concat with a blank\n\n d[c.record_term_lc] = d.get(c.record_term_lc, '') + (cls._convert_to_dict(c,\n replace_value_names)\n or '')\n\n else:\n try:\n d[c.record_term_lc].append(cls._convert_to_dict(c, replace_value_names))\n except KeyError:\n # The c.term property doesn't exist, so add a scalar or a map\n d[c.record_term_lc] = cls._convert_to_dict(c, replace_value_names)\n except AttributeError as e:\n # d[c.term] exists, but is a scalar, so convert it to a list\n\n d[c.record_term_lc] = [d[c.record_term]] + [cls._convert_to_dict(c, replace_value_names)]\n\n if term.value:\n if replace_value_names:\n d[term.term_value_name.lower()] = term.value\n else:\n d['@value'] = term.value\n\n return d\n\n else:\n return term.value", "response": "Converts a record heirarchy to nested dicts."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nyielding the rows for the term for writing terms to a CSV file.", "response": "def rows(self):\n \"\"\"Yield rows for the term, for writing terms to a CSV file. \"\"\"\n\n # Translate the term value name so it can be assigned to a parameter.\n tvm = self.section.doc.decl_terms.get(self.qualified_term, {}).get('termvaluename', '@value')\n assert tvm\n # Terminal children have no arguments, just a value. Here we put the terminal children\n # in a property array, so they can be written to the parent's arg-children columns\n # if the section has any.\n\n properties = {tvm: self.value}\n\n for c in self.children:\n if c.is_terminal:\n if c.record_term_lc:\n # This is rare, but can happen if a property child is not given\n # a property name by the section -- the \"Section\" term has a blank column.\n properties[c.record_term_lc] = c.value\n\n yield (self.qualified_term, properties)\n\n # The non-terminal children have to get yielded normally -- they can't be arg-children\n for c in self.children:\n if not c.is_terminal:\n for row in c.rows:\n yield row"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef descendents(self):\n\n for c in self.children:\n yield c\n\n for d in c.descendents:\n yield d", "response": "Iterate over all descendent terms"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef subclass(cls, t):\n t.doc = None\n t.terms = []\n t.__class__ = SectionTerm\n return t", "response": "Change a term into a Section Term"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef add_arg(self,arg, prepend=False):\n\n self.args = [arg_.strip() for arg_ in self.args if arg_.strip()]\n\n if arg.title() not in self.args:\n if prepend:\n self.args = [arg.title()] + self.args\n else:\n self.args.append(arg.title())", "response": "Append an arg to the arg list"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef remove_arg(self, arg):\n\n self.args = [arg_.strip() for arg_ in self.args if arg_.strip()]\n\n for arg_ in list(self.args):\n if arg_.lower() == arg.lower():\n self.args.remove(arg_)", "response": "Remove an arg from the arg list"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef add_term(self, t):\n if t not in self.terms:\n if t.parent_term_lc == 'root':\n self.terms.append(t)\n\n self.doc.add_term(t, add_section=False)\n\n t.set_ownership()\n\n else:\n raise GenerateError(\"Can only add or move root-level terms. Term '{}' parent is '{}' \"\n .format(t, t.parent_term_lc))\n\n assert t.section or t.join_lc == 'root.root', t", "response": "Add a term to this section and set it s ownership. Should only be used on root level terms"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef new_term(self, term, value, **kwargs):\n\n tc = self.doc.get_term_class(term.lower())\n\n t = tc(term, value, doc=self.doc, parent=None, section=self).new_children(**kwargs)\n\n self.doc.add_term(t)\n return t", "response": "Create a new root - level term in this section"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\ncreate or set top level terms in the section.", "response": "def set_terms(self,*terms, **kw_terms):\n \"\"\"\n Create or set top level terms in the section. After python 3.6.0, the terms entries\n should maintain the same order as the argument list. The term arguments can have any of these forms:\n\n * For position argument, a Term object\n * For kw arguments:\n - 'TermName=TermValue'\n - 'TermName=(TermValue, PropertyDict)\n\n Positional arguments are processed before keyword arguments, and are passed into .add_term()\n\n :param terms: Term arguments\n :return:\n \"\"\"\n\n for t in terms:\n self.add_term(t)\n\n for k,v in kw_terms.items():\n try:\n value, props = v\n except (ValueError, TypeError) as e:\n value, props = v,{}\n\n self.new_term(k,value,**props)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef remove_term(self, term, remove_from_doc=True):\n\n try:\n self.terms.remove(term)\n except ValueError:\n pass\n\n if remove_from_doc:\n self.doc.remove_term(term)", "response": "Remove a term from the terms. Must be the identical term the same object."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef clean(self):\n terms = list(self)\n\n for t in terms:\n self.doc.remove_term(t)", "response": "Remove all of the terms from the section and also remove them from the document"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nsort the terms in the section.", "response": "def sort_by_term(self, order=None):\n \"\"\"\n Sort the terms in the section.\n :param order: If specified, a list of qualified, lowercased term names. These names will appear\n first in the section, and the remaining terms will be sorted alphabetically. If not specified, all terms are\n alphabetized.\n :return:\n \"\"\"\n\n if order is None:\n self.terms = sorted(self.terms, key=lambda e: e.join_lc)\n else:\n\n all_terms = list(self.terms)\n sorted_terms = []\n\n for tn in order:\n for t in list(all_terms):\n\n if t.term_is(tn):\n all_terms.remove(t)\n sorted_terms.append(t)\n\n sorted_terms.extend(sorted(all_terms, key=lambda e: e.join_lc))\n\n self.terms = sorted_terms"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _args(self, term, d):\n\n # Get the term value name, the property name that should be assigned to the term value.\n tvm = self.doc.decl_terms.get(term, {}).get('termvaluename', '@value')\n\n # Convert the keys to lower case\n\n lower_d = {k.lower(): v for k, v in d.items()}\n\n args = []\n for n in [tvm] + self.property_names:\n args.append(lower_d.get(n.lower(), ''))\n\n try:\n del lower_d[n.lower()]\n except KeyError:\n pass\n\n return term, args, lower_d", "response": "Extract the chldren of a term that are arg - children from those that are row - children."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nyield the rows for the section", "response": "def rows(self):\n \"\"\"Yield rows for the section\"\"\"\n for t in self.terms:\n\n for row in t.rows:\n term, value = row # Value can either be a string, or a dict\n\n if isinstance(value, dict): # Dict is for properties, which might be arg-children\n term, args, remain = self._args(term, value)\n yield term, args\n\n # 'remain' is all of the children that didn't have an arg-child column -- the\n # section didn't have a column heder for that ther.\n for k, v in remain.items():\n yield term.split('.')[-1] + '.' + k, v\n else:\n yield row"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\niterate over all of the rows as text lines", "response": "def lines(self):\n \"\"\"Iterate over all of the rows as text lines\"\"\"\n\n # Yield the section header\n if self.name != 'Root':\n yield ('Section', '|'.join([self.value] + self.property_names))\n\n # Yield all of the rows for terms in the section\n for row in self.rows:\n term, value = row\n\n if not isinstance(value, (list, tuple)):\n value = [value]\n\n term = term.replace('root.', '').title()\n yield (term, value[0])\n\n children = list(zip(self.property_names, value[1:]))\n\n for prop, value in children:\n if value and value.strip():\n child_t = '.' + (prop.title())\n yield (\" \"+child_t, value)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nreturn the whole section as a dict", "response": "def as_dict(self, replace_value_names=True):\n \"\"\"Return the whole section as a dict\"\"\"\n old_children = self.children\n self.children = self.terms\n\n d = super(SectionTerm, self).as_dict(replace_value_names)\n\n self.children = old_children\n\n return d"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef find_compiler(cls, preferred_vendor, metadir, cwd,\n use_meta=True):\n \"\"\"\n Identify a suitable C/fortran/other compiler\n\n When it is possible that the user (un)installs a compiler\n inbetween compilations of object files we want to catch\n that. This method allows compiler choice to be stored in a\n pickled metadata file. Provide metadir a dirpath to\n make the class save choice there in a file with\n cls.metadata_filename as name.\n \"\"\"\n cwd = cwd or '.'\n metadir = metadir or '.'\n metadir = os.path.join(cwd, metadir)\n used_metafile = False\n if not preferred_vendor and use_meta:\n try:\n preferred_vendor = cls.get_from_metadata_file(\n metadir, 'vendor')\n used_metafile = True\n except FileNotFoundError:\n pass\n candidates = list(cls.compiler_dict.keys())\n if preferred_vendor:\n if preferred_vendor in candidates:\n candidates = [preferred_vendor]+candidates\n else:\n raise ValueError(\"Unknown vendor {}\".format(\n preferred_vendor))\n name, path = find_binary_of_command([\n cls.compiler_dict[x] for x in candidates])\n if use_meta and not used_metafile:\n if not os.path.isdir(metadir):\n raise FileNotFoundError(\"Not a dir: {}\".format(metadir))\n cls.save_to_metadata_file(metadir, 'compiler',\n (name, path))\n cls.save_to_metadata_file(\n metadir, 'vendor',\n cls.compiler_name_vendor_mapping[name])\n if cls.logger:\n cls.logger.info(\n 'Wrote choice of compiler to: metadir')\n return name, path, cls.compiler_name_vendor_mapping[name]", "response": "Find a suitable C compiler and return its name path and vendor."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nreturn the command to run this module.", "response": "def cmd(self):\n \"\"\"\n The command below covers most cases, if you need\n someting more complex subclass this.\n \"\"\"\n cmd = (\n [self.compiler_binary] +\n self.flags +\n ['-U'+x for x in self.undef] +\n ['-D'+x for x in self.define] +\n ['-I'+x for x in self.include_dirs] +\n self.sources\n )\n if self.run_linker:\n cmd += (['-L'+x for x in self.library_dirs] +\n [(x if os.path.exists(x) else '-l'+x) for x in self.libraries] +\n self.linkline)\n counted = []\n for envvar in re.findall('\\$\\{(\\w+)\\}', ' '.join(cmd)):\n if os.getenv(envvar) is None:\n if envvar not in counted:\n counted.append(envvar)\n msg = \"Environment variable '{}' undefined.\".format(\n envvar)\n self.logger.error(msg)\n raise CompilationError(msg)\n return cmd"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\ndrawing a line on the current layer with the current brush.", "response": "def drawLine(self, x1, y1, x2, y2, silent=False):\n\t\t\"\"\"\n\t\tDraws a line on the current :py:class:`Layer` with the current :py:class:`Brush`.\n\t\tCoordinates are relative to the original layer size WITHOUT downsampling applied.\n\n\t\t:param x1: Starting X coordinate.\n\t\t:param y1: Starting Y coordinate.\n\t\t:param x2: End X coordinate.\n\t\t:param y2: End Y coordinate.\n\t\t:rtype: Nothing.\n\t\t\"\"\"\n\t\tstart = time.time()\n\n\t\t#Downsample the coordinates\n\t\tx1 = int(x1/config.DOWNSAMPLING)\n\t\tx2 = int(x2/config.DOWNSAMPLING)\n\t\ty1 = int(y1/config.DOWNSAMPLING)\n\t\ty2 = int(y2/config.DOWNSAMPLING)\n\t\tif not silent :\n\t\t\tprint('drawing line from: '+str((x1,y1))+' to: '+str((x2,y2)))\n\n\t\t#Calculate the direction and the length of the step\n\t\tdirection = N.arctan2(y2 - y1, x2 - x1)\n\t\tlength = self.brush.spacing\n\n\t\t#Prepare the loop\n\t\tx, y = x1, y1\n\t\ttotalSteps = int(N.sqrt((x2 - x)**2 + (y2 - y)**2)/length)\n\t\tlay = self.image.getActiveLayer()\n\t\tcol = self.color\n\t\tsecCol = self.secondColor\n\t\tmirr = self.mirrorMode\n\n\t\t#If I use source caching..\n\t\tif self.brush.usesSourceCaching:\n\t\t\t#..than optimize it for faster drawing\n\t\t\tlaydata = lay.data\n\t\t\tx -= self.brush.brushSize*0.5\n\t\t\ty -= self.brush.brushSize*0.5\n\t\t\tcolbrsource = self.brush.coloredBrushSource\n\t\t\tcanvSize = config.CANVAS_SIZE\n\t\t\tbrmask = self.brush.brushMask\n\t\t\tfor _ in range(totalSteps):\n\t\t\t\t#Make the dab on this point\n\t\t\t\tapplyMirroredDab_jit(mirr, laydata, int(x), int(y), colbrsource.copy(), canvSize, brmask)\n\n\t\t\t\t#Mode the point for the next step and update the distances\n\t\t\t\tx += lendir_x(length, direction)\n\t\t\t\ty += lendir_y(length, direction)\n\t\t#..if I don't use source caching..\n\t\telse:\n\t\t\t#..do the normal drawing\n\t\t\tfor _ in range(totalSteps):\n\t\t\t\t#Make the dab on this point\n\t\t\t\tself.brush.makeDab(lay, int(x), int(y), col, secCol, mirror=mirr)\n\n\t\t\t\t#Mode the point for the next step and update the distances\n\t\t\t\tx += lendir_x(length, direction)\n\t\t\t\ty += lendir_y(length, direction)"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef drawPoint(self, x, y, silent=True):\n\t\tstart = time.time()\n\n\t\t#Downsample the coordinates\n\t\tx = int(x/config.DOWNSAMPLING)\n\t\ty = int(y/config.DOWNSAMPLING)\n\n\t\t#Apply the dab with or without source caching\n\t\tif self.brush.usesSourceCaching:\n\t\t\tapplyMirroredDab_jit(self.mirrorMode, self.image.getActiveLayer().data, int(x-self.brush.brushSize*0.5), int(y-self.brush.brushSize*0.5), self.brush.coloredBrushSource.copy(), config.CANVAS_SIZE, self.brush.brushMask)\n\t\telse:\n\t\t\tself.brush.makeDab(self.image.getActiveLayer(), int(x), int(y), self.color, self.secondColor, mirror=self.mirrorMode)\n\t\tconfig.AVGTIME.append(time.time()-start)", "response": "Draws a point on the current layer with the current brush."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\ndraws a series of lines on the current layer with the current brush.", "response": "def drawPath(self, pointList):\n\t\t\"\"\"\n\t\tDraws a series of lines on the current :py:class:`Layer` with the current :py:class:`Brush`.\n\t\tNo interpolation is applied to these point and :py:meth:`drawLine` will be used to connect all the points lineraly.\n\t\tCoordinates are relative to the original layer size WITHOUT downsampling applied.\n\t\t\n\t\t:param pointList: A list of point like :code:`[(0, 0), (100, 100), (100, 200)]`.\n\t\t:rtype: Nothing.\n\t\t\"\"\"\n\t\tself.drawLine(pointList[0][0], pointList[0][1], pointList[1][0], pointList[1][1])\n\t\ti = 1\n\t\twhile i str:\n \"\"\"Convert string from camel case to snake case.\"\"\"\n\n return CAMEL_CASE_RE.sub(r'_\\1', s).strip().lower()"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef snake_to_camel(s: str) -> str:\n\n fragments = s.split('_')\n\n return fragments[0] + ''.join(x.title() for x in fragments[1:])", "response": "Convert string from snake case to camel case."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nbuild metrics extractors according to the metrics section", "response": "def _create_extractors(cls, metrics):\n \"\"\"Build metrics extractors according to the `metrics` config\n\n :param metrics: Benchmark `metrics` configuration section\n \"\"\"\n metrics_dict = {}\n # group entries by `category` attribute (default is \"standard\")\n for metric, config in six.iteritems(metrics):\n category = config.get('category', StdBenchmark.DEFAULT_CATEGORY)\n metrics_dict.setdefault(category, {})[metric] = config\n # create one StdExtractor instance per category,\n # passing associated metrics\n return dict(\n (category, StdExtractor(metrics))\n for category, metrics in six.iteritems(metrics_dict)\n )"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef metrics(self):\n return dict(\n (name, getattr(Metrics, config['type']))\n for name, config in six.iteritems(self._metrics)\n )", "response": "Returns a dictionary of metrics extracted by this class"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef froms(self):\n eax = {}\n for name, config in six.iteritems(self._metrics):\n from_ = self._get_property(config, 'from', default=self.stdout)\n eax.setdefault(from_, {})[name] = config\n return eax", "response": "Group metrics according to the from property."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef tesselate_shapes_frame(df_shapes, shape_i_columns):\n '''\n Tesselate each shape path into one or more triangles.\n\n Parameters\n ----------\n df_shapes : pandas.DataFrame\n Table containing vertices of shapes, one row per vertex, with the *at\n least* the following columns:\n - ``x``: The x-coordinate of the vertex.\n - ``y``: The y-coordinate of the vertex.\n shape_i_columns : str or list\n Column(s) forming key to differentiate rows/vertices for each distinct\n shape.\n\n Returns\n -------\n pandas.DataFrame\n\n Table where each row corresponds to a triangle vertex, with the following\n columns:\n\n - ``shape_i_columns[]``: The shape path index column(s).\n - ``triangle_i``: The integer triangle index within each electrode path.\n - ``vertex_i``: The integer vertex index within each triangle.\n '''\n frames = []\n if isinstance(shape_i_columns, bytes):\n shape_i_columns = [shape_i_columns]\n\n for shape_i, df_path in df_shapes.groupby(shape_i_columns):\n points_i = df_path[['x', 'y']].values\n if (points_i[0] == points_i[-1]).all():\n # XXX End point is the same as the start point (do not include it).\n points_i = points_i[:-1]\n try:\n triangulator = Triangulator(points_i)\n except:\n import pdb; pdb.set_trace()\n continue\n if not isinstance(shape_i, (list, tuple)):\n shape_i = [shape_i]\n\n for i, triangle_i in enumerate(triangulator.triangles()):\n triangle_points_i = [shape_i + [i] + [j, x, y]\n for j, (x, y) in enumerate(triangle_i)]\n frames.extend(triangle_points_i)\n frames = None if not frames else frames\n return pd.DataFrame(frames, columns=shape_i_columns +\n ['triangle_i', 'vertex_i', 'x', 'y'])", "response": "Tesselate each shape path into one or more triangles."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef capture_stdout():\n stdout = sys.stdout\n sys.stdout = six.moves.cStringIO()\n try:\n yield sys.stdout\n finally:\n sys.stdout = stdout", "response": "Intercept standard output in a with - context\n "} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nchanges current working directory in a with-context :param mkdir: If True, then directory is created if it does not exist :param cleanup: If True and no pre-existing directory, the directory is cleaned up at the end", "response": "def pushd(path, mkdir=True, cleanup=False):\n \"\"\"Change current working directory in a with-context\n :param mkdir: If True, then directory is created if it does not exist\n :param cleanup: If True and no pre-existing directory, the directory is\n cleaned up at the end\n \"\"\"\n cwd = os.getcwd()\n exists = osp.exists(path)\n if mkdir and not exists:\n os.makedirs(path)\n os.chdir(path)\n try:\n yield path\n finally:\n os.chdir(cwd)\n if not exists and cleanup:\n # NB: should we be checking for rmtree.avoids_symlink_attacks ?\n shutil.rmtree(path)"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\ncreate a temporary directory in a with - context keyword arguments.", "response": "def mkdtemp(*args, **kwargs):\n \"\"\"Create a temporary directory in a with-context\n\n keyword remove: Remove the directory when leaving the\n context if True. Default is True.\n other keywords arguments are given to the tempfile.mkdtemp\n function.\n \"\"\"\n remove = kwargs.pop('remove', True)\n path = tempfile.mkdtemp(*args, **kwargs)\n try:\n yield path\n finally:\n if remove:\n shutil.rmtree(path)"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef elapsed(self):\n if self.end is None:\n return self() - self.end\n return self.end - self.start", "response": "Returns the amount of time spent in the context."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nreturn a slug of the given string.", "response": "def slugify(value):\n \"\"\"\n Normalizes string, converts to lowercase, removes non-alpha characters,\n and converts spaces to hyphens.type(\n \"\"\"\n import re\n import unicodedata\n value = str(value)\n value = unicodedata.normalize('NFKD', value).encode('ascii', 'ignore').decode('utf8').strip().lower()\n value = re.sub(r'[^\\w\\s\\-\\.]', '', value)\n value = re.sub(r'[-\\s]+', '-', value)\n return value"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef import_name_or_class(name):\n \" Import an obect as either a fully qualified, dotted name, \"\n\n if isinstance(name, str):\n\n # for \"a.b.c.d\" -> [ 'a.b.c', 'd' ]\n module_name, object_name = name.rsplit('.',1)\n # __import__ loads the multi-level of module, but returns\n # the top level, which we have to descend into\n mod = __import__(module_name)\n\n components = name.split('.')\n\n for comp in components[1:]: # Already got the top level, so start at 1\n\n mod = getattr(mod, comp)\n return mod\n else:\n return name", "response": "Import an obect as either a fully qualified dotted name"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef pip_installer_url(version=None):\n version = version or hpcbench.__version__\n version = str(version)\n if '.dev' in version:\n git_rev = 'master'\n if 'TRAVIS_BRANCH' in os.environ:\n git_rev = version.split('+', 1)[-1]\n if '.' in git_rev: # get rid of date suffix\n git_rev = git_rev.split('.', 1)[0]\n git_rev = git_rev[1:] # get rid of scm letter\n return 'git+{project_url}@{git_rev}#egg=hpcbench'.format(\n project_url='http://github.com/BlueBrain/hpcbench',\n git_rev=git_rev or 'master',\n )\n return 'hpcbench=={}'.format(version)", "response": "Return the URL to pip to install HPCBench."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nloads the campaign from YAML file", "response": "def from_file(campaign_file, **kwargs):\n \"\"\"Load campaign from YAML file\n\n :return: memory representation of the YAML file\n :rtype: dictionary\n \"\"\"\n realpath = osp.realpath(campaign_file)\n if osp.isdir(realpath):\n campaign_file = osp.join(campaign_file, YAML_CAMPAIGN_FILE)\n campaign = Configuration.from_file(campaign_file)\n return default_campaign(campaign, **kwargs)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef default_campaign(\n campaign=None, expandcampvars=True, exclude_nodes=None, frozen=True\n):\n \"\"\"Fill an existing campaign with default\n values for optional keys\n\n :param campaign: dictionary\n :type campaign: str\n :param exclude_nodes: node set to exclude from allocations\n :type exclude_nodes: str\n :param expandcampvars: should env variables be expanded? True by default\n :type expandcampvars: bool\n :param frozen: whether the returned data-structure is immutable or not\n :type frozen: bool\n :return: object provided in parameter\n :rtype: dictionary\n \"\"\"\n campaign = campaign or nameddict()\n\n def _merger(_camp, _deft):\n for key in _deft.keys():\n if (\n key in _camp\n and isinstance(_camp[key], dict)\n and isinstance(_deft[key], collections.Mapping)\n ):\n _merger(_camp[key], _deft[key])\n elif key not in _camp:\n _camp[key] = _deft[key]\n\n _merger(campaign, DEFAULT_CAMPAIGN)\n campaign.setdefault('campaign_id', str(uuid.uuid4()))\n\n for precondition in campaign.precondition.keys():\n config = campaign.precondition[precondition]\n if not isinstance(config, list):\n campaign.precondition[precondition] = [config]\n\n def _expandvars(value):\n if isinstance(value, six.string_types):\n return expandvars(value)\n return value\n\n if expandcampvars:\n campaign = nameddict(dict_map_kv(campaign, _expandvars))\n else:\n campaign = nameddict(campaign)\n if expandcampvars:\n if campaign.network.get('tags') is None:\n campaign.network['tags'] = {}\n NetworkConfig(campaign).expand()\n return freeze(campaign) if frozen else campaign", "response": "Fill an existing campaign with default values for optional keys\n "} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\ngetting of benchmarks referenced in the configuration", "response": "def get_benchmark_types(campaign):\n \"\"\"Get of benchmarks referenced in the configuration\n\n :return: benchmarks\n :rtype: string generator\n \"\"\"\n for benchmarks in campaign.benchmarks.values():\n for name, benchmark in benchmarks.items():\n if name != 'sbatch': # exclude special sbatch name\n yield benchmark.type"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef get_metrics(campaign, report, top=True):\n if top and campaign.process.type == 'slurm':\n for path, _ in report.collect('jobid', with_path=True):\n for child in ReportNode(path).children.values():\n for metrics in get_metrics(campaign, child, top=False):\n yield metrics\n else:\n\n def metrics_node_extract(report):\n metrics_file = osp.join(report.path, JSON_METRICS_FILE)\n if osp.exists(metrics_file):\n with open(metrics_file) as istr:\n return json.load(istr)\n\n def metrics_iterator(report):\n return filter(\n lambda eax: eax[1] is not None,\n report.map(metrics_node_extract, with_path=True),\n )\n\n for path, metrics in metrics_iterator(report):\n yield report.path_context(path), metrics", "response": "Extract metrics from existing campaign\n "} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nwrites YAML campaign template to the given open file.", "response": "def write(self, file):\n \"\"\"Write YAML campaign template to the given open file\n \"\"\"\n render(\n self.template,\n file,\n benchmarks=self.benchmarks,\n hostname=socket.gethostname(),\n )"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef expand(self):\n if self.slurm:\n self._introspect_slurm_cluster()\n self.network.nodes = self._expand_nodes(self.network.nodes)\n self._expand_tags()", "response": "Perform node expansion of network section."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef ensure_has_same_campaigns(self):\n lhs_yaml = osp.join(self.lhs, 'campaign.yaml')\n rhs_yaml = osp.join(self.rhs, 'campaign.yaml')\n assert osp.isfile(lhs_yaml)\n assert osp.isfile(rhs_yaml)\n assert filecmp.cmp(lhs_yaml, rhs_yaml)", "response": "Ensure that the 2 campaigns to merge have been generated from the same campaign. yaml"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nbuild a dictionary of fields extracted from the given path", "response": "def path_context(self, path):\n \"\"\"Build of dictionary of fields extracted from\n the given path\"\"\"\n prefix = os.path.commonprefix([path, self._path])\n relative_path = path[len(prefix) :]\n relative_path = relative_path.strip(os.sep)\n attrs = self.CONTEXT_ATTRS\n for i, elt in enumerate(relative_path.split(os.sep)):\n yield attrs[i], elt\n yield 'path', path"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef children(self):\n for child in self.data.get('children', []):\n if osp.exists(osp.join(self.path, child, YAML_REPORT_FILE)):\n yield child, self.__class__(osp.join(self.path, child))", "response": "get children node referenced as children in the\n report."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef collect(self, *keys, **kwargs):\n if not keys:\n raise Exception('Missing key')\n has_values = functools.reduce(\n operator.__and__, [key in self.data for key in keys], True\n )\n if has_values:\n values = tuple([self.data[key] for key in keys])\n if len(values) == 1:\n values = values[0]\n if kwargs.get('with_path', False):\n yield self.path, values\n else:\n yield values\n if kwargs.get('recursive', True):\n for child in self.children.values():\n for value in child.collect(*keys, **kwargs):\n yield value", "response": "Generator function traversing the tree structure to collect values of a specified key."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef collect_one(self, *args, **kwargs):\n generator = self.collect(*args, **kwargs)\n try:\n value = next(generator)\n except StopIteration:\n raise Exception(\"Expected exactly one value don't have any\")\n try:\n next(generator)\n except StopIteration:\n return value\n raise Exception('Expected exactly one value but have more')", "response": "Same as collect but expects to have only one result."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nset properties of the beams in the structure.", "response": "def set_beam_prop(self, prop, values, repeat=\"up\"):\n \"\"\"\n Specify the properties of the beams\n\n :param values:\n :param repeat: if 'up' then duplicate up the structure\n :return:\n \"\"\"\n values = np.array(values)\n if repeat == \"up\":\n assert len(values.shape) == 1\n values = [values for ss in range(self.n_storeys)]\n else:\n assert len(values.shape) == 2\n if len(values[0]) != self.n_bays:\n raise ModelError(\"beam depths does not match number of bays (%i).\" % self.n_bays)\n for ss in range(self.n_storeys):\n for i in range(self.n_bays):\n self._beams[ss][i].set_section_prop(prop, values[0][i])"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef set_column_prop(self, prop, values, repeat=\"up\"):\n values = np.array(values)\n if repeat == \"up\":\n assert len(values.shape) == 1\n values = [values for ss in range(self.n_storeys)]\n else:\n assert len(values.shape) == 2\n if len(values[0]) != self.n_cols:\n raise ModelError(\"column props does not match n_cols (%i).\" % self.n_cols)\n for ss in range(self.n_storeys):\n for i in range(self.n_cols):\n self._columns[ss][i].set_section_prop(prop, values[0][i])", "response": "Set the properties of the columns in the structure."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nload the config file.", "response": "def load(self, path):\n \"\"\" \n Load yaml-formatted config file.\n\n Parameters\n ----------\n path : str\n path to config file\n \"\"\"\n with open(path) as f:\n self.config = full_load(f)\n if self.config is None:\n sys.stderr.write(\"Warning: config file is empty!\\n\")\n self.config = {}"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nsaving current state of config dictionary.", "response": "def save(self):\n \"\"\" \n Save current state of config dictionary.\n \"\"\"\n with open(self.config_file, \"w\") as f:\n f.write(dump(self.config, default_flow_style=False))"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nregistering an extension into the Nautilus Router", "response": "def register(self, extension, extension_name):\n \"\"\" Register an extension into the Nautilus Router\n\n :param extension: Extension\n :param extension_name: Name of the Extension\n :return:\n \"\"\"\n self._extensions[extension_name] = extension\n self.ROUTES.extend([\n tuple(list(t) + [extension_name])\n for t in extension.ROUTES\n ])\n self.CACHED.extend([\n (f_name, extension_name)\n for f_name in extension.CACHED\n ])\n\n # This order allows for user defaults to overwrite extension ones\n self.Access_Control_Allow_Methods.update({\n k: v\n for k, v in extension.Access_Control_Allow_Methods.items()\n if k not in self.Access_Control_Allow_Methods\n })"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef setLogger(self, logger):\n self.logger = logger\n if logger is None:\n self.logger = logging.getLogger(\"capitains_nautilus\")\n formatter = logging.Formatter(\"[%(asctime)s] {%(pathname)s:%(lineno)d} %(levelname)s - %(message)s\")\n stream = FlaskNautilus.LoggingHandler()\n stream.setLevel(logging.INFO)\n stream.setFormatter(formatter)\n self.logger.addHandler(stream)\n\n if self.resolver:\n self.resolver.logger = self.logger\n\n return self.logger", "response": "Set up the Logger for the application."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef init_app(self, app):\n\n self.init_blueprint(app)\n\n if self.flaskcache is not None:\n for func, extension_name in self.CACHED:\n func = getattr(self._extensions[extension_name], func)\n setattr(\n self._extensions[extension_name],\n func.__name__,\n self.flaskcache.memoize()(func)\n )\n\n return self.blueprint", "response": "Initiate the extension on the application and return the blueprint for the application that has been registered in app\n ."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef init_blueprint(self, app):\n self.blueprint = Blueprint(\n self.name,\n self.name,\n template_folder=resource_filename(\"capitains_nautilus\", \"data/templates\"),\n url_prefix=self.prefix\n )\n\n # Register routes\n for url, name, methods, extension_name in self.ROUTES:\n self.blueprint.add_url_rule(\n url,\n view_func=self.view(name, extension_name),\n endpoint=name[2:],\n methods=methods\n )\n\n app.register_blueprint(self.blueprint)\n return self.blueprint", "response": "This method creates the blueprint for the extension and registers routes and filters and connecting the app and the blueprints of the extension."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nbuild a response according to a function name and extension name.", "response": "def view(self, function_name, extension_name):\n \"\"\" Builds response according to a function name\n\n :param function_name: Route name / function name\n :param extension_name: Name of the extension holding the function\n :return: Function\n \"\"\"\n if isinstance(self.Access_Control_Allow_Origin, dict):\n d = {\n \"Access-Control-Allow-Origin\": self.Access_Control_Allow_Origin[function_name],\n \"Access-Control-Allow-Methods\": self.Access_Control_Allow_Methods[function_name]\n }\n else:\n d = {\n \"Access-Control-Allow-Origin\": self.Access_Control_Allow_Origin,\n \"Access-Control-Allow-Methods\": self.Access_Control_Allow_Methods[function_name]\n }\n\n def r(*x, **y):\n val = getattr(self._extensions[extension_name], function_name)(*x, **y)\n if isinstance(val, Response):\n val.headers.extend(d)\n return val\n else:\n val = list(val)\n val[2].update(d)\n return tuple(val)\n return r"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\npreparing a slurm script that executes the experiment for a given project.", "response": "def script(experiment, projects):\n \"\"\"\n Prepare a slurm script that executes the experiment for a given project.\n\n Args:\n experiment: The experiment we want to execute\n projects: All projects we generate an array job for.\n \"\"\"\n benchbuild_c = local[local.path(sys.argv[0])]\n slurm_script = local.cwd / experiment.name + \"-\" + str(\n CFG['slurm']['script'])\n\n srun = local[\"srun\"]\n srun_args = []\n if not CFG[\"slurm\"][\"multithread\"]:\n srun_args.append(\"--hint=nomultithread\")\n if not CFG[\"slurm\"][\"turbo\"]:\n srun_args.append(\"--pstate-turbo=off\")\n\n srun = srun[srun_args]\n srun = srun[benchbuild_c[\"run\"]]\n\n return __save__(slurm_script, srun, experiment, projects)"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nfetching the text structure of a given work from a CTS endpoint.", "response": "def fetch_text_structure(urn, endpoint=\"http://cts.perseids.org/api/cts\"):\n \"\"\"\n Fetches the text structure of a given work from a CTS endpoint.\n\n :param urn: the work's CTS URN (at the work-level!,\n e.g.\"urn:cts:greekLit:tlg0012.tlg001\")\n :type urn: string\n :param endpoint: the URL of the CTS endpoint to use (defaults to Perseids')\n :type endpoint: string\n :return: a dict with keys \"urn\", \"provenance\", \"valid_reffs\", \"levels\"\n :rtype: dict\n \"\"\"\n\n structure = {\n \"urn\": urn,\n \"provenance\": endpoint,\n \"valid_reffs\": {}\n }\n\n orig_edition = None\n suffix = 'grc' if 'greekLit' in urn else 'lat'\n resolver = HttpCtsResolver(HttpCtsRetriever(endpoint))\n work_metadata = resolver.getMetadata(urn)\n\n # among all editions for this work, pick the one in Greek or Latin\n try:\n orig_edition = next(iter(\n work_metadata.children[edition]\n for edition in work_metadata.children\n if suffix in str(work_metadata.children[edition].urn)\n ))\n except Exception as e:\n print(e)\n return None\n\n assert orig_edition is not None\n\n # get information about the work's citation scheme\n structure[\"levels\"] = [\n (n + 1, level.name.lower())\n for n, level in enumerate(orig_edition.citation)\n ]\n\n # for each hierarchical level of the text\n # fetch all citable text elements\n for level_n, level_label in structure[\"levels\"]:\n\n structure[\"valid_reffs\"][level_n] = []\n for ref in resolver.getReffs(urn, level=level_n):\n print(ref)\n element = {\n \"current\": \"{}:{}\".format(urn, ref),\n }\n if \".\" in ref:\n element[\"parent\"] = \"{}:{}\".format(\n urn,\n \".\".join(ref.split(\".\")[:level_n - 1])\n )\n # TODO: parallelize this bit\n textual_node = resolver.getTextualNode(\n textId=urn,\n subreference=ref,\n prevnext=True\n )\n if textual_node.nextId is not None:\n element[\"previous\"] = \"{}:{}\".format(urn, textual_node.nextId)\n if textual_node.prevId is not None:\n element[\"following\"] = \"{}:{}\".format(urn, textual_node.prevId)\n structure[\"valid_reffs\"][level_n].append(element)\n\n return structure"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nexecuting all actions but don t do anything as extension.", "response": "def actions_for_project(self, project):\n \"\"\"Execute all actions but don't do anything as extension.\"\"\"\n project.runtime_extension = run.RuntimeExtension(project, self)\n return self.default_runtime_actions(project)"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef install_uchroot(_):\n builddir = local.path(str(CFG[\"build_dir\"].value))\n with local.cwd(builddir):\n erlent_src = local.path('erlent')\n erlent_git = erlent_src / '.git'\n erlent_repo = str(CFG['uchroot']['repo'])\n erlent_build = erlent_src / 'build'\n if not erlent_git.exists():\n git(\"clone\", erlent_repo)\n else:\n with local.cwd(erlent_src):\n git(\"pull\", \"--rebase\")\n\n erlent_build.mkdir()\n with local.cwd(erlent_build):\n cmake(\"../\")\n make()\n\n os.environ[\"PATH\"] = os.path.pathsep.join(\n [erlent_build, os.environ[\"PATH\"]])\n local.env.update(PATH=os.environ[\"PATH\"])\n\n if not find_package(\"uchroot\"):\n LOG.error('uchroot not found, after updating PATH to %s',\n os.environ['PATH'])\n sys.exit(-1)\n\n env = CFG['env'].value\n if 'PATH' not in env:\n env['PATH'] = []\n env['PATH'].append(str(erlent_build))", "response": "Installs erlent and erlent build."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nprint a small summary of the executed actions.", "response": "def print_summary(num_actions, failed, duration):\n \"\"\"\n Print a small summary of the executed plan.\n\n Args:\n num_actions (int): Total size of the executed plan.\n failed (:obj:`list` of :obj:`actions.Step`): List of failed actions.\n duration: Time we spent executing the plan.\n \"\"\"\n num_failed = len(failed)\n print(\"\"\"\nSummary:\n{num_total} actions were in the queue.\n{num_failed} actions failed to execute.\n\nThis run took: {elapsed_time:8.3f} seconds.\n \"\"\".format(\n num_total=num_actions, num_failed=num_failed, elapsed_time=duration))"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef read_json_sensor_file(ffp):\n sensor_path = ffp\n si = json.load(open(sensor_path))\n for m_type in si:\n # Convert keys from strings to integers\n si[m_type] = {int(k): v for k, v in si[m_type].items()}\n return si", "response": "Reads the sensor file and stores it as a dictionary."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nget all sensor codes that match a wild sensor code", "response": "def get_all_sensor_codes(si, wild_sensor_code):\n \"\"\"\n Get all sensor sensor_codes that match a wild sensor code\n\n :param si: dict, sensor index json dictionary\n :param wild_sensor_code: str, a sensor code with \"*\" for wildcards (e.g. ACCX-*-L2C-*)\n :return:\n \"\"\"\n mtype_and_ory, x, y, z = wild_sensor_code.split(\"-\")\n if mtype_and_ory == \"*\":\n mtypes = list(si)\n elif mtype_and_ory[-1] in \"XYZ\" and \"ACCX\" not in si: # Need to support old sensor_file.json files.\n mtypes = [mtype_and_ory[:-1]]\n else:\n mtypes = [mtype_and_ory]\n\n all_sensor_codes = []\n for mtype in mtypes:\n for m_number in si[mtype]:\n if x in [\"*\", si[mtype][m_number]['X-CODE']] and \\\n y in [\"*\", si[mtype][m_number]['Y-CODE']] and \\\n z in [\"*\", si[mtype][m_number]['Z-CODE']]:\n cc = get_sensor_code_by_number(si, mtype, m_number)\n all_sensor_codes.append(cc)\n\n return all_sensor_codes"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef get_sensor_code_by_number(si, mtype, sensor_number, quiet=False):\n try:\n if 'Orientation' in si[mtype][sensor_number]:\n orientation = si[mtype][sensor_number]['Orientation']\n else:\n orientation = \"\"\n return \"%s%s-%s-%s-%s\" % (mtype,\n orientation,\n si[mtype][sensor_number]['X-CODE'],\n si[mtype][sensor_number]['Y-CODE'],\n si[mtype][sensor_number]['Z-CODE'])\n except KeyError:\n if quiet:\n return None\n raise", "response": "Given a sensor number get the full sensor code"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef get_mtype_and_number_from_code(si, sensor_code):\n mtype_and_ory, x, y, z = sensor_code.split(\"-\")\n if mtype_and_ory[-1] in \"XYZ\" and \"ACCX\" not in si: # Need to support old sensor_file.json files.\n mtype = mtype_and_ory[:-1]\n else:\n mtype = mtype_and_ory\n for m_number in si[mtype]:\n cc = get_sensor_code_by_number(si, mtype, m_number)\n if cc == sensor_code:\n return mtype, m_number\n return None, None", "response": "Given a sensor code get the motion type and sensor number from the sensor index json dictionary."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef PCExtension(*args, **kwargs):\n vals = {}\n\n intercept = {\n 'build_callbacks': (), # tuple of (callback, args, kwargs)\n 'link_ext': True,\n 'build_files': (),\n 'dist_files': (), # work around stackoverflow.com/questions/2994396/\n 'template_regexps': [],\n 'pass_extra_compile_args': False, # use distutils extra_compile_args?\n 'pycompilation_compile_kwargs': {},\n 'pycompilation_link_kwargs': {},\n }\n for k, v in intercept.items():\n vals[k] = kwargs.pop(k, v)\n\n intercept2 = {\n 'logger': None,\n 'only_update': True,\n }\n for k, v in intercept2.items():\n vck = kwargs.pop(k, v)\n vck = vals['pycompilation_compile_kwargs'].pop(k, vck)\n vck = vck or vals['pycompilation_link_kwargs'].pop(k, vck)\n vals[k] = vck\n\n instance = Extension(*args, **kwargs)\n\n if vals['logger'] is True:\n # interpret as we should instantiate a logger\n import logging\n logging.basicConfig(level=logging.DEBUG)\n vals['logger'] = logging.getLogger('PCExtension')\n\n for k, v in vals.items():\n setattr(instance, k, v)\n\n return instance", "response": "Returns a new object of type Extension with the given arguments and keyword arguments."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\ncopies or render a source file to output_dir.", "response": "def _copy_or_render_source(ext, f, output_dir, render_callback,\n skip_copy=False):\n \"\"\"\n Tries to do regex match for each (pattern, target, subsd) tuple\n in ext.template_regexps for file f.\n \"\"\"\n # Either render a template or copy the source\n dirname = os.path.dirname(f)\n filename = os.path.basename(f)\n for pattern, target, subsd in ext.template_regexps:\n if re.match(pattern, filename):\n tgt = os.path.join(dirname, re.sub(\n pattern, target, filename))\n rw = MetaReaderWriter('.metadata_subsd')\n try:\n prev_subsd = rw.get_from_metadata_file(output_dir, f)\n except (FileNotFoundError, KeyError):\n prev_subsd = None\n\n render_callback(\n get_abspath(f),\n os.path.join(output_dir, tgt),\n subsd,\n only_update=ext.only_update,\n prev_subsd=prev_subsd,\n create_dest_dirs=True,\n logger=ext.logger)\n rw.save_to_metadata_file(output_dir, f, subsd)\n return tgt\n else:\n if not skip_copy:\n copy(f,\n os.path.join(output_dir,\n os.path.dirname(f)),\n only_update=ext.only_update,\n dest_is_dir=True,\n create_dest_dirs=True,\n logger=ext.logger)\n return f"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef render_python_template_to(src, dest, subsd, only_update=False,\n prev_subsd=None, create_dest_dirs=True,\n logger=None):\n \"\"\"\n Overload this function if you want to use a template engine such as\n e.g. mako.\n \"\"\"\n if only_update:\n if subsd == prev_subsd:\n if not missing_or_other_newer(dest, src):\n if logger:\n msg = (\"Did not re-render {}. \"\n \"(destination newer + same dict)\")\n logger.info(msg.format(src))\n return\n\n with open(src, 'rt') as ifh:\n data = ifh.read() # Don't go crazy on file size...\n\n if create_dest_dirs:\n dest_dir = os.path.dirname(dest)\n if not os.path.exists(dest_dir):\n make_dirs(dest_dir)\n\n with open(dest, 'wt') as ofh:\n ofh.write(data % subsd)", "response": "Render a python template file to a new location."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nreturning a list of names in that language.", "response": "def get_names(self):\n \"\"\"\n Returns a dict where key is the language and value is the name in that language.\n\n Example:\n {'it':\"Sofocle\"}\n \"\"\"\n names = [id for id in self.ecrm_P1_is_identified_by if id.uri == surf.ns.EFRBROO['F12_Name']]\n self.names = []\n for name in names:\n for variant in name.rdfs_label:\n self.names.append((variant.language,variant.title()))\n return self.names"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef add_name(self, name, lang=None):\n try:\n assert (lang, name) not in self.get_names()\n except Exception as e:\n # TODO: raise a custom exception\n logger.warning(\"Duplicate name detected while adding \\\"%s (lang=%s)\\\"\"%(name, lang))\n return False\n newlabel = Literal(name, lang=lang) if lang is not None else \\\n Literal(name)\n name = [\n id\n for id in self.ecrm_P1_is_identified_by\n if id.uri == surf.ns.EFRBROO['F12_Name']\n ][0]\n try:\n name.rdfs_label.append(newlabel)\n name.update()\n return True\n except Exception as e:\n raise e", "response": "Adds a new name variant to an author."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef add_abbreviation(self, new_abbreviation):\n try:\n assert new_abbreviation not in self.get_abbreviations()\n except Exception as e:\n # TODO: raise a custom exception\n logger.warning(\"Duplicate abbreviation detected while adding \\\"%s\\\"\"%new_abbreviation)\n return False\n\n try:\n type_abbreviation = self.session.get_resource(BASE_URI_TYPES % \"abbreviation\"\n , self.session.get_class(surf.ns.ECRM['E55_Type']))\n abbreviation = [abbreviation\n for name in self.ecrm_P1_is_identified_by\n for abbreviation in name.ecrm_P139_has_alternative_form\n if name.uri == surf.ns.EFRBROO['F12_Name']\n and abbreviation.ecrm_P2_has_type.first == type_abbreviation][0]\n abbreviation.rdfs_label.append(Literal(new_abbreviation))\n abbreviation.update()\n return True\n except IndexError as e:\n # means there is no abbreviation instance yet\n type_abbreviation = self.session.get_resource(BASE_URI_TYPES % \"abbreviation\"\n , self.session.get_class(surf.ns.ECRM['E55_Type']))\n Appellation = self.session.get_class(surf.ns.ECRM['E41_Appellation'])\n abbreviation_uri = \"%s/abbr\" % str(self.subject)\n abbreviation = Appellation(abbreviation_uri)\n abbreviation.ecrm_P2_has_type = type_abbreviation\n abbreviation.rdfs_label.append(Literal(new_abbreviation))\n abbreviation.save()\n\n name = (name for name in self.ecrm_P1_is_identified_by if name.uri == surf.ns.EFRBROO['F12_Name']).next()\n name.ecrm_P139_has_alternative_form = abbreviation\n name.update()\n return True\n except Exception as e:\n raise e", "response": "Adds a new name variant to an author."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\ngetting abbreviations of the names of the author.", "response": "def get_abbreviations(self):\n \"\"\"\n Get abbreviations of the names of the author.\n\n :return: a list of strings (empty list if no abbreviations available).\n \"\"\"\n abbreviations = []\n try:\n type_abbreviation = self.session.get_resource(BASE_URI_TYPES % \"abbreviation\"\n , self.session.get_class(surf.ns.ECRM['E55_Type']))\n abbreviations = [unicode(label)\n for name in self.ecrm_P1_is_identified_by\n for abbreviation in name.ecrm_P139_has_alternative_form\n for label in abbreviation.rdfs_label\n if name.uri == surf.ns.EFRBROO['F12_Name']\n and abbreviation.ecrm_P2_has_type.first == type_abbreviation]\n except Exception as e:\n logger.debug(\"Exception raised when getting abbreviations for %a\"%self)\n finally:\n return abbreviations"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef get_urn(self):\n # TODO: check type\n try:\n type_ctsurn = self.session.get_resource(BASE_URI_TYPES % \"CTS_URN\"\n , self.session.get_class(surf.ns.ECRM['E55_Type']))\n urn = [CTS_URN(urnstring.rdfs_label.one)\n for urnstring in self.ecrm_P1_is_identified_by\n if urnstring.uri == surf.ns.ECRM['E42_Identifier']\n and urnstring.ecrm_P2_has_type.first == type_ctsurn][0]\n return urn\n except Exception as e:\n return None", "response": "Returns the Urn of the HucitAuthor s CTS."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nchanging the CTS URN of the author or adds a new one.", "response": "def set_urn(self,urn):\n \"\"\"\n Change the CTS URN of the author or adds a new one (if no URN is assigned).\n \"\"\"\n Type = self.session.get_class(surf.ns.ECRM['E55_Type'])\n Identifier = self.session.get_class(surf.ns.ECRM['E42_Identifier'])\n id_uri = \"%s/cts_urn\"%str(self.subject)\n try:\n id = Identifier(id_uri)\n id.rdfs_label = Literal(urn)\n id.ecrm_P2_has_type = Type(BASE_URI_TYPES % \"CTS_URN\")\n id.save()\n return True\n except Exception, e:\n raise e"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef to_json(self):\n names = self.get_names()\n return json.dumps({\n \"uri\" : self.subject\n , \"urn\" : str(self.get_urn())\n , \"names\" : [{\"language\":lang, \"label\":label} for lang, label in names]\n , \"name_abbreviations\" : self.get_abbreviations()\n , \"works\" : [json.loads(work.to_json()) for work in self.get_works()]\n }, indent=2)", "response": "Serialises a HucitAuthor to a JSON formatted string."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nreturn a list of tuples containing language and title.", "response": "def get_titles(self):\n \"\"\"TODO\"\"\"\n return [(label.language, unicode(label)) for label in self.efrbroo_P102_has_title.first.rdfs_label]"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef get_abbreviations(self, combine=False):\n abbreviations = []\n try:\n type_abbreviation = self.session.get_resource(BASE_URI_TYPES % \"abbreviation\"\n , self.session.get_class(surf.ns.ECRM['E55_Type']))\n abbreviations = [unicode(label)\n for title in self.efrbroo_P102_has_title\n for abbreviation in title.ecrm_P139_has_alternative_form\n for label in abbreviation.rdfs_label\n if title.uri == surf.ns.EFRBROO['E35_Title']\n and abbreviation.ecrm_P2_has_type.first == type_abbreviation]\n\n if (combine and len(abbreviations)>0 and len(self.author.get_abbreviations())>=1):\n abbreviations = [\"%s %s\"%(author_abbrev, work_abbrev)\n for author_abbrev, work_abbrev in itertools.product(self.author.get_abbreviations()\n , self.get_abbreviations())]\n except Exception as e:\n logger.debug(\"Exception raised when getting abbreviations for %a\"%self)\n finally:\n return abbreviations", "response": "Get the abbreviations of the titles of the work."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nadding a citable text structure to the work.", "response": "def add_text_structure(self, label):\n \"\"\"\n Adds a citable text structure to the work.\n \"\"\"\n\n ts = self.session.get_resource(\"%s/text_structure\" % self.subject\n , self.session.get_class(surf.ns.HUCIT['TextStructure']))\n ts.rdfs_label.append(Literal(label))\n ts.save()\n self.hucit_has_structure = ts\n self.update()\n return self.hucit_has_structure.one"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef remove_text_structure(self, text_structure): # TODO: delete also TextElements one by one\n idx = self.hucit_has_structure.index(text_structure)\n ts = self.hucit_has_structure.pop(idx)\n ts.remove()\n self.update()\n return", "response": "Remove any citable text structure from the work."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef _get_opus_maximum(self):\n label = \"\"\"The opux maximum of a given author\n that is, the only preserved work by that\n author or the most known one.\"\"\"\n\n opmax = self.session.get_resource(\n BASE_URI_TYPES % \"opmax\",\n self.session.get_class(surf.ns.ECRM['E55_Type'])\n )\n if opmax.is_present():\n return opmax\n else:\n opmax.rdfs_label.append(Literal(label, \"en\"))\n logger.debug(\"Created a new opus maximum type instance\")\n opmax.save()\n return opmax", "response": "Instantiate an opus maximum type."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef set_as_opus_maximum(self): # TODO: test\n if self.is_opus_maximum():\n return False\n else:\n opmax = self._get_opus_maximum()\n self.ecrm_P2_has_type = opmax\n self.update()\n return True", "response": "Mark the work as the author s opus maximum."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nchecking whether the work is the author s opus maximum.", "response": "def is_opus_maximum(self):\n \"\"\"Check whether the work is the author's opus maximum.\n\n Two cases:\n 1. the work is flagged as opus max\n 2. there is only one work by this author\n\n :return: boolean\n \"\"\"\n opmax = self._get_opus_maximum()\n types = self.ecrm_P2_has_type\n\n if opmax in types:\n return True\n else:\n if len(self.author.get_works()) == 1:\n return True\n else:\n return False"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nreturn the author to whom the work is attributed.", "response": "def author(self):\n \"\"\"\n Returns the author to whom the work is attributed.\n\n :return: an instance of `HucitWork` # TODO: check that's the case\n \"\"\"\n CreationEvent = self.session.get_class(surf.ns.EFRBROO['F27_Work_Conception'])\n Person = self.session.get_class(surf.ns.EFRBROO['F10_Person'])\n creation_event = CreationEvent.get_by(efrbroo_R16_initiated=self).first()\n return Person.get_by(efrbroo_P14i_performed = creation_event).first()"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nserialises a HucitWork to a JSON formatted string.", "response": "def to_json(self):\n \"\"\"\n Serialises a HucitWork to a JSON formatted string.\n \"\"\"\n titles = self.get_titles()\n return json.dumps({\n \"uri\" : self.subject\n , \"urn\" : str(self.get_urn())\n , \"titles\" : [{\"language\":lang, \"label\":label} for lang, label in titles]\n , \"title_abbreviations\" : self.get_abbreviations()\n\n }, indent=2)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\ngetting the ULID of the one - item item.", "response": "def get_urn(self):\n \"\"\"\n TODO\n \"\"\"\n urn = self.ecrm_P1_is_identified_by.one\n try:\n return CTS_URN(urn)\n except Exception, e:\n raise e"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef nodes(self, tag):\n if tag == '*':\n return sorted(list(set(self.campaign.network.nodes)))\n definitions = self.campaign.network.tags.get(tag)\n if definitions is None:\n return []\n nodes = set()\n for definition in definitions:\n if len(definition.items()) == 0:\n continue\n mode, value = list(definition.items())[0]\n if mode == 'match':\n nodes = nodes.union(\n set(\n [\n node\n for node in self.campaign.network.nodes\n if value.match(node)\n ]\n )\n )\n elif mode == 'constraint':\n slurm_nodes = os.environ.get('SLURM_JOB_NODELIST')\n if slurm_nodes:\n nodes = NodeSet(slurm_nodes)\n else:\n return ConstraintTag(tag, value)\n else:\n assert mode == 'nodes'\n nodes = nodes.union(set(value))\n return sorted(list(nodes))", "response": "get nodes that belong to a tag"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef execution_cls(self):\n name = self.campaign.process.type\n for clazz in [ExecutionDriver, SrunExecutionDriver]:\n if name == clazz.name:\n return clazz\n raise NameError(\"Unknown execution layer: '%s'\" % name)", "response": "Get execution layer class"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef children(self):\n tags = {'*'}\n if self.tag:\n network_tags = {self.tag: self.campaign.network.tags[self.tag]}\n else:\n network_tags = self.campaign.network.tags\n for tag, configs in network_tags.items():\n for config in configs:\n for mode, kconfig in config.items():\n if mode == 'match':\n if kconfig.match(self.name) or kconfig.match(LOCALHOST):\n tags.add(tag)\n break\n elif mode == 'nodes':\n if self.name in kconfig or LOCALHOST in kconfig:\n tags.add(tag)\n break\n elif mode == 'constraint':\n tags.add(tag)\n break\n if tag in tags:\n break\n return tags", "response": "Retrieve tags associated to the current node"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef activated(self, value):\n\n if value is not None:\n assert type(value) is bool, \"'{0}' attribute: '{1}' type is not 'bool'!\".format(\"activated\", value)\n self.__activated = value", "response": "Sets the activated flag."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nsetting the value of the isSetted attribute.", "response": "def initialized(self, value):\n \"\"\"\n Setter for **self.__initialized** attribute.\n\n :param value: Attribute value.\n :type value: bool\n \"\"\"\n\n if value is not None:\n assert type(value) is bool, \"'{0}' attribute: '{1}' type is not 'bool'!\".format(\"initialized\", value)\n self.__initialized = value"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef activate(self):\n\n raise NotImplementedError(\"{0} | '{1}' must be implemented by '{2}' subclasses!\".format(\n self.__class__.__name__, self.activate.__name__, self.__class__.__name__))", "response": "Sets Component activation state."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef reservations(self):\n command = [SINFO, '--reservation']\n output = subprocess.check_output(command, env=SINFO_ENV)\n output = output.decode()\n it = iter(output.splitlines())\n next(it)\n for line in it:\n rsv = Reservation.from_sinfo(line)\n yield rsv.name, rsv", "response": "get nodes of every reservations"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef set(self, values):\n if hasattr(self, \"inputs\"):\n for item in self.inputs:\n if hasattr(self, item):\n setattr(self, item, values[item])", "response": "Set the object parameters using a dictionary"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nexecuting the plan. Args: plan (:obj:`list` of :obj:`actions.Step`): The plan we want to execute. Returns: (:obj:`list` of :obj:`actions.Step`): A list of failed actions.", "response": "def execute_plan(plan):\n \"\"\"\"Execute the plan.\n\n Args:\n plan (:obj:`list` of :obj:`actions.Step`): The plan we want to execute.\n\n Returns:\n (:obj:`list` of :obj:`actions.Step`): A list of failed actions.\n \"\"\"\n results = [action() for action in plan]\n return [result for result in results if actns.step_has_failed(result)]"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef _set_log_format(color, include_caller):\n level_name = '* %(levelname)1s'\n time = '%(asctime)s,%(msecs)03d'\n message = '%(message)s'\n color_start = '%(color)s'\n color_end = '%(end_color)s'\n caller = '[%(module)s:%(lineno)d]'\n\n if color:\n if include_caller:\n return '{}{}{} {} {} {}'.format(color_start, level_name, color_end, time, message, caller)\n else:\n return '{}{}{} {} {}'.format(color_start, level_name, color_end, time, message)\n else:\n if include_caller:\n return '{} {} {} {}'.format(level_name, time, message, caller)\n else:\n return '{} {} {}'.format(level_name, time, message)", "response": "Set log format\n :param color: Log message is colored\n :param include_caller: At the end, put a [caller:line-of-code], e.g. [script:123]\n :return: string of log format"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef format_repr(obj, attributes) -> str:\n\n attribute_repr = ', '.join(('{}={}'.format(attr, repr(getattr(obj, attr)))\n for attr in attributes))\n return \"{0}({1})\".format(obj.__class__.__qualname__, attribute_repr)", "response": "Format an object s repr method with specific attributes."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nbuild a list of the decription of all the inputs of an object that have user defined parameters.", "response": "def build_parameter_descriptions(obj, user_p=None, output=\"csv\", show_none=True, ignore=None, plist=None):\n \"\"\"\n Creates a list of the decription of all the inputs of an object\n\n :param obj: object, that has parameters\n :param user_p: dict, user defined parameter descriptions\n :param show_none: if false, only shows descriptions of parameters that are not None\n :param ignore: list of parameters to not build\n :return:\n \"\"\"\n if user_p is None:\n user_p = {}\n if ignore is None:\n ignore = []\n para = [[obj.__class__.__name__ + \" inputs:\", \"\", \"\"]]\n if plist is None:\n if not hasattr(obj, 'inputs'):\n raise exceptions.ModelError(\"Object must contain parameter 'inputs' or set plist as an input\")\n plist = obj.inputs\n\n p_dict = {}\n if hasattr(obj, 'ancestor_types'):\n bt = obj.ancestor_types\n for otype in bt: # cycles from lowest class, so descriptions get overridden\n if otype in vp:\n for item in vp[otype]:\n p_dict[item] = vp[otype][item]\n for item in user_p: # user defined definitions override base ones\n p_dict[item] = user_p[item]\n else:\n p_dict = user_p\n\n for item in plist:\n if show_none is False and getattr(obj, item) is None:\n continue\n if item in ignore:\n continue\n if item in p_dict:\n para.append([item, p_dict[item][1], p_dict[item][0]])\n else:\n para.append([item, \"\", \"\"])\n if output == \"csv\":\n out_obj = []\n for i in range(len(para)):\n out_obj.append(\",\".join(para[i]))\n elif output == \"list\":\n out_obj = para\n elif output == \"dict\":\n out_obj = OrderedDict()\n for i in range(len(para)):\n out_obj[para[i][0]] = {\"description\": para[i][2], \"units\": para[i][1]}\n else:\n raise ValueError(\"output must be either: 'csv', 'dict' or 'list'.\")\n return out_obj"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef all_descriptions():\n para = []\n para += build_parameter_descriptions(models.Soil()) + [\",,\\n\"]\n para += build_parameter_descriptions(models.SoilProfile()) + [\",,\\n\"]\n para += build_parameter_descriptions(models.Foundation()) + [\",,\\n\"]\n para += build_parameter_descriptions(models.PadFoundation()) + [\",,\\n\"]\n para += build_parameter_descriptions(models.SDOFBuilding()) + [\",,\\n\"]\n para += build_parameter_descriptions(models.FrameBuilding2D(1, 1))\n\n return para", "response": "Generates a list of descriptions of all the models\n "} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nplotting shapes from table or data frame where each row corresponds to a vertex of .", "response": "def plot_shapes(df_shapes, shape_i_columns, axis=None, autoxlim=True,\n autoylim=True, **kwargs):\n '''\n Plot shapes from table/data-frame where each row corresponds to a vertex of\n a shape. Shape vertices are grouped by `shape_i_columns`.\n\n For example, consider the following dataframe:\n\n shape_i vertex_i x y\n 0 0 0 81.679949 264.69306\n 1 0 1 81.679949 286.51788\n 2 0 2 102.87004 286.51788\n 3 0 3 102.87004 264.69306\n 4 1 0 103.11417 264.40011\n 5 1 1 103.11417 242.72177\n 6 1 2 81.435824 242.72177\n 7 1 3 81.435824 264.40011\n 8 2 0 124.84134 264.69306\n 9 2 1 103.65125 264.69306\n 10 2 2 103.65125 286.37141\n 11 2 3 124.84134 286.37141\n\n This dataframe corresponds to three shapes, with (ordered) shape vertices\n grouped by `shape_i`. Note that the column `vertex_i` is not required.\n '''\n if axis is None:\n fig, axis = plt.subplots()\n props = itertools.cycle(mpl.rcParams['axes.prop_cycle'])\n color = kwargs.pop('fc', None)\n\n # Cycle through default colors to set face color, unless face color was set\n # explicitly.\n patches = [Polygon(df_shape_i[['x', 'y']].values, fc=props.next()['color']\n if color is None else color, **kwargs)\n for shape_i, df_shape_i in df_shapes.groupby(shape_i_columns)]\n\n collection = PatchCollection(patches)\n\n axis.add_collection(collection)\n\n xy_stats = df_shapes[['x', 'y']].describe()\n if autoxlim:\n axis.set_xlim(*xy_stats.x.loc[['min', 'max']])\n if autoylim:\n axis.set_ylim(*xy_stats.y.loc[['min', 'max']])\n return axis"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nplots the polygon shapes colored based on values mapped onto the color map.", "response": "def plot_shapes_heat_map(df_shapes, shape_i_columns, values, axis=None,\n vmin=None, vmax=None, value_formatter=None,\n color_map=None):\n '''\n Plot polygon shapes, colored based on values mapped onto\n a colormap.\n\n Args\n ----\n\n df_shapes (pandas.DataFrame) : Polygon table containing\n the columns `'id', 'x', 'y'`. Coordinates must be\n ordered to be grouped by `'id'`.\n values (pandas.Series) : Numeric values indexed by `'id'`.\n These values are used to look up color in the color map.\n axis : A matplotlib axis. If `None`, an axis is created.\n vmin : Minimum value to clip values at.\n vmax : Maximum value to clip values at.\n color_map : A matplotlib color map (see `matplotlib.cm`).\n\n Returns\n -------\n\n (tuple) : First element is heat map axis, second element\n is colorbar axis.\n '''\n df_shapes = df_shapes.copy()\n\n # Matplotlib draws from bottom to top, so invert `y` coordinates.\n df_shapes.loc[:, 'y'] = df_shapes.y.max() - df_shapes.y.copy().values\n\n aspect_ratio = ((df_shapes.x.max() - df_shapes.x.min()) /\n (df_shapes.y.max() - df_shapes.y.min()))\n\n if vmin is not None or vmax is not None:\n norm = mpl.colors.Normalize(vmin=vmin or min(values),\n vmax=vmax or max(values))\n else:\n norm = None\n\n if axis is None:\n fig, axis = plt.subplots(figsize=(10, 10 * aspect_ratio))\n else:\n fig = axis.get_figure()\n\n patches = OrderedDict([(id, Polygon(df_shape_i[['x', 'y']].values))\n for id, df_shape_i in\n df_shapes.groupby(shape_i_columns)])\n patches = pd.Series(patches)\n\n collection = PatchCollection(patches.values, cmap=color_map,\n norm=norm)\n collection.set_array(values.ix[patches.index])\n\n axis.add_collection(collection)\n\n axis_divider = make_axes_locatable(axis)\n\n # Append color axis to the right of `axis`, with 10% width of `axis`.\n color_axis = axis_divider.append_axes(\"right\", size=\"10%\", pad=0.05)\n\n colorbar = fig.colorbar(collection, format=value_formatter,\n cax=color_axis)\n\n tick_labels = colorbar.ax.get_yticklabels()\n if vmin is not None:\n tick_labels[0] = '$\\leq$%s' % tick_labels[0].get_text()\n if vmax is not None:\n tick_labels[-1] = '$\\geq$%s' % tick_labels[-1].get_text()\n colorbar.ax.set_yticklabels(tick_labels)\n\n axis.set_xlim(df_shapes.x.min(), df_shapes.x.max())\n axis.set_ylim(df_shapes.y.min(), df_shapes.y.max())\n return axis, colorbar"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef plot_color_map_bars(values, vmin=None, vmax=None, color_map=None,\n axis=None, **kwargs):\n '''\n Plot bar for each value in `values`, colored based on values mapped onto\n the specified color map.\n\n Args\n ----\n\n values (pandas.Series) : Numeric values to plot one bar per value.\n axis : A matplotlib axis. If `None`, an axis is created.\n vmin : Minimum value to clip values at.\n vmax : Maximum value to clip values at.\n color_map : A matplotlib color map (see `matplotlib.cm`).\n **kwargs : Extra keyword arguments to pass to `values.plot`.\n\n Returns\n -------\n\n (axis) : Bar plot axis.\n '''\n if axis is None:\n fig, axis = plt.subplots()\n\n norm = mpl.colors.Normalize(vmin=vmin or min(values),\n vmax=vmax or max(values), clip=True)\n if color_map is None:\n color_map = mpl.rcParams['image.cmap']\n colors = color_map(norm(values.values).filled())\n\n values.plot(kind='bar', ax=axis, color=colors, **kwargs)\n return axis", "response": "Plots a series of numeric values on a matplotlib axis colored based on values mapped onto the specified color map."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef find_in_paths(name, paths):\n for path in paths:\n file_ = osp.join(path, name)\n if osp.exists(file_) and not osp.isdir(file_):\n abs_file = osp.abspath(file_)\n if os.access(file_, os.X_OK):\n return abs_file", "response": "Find an executable is a list of directories\n "} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef find_executable(name, names=None, required=True):\n path_from_env = os.environ.get(name.upper())\n if path_from_env is not None:\n return path_from_env\n names = [name] + (names or [])\n for _name in names:\n if osp.isabs(_name):\n return _name\n paths = os.environ.get('PATH', '').split(os.pathsep)\n eax = find_in_paths(_name, paths)\n if eax:\n return eax\n if required:\n raise NameError('Could not find %s executable' % name)\n else:\n return name", "response": "Utility function to find an executable in PATH"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\ngetting cpus identifiers for instance set([ 0 1 2 3 )", "response": "def physical_cpus():\n \"\"\"Get cpus identifiers, for instance set([\"0\", \"1\", \"2\", \"3\"])\n\n :return Number of physical CPUs available\n :rtype: int\n \"\"\"\n if platform.system() == 'Darwin':\n ncores = subprocess.check_output(\n ['/usr/sbin/sysctl', '-n', 'hw.ncpu'], shell=False\n )\n return int(ncores.strip())\n\n sockets = set()\n with open('/proc/cpuinfo') as istr:\n for line in istr:\n if line.startswith('physical id'):\n sockets.add(line.split(':')[-1].strip())\n return len(sockets)"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef showLayer(self, title='', debugText=''):\n\t\timg = PIL.Image.fromarray(self.data, 'RGBA')\n\t\tif debugText!='':\n\t\t\tdraw = PIL.ImageDraw.Draw(img)\n\t\t\tfont = PIL.ImageFont.truetype(\"DejaVuSansMono.ttf\", 24)\n\t\t\tdraw.text((0, 0),debugText,(255,255,255),font=font)\n\t\timg.show(title=title)", "response": "Shows the single layer."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef make_converters(data_types) -> dict:\n\n return {k: TYPE_CASTERS.get(v, v) for k, v in data_types.items()}", "response": "Returns a mapping between data type names and casting functions and\n or class definitions to convert text into its Python object."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef parse_broken_json(json_text: str) -> dict:\n\n # Add spacing between Key and Value to prevent parsing error\n json_text = json_text.replace(\":\", \": \")\n json_dict = yaml.load(json_text)\n\n return json_dict", "response": "Parses broken JSON that the standard Python JSON module cannot parse."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\ncreates shell - script in charge of executing the benchmark and return its path.", "response": "def _executor_script(self):\n \"\"\"Create shell-script in charge of executing the benchmark\n and return its path.\n \"\"\"\n fd, path = tempfile.mkstemp(suffix='.sh', dir=os.getcwd())\n os.close(fd)\n with open(path, 'w') as ostr:\n self._write_executor_script(ostr)\n mode = os.stat(path).st_mode\n os.chmod(path, mode | stat.S_IEXEC | stat.S_IRGRP | stat.S_IRUSR)\n return path"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nreturns the Jinja template of the shell - script executor", "response": "def _jinja_executor_template(self):\n \"\"\":return Jinja template of the shell-script executor\"\"\"\n template = self.campaign.process.executor_template\n if template.startswith('#!'):\n return jinja_environment.from_string(template)\n else:\n return jinja_environment.get_template(template)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef _write_executor_script(self, ostr):\n environment = self.execution.get('environment') or {}\n if not isinstance(environment, Mapping):\n msg = 'Expected mapping for environment but got '\n msg += str(type(environment))\n raise Exception(msg)\n escaped_environment = dict(\n (var, six.moves.shlex_quote(str(value)))\n for var, value in environment.items()\n )\n modules = self.execution.get('modules') or []\n properties = dict(\n command=self.command,\n cwd=os.getcwd(),\n modules=modules,\n environment=escaped_environment,\n )\n self._jinja_executor_template.stream(**properties).dump(ostr)", "response": "Write the shell script in charge of executing the command"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef command(self):\n exec_prefix = self.parent.parent.parent.config.get('exec_prefix', [])\n command = self.execution['command']\n if isinstance(command, SEQUENCES):\n command = [arg.format(**self.command_expansion_vars) for arg in command]\n else:\n command = command.format(**self.command_expansion_vars)\n if self.execution.get('shell', False):\n if not isinstance(exec_prefix, six.string_types):\n exec_prefix = ' '.join(exec_prefix)\n if not isinstance(command, six.string_types):\n msg = \"Expected string for shell command, not {type}: {value}\"\n msg = msg.format(type=type(command).__name__, value=repr(command))\n raise Exception(msg)\n eax = [exec_prefix + command]\n else:\n if not isinstance(exec_prefix, SEQUENCES):\n exec_prefix = shlex.split(exec_prefix)\n if not isinstance(command, SEQUENCES):\n eax = exec_prefix + shlex.split(command)\n else:\n eax = exec_prefix + command\n eax = [six.moves.shlex_quote(arg) for arg in eax]\n return eax", "response": "return command to execute inside the generated shell - script"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef command_str(self):\n if isinstance(self.command, six.string_types):\n return self.command\n return ' '.join(map(six.moves.shlex_quote, self.command))", "response": "get command to execute properly escaped\n\n "} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nbuilding popen object to run", "response": "def popen(self, stdout, stderr):\n \"\"\"Build popen object to run\n\n :rtype: subprocess.Popen\n \"\"\"\n self.logger.info('Executing command: %s', self.command_str)\n return subprocess.Popen([self._executor_script], stdout=stdout, stderr=stderr)"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef srun_cartesian_product(cls, campaign, config):\n options = copy.copy(cls.common_srun_options(campaign))\n options.update(config.get('srun') or {})\n options_tuple = []\n for k, v in options.items():\n if not isinstance(v, SEQUENCES):\n v = [v]\n options_tuple.append([(k, e) for e in v])\n return [dict(combination) for combination in itertools.product(*options_tuple)]", "response": "returns a list of dicts with the srun options for a given campaign and configuration"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\ngets path to srun executable", "response": "def srun(self):\n \"\"\"Get path to srun executable\n\n :rtype: string\n \"\"\"\n commands = self.campaign.process.get('commands', {})\n srun = find_executable(commands.get('srun', 'srun'))\n if six.PY2:\n srun = srun.encode('utf-8')\n return srun"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef common_srun_options(cls, campaign):\n default = dict(campaign.process.get('srun') or {})\n default.update(output='slurm-%N-%t.stdout', error='slurm-%N-%t.error')\n return default", "response": "Get options to be given to all srun commands"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef command(self):\n srun_optlist = build_slurm_arguments(self.parent.command.srun or {})\n if not isinstance(self.root.network.nodes(self.tag), ConstraintTag):\n pargs = parse_constraint_in_args(srun_optlist)\n self.command_expansion_vars['process_count'] = pargs.ntasks\n if not pargs.constraint:\n # Expand nodelist if --constraint option is not specified\n # in srun options\n srun_optlist += [\n '--nodelist=' + ','.join(self.srun_nodes),\n '--nodes=' + str(len(self.srun_nodes)),\n ]\n command = super(SrunExecutionDriver, self).command\n return [self.srun] + srun_optlist + command", "response": "get command to execute\nMimeType"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\ngets list of nodes where to execute the command", "response": "def srun_nodes(self):\n \"\"\"Get list of nodes where to execute the command\n \"\"\"\n count = self.execution.get('srun_nodes', 0)\n if isinstance(count, six.string_types):\n tag = count\n count = 0\n elif isinstance(count, SEQUENCES):\n return count\n else:\n assert isinstance(count, int)\n tag = self.tag\n nodes = self._srun_nodes(tag, count)\n if 'srun_nodes' in self.execution:\n self.execution['srun_nodes'] = nodes\n self.execution['srun_nodes_count'] = len(nodes)\n return nodes"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\ncleaning up a graph by removing it", "response": "def clear_graph(identifier=None):\n \"\"\" Clean up a graph by removing it\n\n :param identifier: Root identifier of the graph\n :return:\n \"\"\"\n graph = get_graph()\n if identifier:\n graph.destroy(identifier)\n try:\n graph.close()\n except:\n warn(\"Unable to close the Graph\")"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef generate_alchemy_graph(alchemy_uri, prefixes=None, identifier=\"NautilusSparql\"):\n registerplugins()\n ident = URIRef(identifier)\n uri = Literal(alchemy_uri)\n store = plugin.get(\"SQLAlchemy\", Store)(identifier=ident)\n graph = Graph(store, identifier=ident)\n graph.open(uri, create=True)\n\n for prefix, ns in (prefixes or GRAPH_BINDINGS).items():\n if prefix == \"\":\n prefix = \"cts\" # Fix until ALchemy Store accepts empty prefixes\n graph.bind(prefix, ns)\n\n return graph, identifier, uri", "response": "Generate a graph and change the global graph to this one"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\ngenerating a Graph object and bind it to the one we have", "response": "def generate_sleepy_cat_graph(filepath, prefixes=None, identifier=\"NautilusSparql\"):\n \"\"\" Generate a graph and change the global graph to this one\n\n :param filepath: A Uri for the graph\n :param prefixes: A dictionary of prefixes and namespaces to bind to the graph\n :param identifier: An identifier that will identify the Graph root\n \"\"\"\n registerplugins()\n ident = URIRef(identifier)\n graph = Graph('Sleepycat', identifier=ident)\n graph.open(filepath, create=True)\n\n for prefix, ns in (prefixes or GRAPH_BINDINGS).items():\n if prefix == \"\":\n prefix = \"cts\" # Fix until ALchemy Store accepts empty prefixes\n graph.bind(prefix, ns)\n\n return graph, identifier, filepath"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef set_label(self, label, lang):\n try:\n self.metadata.add(SKOS.prefLabel, Literal(label, lang=lang))\n self.graph.addN([\n (self.asNode(), RDFS.label, Literal(label, lang=lang), self.graph),\n ])\n except Exception as E:\n pass", "response": "Add the label of the collection in given lang"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef members(self):\n return list(\n [\n self.children_class(child)\n for child in self.graph.subjects(RDF_NAMESPACES.DTS.parent, self.asNode())\n ]\n )", "response": "Returns a list of all the items in the collection."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef parent(self):\n parent = list(self.graph.objects(self.asNode(), RDF_NAMESPACES.DTS.parent))\n if parent:\n return self.parent_class(parent[0])\n return None", "response": "Returns the parent of the current object"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\ncompile & Run the experiment with - O3 enabled.", "response": "def actions_for_project(self, project):\n \"\"\"Compile & Run the experiment with -O3 enabled.\"\"\"\n project.cflags = [\"-O3\", \"-fno-omit-frame-pointer\"]\n project.runtime_extension = time.RunWithTime(\n run.RuntimeExtension(project, self))\n return self.default_runtime_actions(project)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nconverting a function return to a list of StepResults.", "response": "def to_step_result(func):\n \"\"\"Convert a function return to a list of StepResults.\n\n All Step subclasses automatically wrap the result of their\n __call__ method's result with this wrapper.\n If the result is not a list of StepResult values, one will\n be generated.\n\n result of `[StepResult.OK]`, or convert the given result into\n a list.\n\n Args:\n func: The function to wrap.\n \"\"\"\n\n @ft.wraps(func)\n def wrapper(*args, **kwargs):\n \"\"\"Wrapper stub.\"\"\"\n res = func(*args, **kwargs)\n if not res:\n res = [StepResult.OK]\n\n if not hasattr(res, \"__iter__\"):\n res = [res]\n return res\n\n return wrapper"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nprint the beginning and the end of a `func`.", "response": "def notify_step_begin_end(func):\n \"\"\"Print the beginning and the end of a `func`.\"\"\"\n\n @ft.wraps(func)\n def wrapper(self, *args, **kwargs):\n \"\"\"Wrapper stub.\"\"\"\n cls = self.__class__\n on_step_begin = cls.ON_STEP_BEGIN\n on_step_end = cls.ON_STEP_END\n\n for begin_listener in on_step_begin:\n begin_listener(self)\n\n res = func(self, *args, **kwargs)\n\n for end_listener in on_step_end:\n end_listener(self, func)\n return res\n\n return wrapper"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nlogging customized stirng before & after running func.", "response": "def log_before_after(name: str, desc: str):\n \"\"\"Log customized stirng before & after running func.\"\"\"\n\n def func_decorator(f):\n \"\"\"Wrapper stub.\"\"\"\n\n @ft.wraps(f)\n def wrapper(*args, **kwargs):\n \"\"\"Wrapper stub.\"\"\"\n LOG.info(\"\\n%s - %s\", name, desc)\n res = f(*args, **kwargs)\n if StepResult.ERROR not in res:\n LOG.info(\"%s - OK\\n\", name)\n else:\n LOG.error(\"%s - ERROR\\n\", name)\n return res\n\n return wrapper\n\n return func_decorator"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef quickview(filename,\n noIFFT=False):\n '''\n Display processed MRI data from `.hdf5`, `.npz`, or `.dat` files. No arguments displays the IFFT of the k-space data. The type of file is guessed by the file extension (i.e., if extension is `.dat` then readMeasData15 will be run to get the data).\n\n Command-line Options:\n -nifft (no IFFT)\n Display k-space data, log magnitude and phase plots.\n\n '''\n\n if filename.endswith('.npz'):\n data = np.load(filename)\n elif filename.endswith('.dat'):\n from rawdatarinator.readMeasDataVB15 import readMeasDataVB15 as rmd\n data = rmd(filename)\n elif filename.endswith('.mat'):\n import scipy.io\n data = scipy.io.loadmat('file.mat')\n else:\n data = h5py.File(filename,'r')\n\n if 'kSpace' in data:\n key = 'kSpace'\n else:\n key = 'imSpace'\n noIFFT = not noIFFT\n\n # Average over all the averages, use first coil\n coil = 0\n num_avgs = data[key].shape[2]\n avg = (np.squeeze(np.sum(data[key],axis=2))/num_avgs)[:,:,coil]\n\n if noIFFT is False:\n imData = np.fft.ifftshift(np.fft.ifft2(avg))\n plt.imshow(np.absolute(imData),cmap='gray')\n plt.title('Image Data')\n else:\n mag = np.log(np.absolute(avg))\n phase = np.angle(avg)\n f,(ax1,ax2) = plt.subplots(1,2,sharey=True)\n ax1.imshow(mag,cmap='gray')\n ax2.imshow(phase,cmap='gray')\n ax1.set_title('log(Magnitude)')\n ax2.set_title('Phase')\n\n plt.show()", "response": "Quick view of the MRI data from. hdf5. npz or. dat files."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nreturns the Greatest Common Divisor of a and b", "response": "def euclid(a, b):\n \"\"\"returns the Greatest Common Divisor of a and b\"\"\"\n a = abs(a)\n b = abs(b)\n if a < b:\n a, b = b, a\n while b != 0:\n a, b = b, a % b\n return a"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nreturn True if all the values in the list L are all co - prime otherwise returns False.", "response": "def coPrime(l):\n \"\"\"returns 'True' if the values in the list L are all co-prime\n otherwise, it returns 'False'. \"\"\"\n for i, j in combinations(l, 2):\n if euclid(i, j) != 1:\n return False\n return True"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef extendedEuclid(a, b):\n if a == 0:\n return b, 0, 1\n else:\n g, y, x = extendedEuclid(b % a, a)\n return g, x - (b // a) * y, y", "response": "extendedEuclid returns a tuple of three values x y z such that x is\n the GCD of a and b is extendedEuclid the GCD of b and x is extendedEuclid the GCD of a and b."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nreturns the multiplicative inverse of a in modulo m as a positive value between zero and m - 1", "response": "def modInv(a, m):\n \"\"\"returns the multiplicative inverse of a in modulo m as a\n positive value between zero and m-1\"\"\"\n # notice that a and m need to co-prime to each other.\n if coPrime([a, m]):\n linearCombination = extendedEuclid(a, m)\n return linearCombination[1] % m\n else:\n return 0"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef extractTwos(m):\n # the problem can be break down to count how many '0's are there in\n # the end of bin(m). This can be done this way: m & a stretch of '1's\n # which can be represent as (2 ** n) - 1.\n assert m >= 0\n i = 0\n while m & (2 ** i) == 0:\n i += 1\n return i, m >> i", "response": "m is a positive integer. A tuple s of integers is returned\n such that m = 2^n and d = 2^n."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef int2baseTwo(x):\n # repeating x >>= 1 and x & 1 will do the trick\n assert x >= 0\n bitInverse = []\n while x != 0:\n bitInverse.append(x & 1)\n x >>= 1\n return bitInverse", "response": "Convert an integer x to base two."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef modExp(a, d, n):\n assert d >= 0\n assert n >= 0\n base2D = int2baseTwo(d)\n base2DLength = len(base2D)\n modArray = []\n result = 1\n for i in range(1, base2DLength + 1):\n if i == 1:\n modArray.append(a % n)\n else:\n modArray.append((modArray[i - 2] ** 2) % n)\n for i in range(0, base2DLength):\n if base2D[i] == 1:\n result *= base2D[i] * modArray[i]\n return result % n", "response": "returns a ** d n"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef millerRabin(n, k):\n assert n >= 1\n # ensure n is bigger than 1\n assert k > 0\n # ensure k is a positive integer so everything down here makes sense\n\n if n == 2:\n return True\n # make sure to return True if n == 2\n\n if n % 2 == 0:\n return False\n # immediately return False for all the even numbers bigger than 2\n\n extract2 = extractTwos(n - 1)\n s = extract2[0]\n d = extract2[1]\n assert 2 ** s * d == n - 1\n\n def tryComposite(a):\n \"\"\"Inner function which will inspect whether a given witness\n will reveal the true identity of n. Will only be called within\n millerRabin\"\"\"\n x = modExp(a, d, n)\n if x == 1 or x == n - 1:\n return None\n else:\n for j in range(1, s):\n x = modExp(x, 2, n)\n if x == 1:\n return False\n elif x == n - 1:\n return None\n return False\n\n for i in range(0, k):\n a = random.randint(2, n - 2)\n if tryComposite(a) == False:\n return False\n return True", "response": "Returns True if n is likely a pseudo - prime and False otherwise."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nreturns a list with length k + 1 showing the prime of the archive", "response": "def primeSieve(k):\n \"\"\"return a list with length k + 1, showing if list[i] == 1, i is a prime\n else if list[i] == 0, i is a composite, if list[i] == -1, not defined\"\"\"\n\n def isPrime(n):\n \"\"\"return True is given number n is absolutely prime,\n return False is otherwise.\"\"\"\n for i in range(2, int(n ** 0.5) + 1):\n if n % i == 0:\n return False\n return True\n result = [-1] * (k + 1)\n for i in range(2, int(k + 1)):\n if isPrime(i):\n result[i] = 1\n else:\n result[i] = 0\n return result"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nfinds a pseudo prime number roughly between a and b. Raise ValueError if cannot find a pseudo prime number.", "response": "def findAPrime(a, b, k):\n \"\"\"Return a pseudo prime number roughly between a and b,\n (could be larger than b). Raise ValueError if cannot find a\n pseudo prime after 10 * ln(x) + 3 tries. \"\"\"\n x = random.randint(a, b)\n for i in range(0, int(10 * math.log(x) + 3)):\n if millerRabin(x, k):\n return x\n else:\n x += 1\n raise ValueError"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef newKey(a, b, k):\n try:\n p = findAPrime(a, b, k)\n while True:\n q = findAPrime(a, b, k)\n if q != p:\n break\n except:\n raise ValueError\n\n n = p * q\n m = (p - 1) * (q - 1)\n\n while True:\n e = random.randint(1, m)\n if coPrime([e, m]):\n break\n\n d = modInv(e, m)\n return (n, e, d)", "response": "Generate public and private keys for RSA encryption."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef numList2blocks(l, n):\n # Note that ASCII printable characters range is 0x20 - 0x7E\n returnList = []\n toProcess = copy.copy(l)\n ''' copy message ascii list'''\n if len(toProcess) % n != 0:\n for i in range(0, n - len(toProcess) % n):\n ''' append rand str to list'''\n toProcess.append(random.randint(32, 126))\n toProcess[len(l)] = 0 # 0 after origin message list\n\n ''' generate int from ascii number list'''\n for i in range(0, len(toProcess), n):\n block = 0\n for j in range(0, n):\n block += toProcess[i + j] << (8 * (n - j - 1))\n returnList.append(block)\n return returnList", "response": "Take a list of integers each between 0 and 127 and generates them\n into a list of blocks of size n using base 256."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef blocks2numList(blocks, n):\n toProcess = copy.copy(blocks)\n returnList = []\n for numBlock in toProcess:\n inner = []\n for i in range(0, n):\n inner.append(numBlock % 256)\n numBlock >>= 8\n inner.reverse()\n returnList.extend(inner)\n return returnList", "response": "inverse function of numList2blocks."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\ngiving a string message public keys and blockSize encrypt using RSA algorithms.", "response": "def encrypt(message, modN, e, blockSize):\n \"\"\"given a string message, public keys and blockSize, encrypt using\n RSA algorithms.\"\"\"\n numList = string2numList(message)\n numBlocks = numList2blocks(numList, blockSize) # only one block\n message = numBlocks[0]\n # return [modExp(blocks, e, modN) for blocks in numBlocks]\n return modExp(message, e, modN)"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nreversing function of encrypt", "response": "def decrypt(secret, modN, d, blockSize):\n \"\"\"reverse function of encrypt\"\"\"\n numBlocks = [modExp(blocks, d, modN) for blocks in secret]\n numList = blocks2numList(numBlocks, blockSize)\n return numList2string(numList)"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef match_files(files, pattern: Pattern):\n\n for name in files:\n if re.match(pattern, name):\n yield name", "response": "Yields file name if matches a regular expression pattern."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef match_zipfile_members(zipfile_path: str, pattern: Pattern):\n\n with ZipFile(zipfile_path, mode='r') as zfile:\n members = zfile.namelist()\n\n yield from match_files(members, pattern)", "response": "Match files to a pattern within a zip file s content."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef directory_files(path):\n\n for entry in os.scandir(path):\n if not entry.name.startswith('.') and entry.is_file():\n yield entry.name", "response": "Yields the names of all files in a directory."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nreturn sha256 string for group of FTP listings.", "response": "def get_file_listing_sha(listing_paths: Iterable) -> str:\n \"\"\"Return sha256 string for group of FTP listings.\"\"\"\n\n return sha256(''.join(sorted(listing_paths)).encode('utf-8')).hexdigest()"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nsets the finger number to button number mapping for tap - to - click.", "response": "def set_button_map(self, button_map):\n\t\t\"\"\"Set the finger number to button number mapping for tap-to-click.\n\n\t\tThe default mapping on most devices is to have a 1, 2 and 3 finger tap\n\t\tto map to the left, right and middle button, respectively. A device may\n\t\tpermit changing the button mapping but disallow specific maps. In this\n\t\tcase :attr:`~libinput.constant.ConfigStatus.UNSUPPORTED` is returned,\n\t\tthe caller is expected to handle this case correctly.\n\n\t\tChanging the button mapping may not take effect immediately, the device\n\t\tmay wait until it is in a neutral state before applying any changes.\n\n\t\tThe mapping may be changed when tap-to-click is disabled. The new\n\t\tmapping takes effect when tap-to-click is enabled in the future.\n\n\t\tIf :attr:`finger_count` is 0, this method raises :exc:`AssertionError`.\n\n\t\tArgs:\n\t\t\tbutton_map (~libinput.constant.TapButtonMap): The new\n\t\t\t\tfinger-to-button number mapping.\n\t\tReturns:\n\t\t\t~libinput.constant.ConfigStatus: A config status code.\n\t\tRaises:\n\t\t\tAssertionError\n\t\t\"\"\"\n\n\t\tassert self.finger_count > 0, 'This device does not support tapping'\n\t\treturn self._libinput.libinput_device_config_tap_set_button_map(\n\t\t\tself._handle, button_map)"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef set_matrix(self, matrix):\n\n\t\tmatrix = (c_float * 6)(*matrix)\n\t\treturn self._libinput.libinput_device_config_calibration_set_matrix(\n\t\t\tself._handle, matrix)", "response": "This function applies the 3x3 transformation matrix to absolute device coordinates."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef matrix(self):\n\n\t\tmatrix = (c_float * 6)()\n\t\trc = self._libinput.libinput_device_config_calibration_get_matrix(\n\t\t\tself._handle, matrix)\n\t\treturn rc, tuple(matrix)", "response": "Returns the current calibration matrix for this device."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef name(self):\n\n\t\tpchar = self._libinput.libinput_device_get_name(self._handle)\n\t\treturn string_at(pchar).decode()", "response": "The descriptive device name as advertised by the kernel\n\tand or the hardware itself."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nchange the logical seat name associated with this device.", "response": "def set_seat_logical_name(self, seat):\n\t\t\"\"\"Change the logical seat associated with this device by removing\n\t\tthe device and adding it to the new seat.\n\n\t\tThis command is identical to physically unplugging the device, then\n\t\tre-plugging it as a member of the new seat. libinput will generate\n\t\ta :attr:`~libinput.constant.EventType.DEVICE_REMOVED` event and this\n\t\t:class:`Device` is considered removed from the context; it will not\n\t\tgenerate further events.\n\t\tA :attr:`~libinput.constant.EventType.DEVICE_ADDED` event is\n\t\tgenerated with a new :class:`Device`. It is the caller's\n\t\tresponsibility to update references to the new device accordingly.\n\n\t\tIf the logical seat name already exists in the device's physical seat,\n\t\tthe device is added to this seat. Otherwise, a new seat is created.\n\n\t\tNote:\n\t\t\tThis change applies to this device until removal or\n\t\t\t:meth:`~libinput.LibInput.suspend`, whichever happens earlier.\n\t\tArgs:\n\t\t\tseat (str): The new logical seat name.\n\t\tRaises:\n\t\t\tAssertionError\n\t\t\"\"\"\n\n\t\trc = self._libinput.libinput_device_set_seat_logical_name(\n\t\t\tself._handle, seat.encode())\n\t\tassert rc == 0, 'Cannot assign device to {}'.format(seat)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nchecking if this device has a given button.", "response": "def has_button(self, button):\n\t\t\"\"\"Check if this device has a given button.\n\n\t\tArgs:\n\t\t\tbutton (int): Button to check for, see ``input.h`` for button\n\t\t\t\tdefinitions.\n\t\tReturns:\n\t\t\tbool: :obj:`True` if the device has this button, :obj:`False` if\n\t\t\tit does not.\n\t\tRaises:\n\t\t\tAssertionError\n\t\t\"\"\"\n\n\t\trc = self._libinput.libinput_device_pointer_has_button(\n\t\t\tself._handle, button)\n\t\tassert rc >= 0, 'This device is not a pointer device'\n\t\treturn bool(rc)"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nchecking if a keyboard device has a given key.", "response": "def has_key(self, key):\n\t\t\"\"\"Check if a :attr:`~libinput.constant.DeviceCapability.KEYBOARD`\n\t\tdevice has a given key.\n\n\t\tArgs:\n\t\t\tkey (int): Key to check for, see ``input.h`` for key definitions.\n\t\tReturns:\n\t\t\tbool: :obj:`True` if the device has this key, :obj:`False` if\n\t\t\tit does not.\n\t\tRaises:\n\t\t\tAssertionError\n\t\t\"\"\"\n\n\t\trc = self._libinput.libinput_device_keyboard_has_key(self._handle, key)\n\t\tassert rc >= 0, 'This device is not a keyboard device'\n\t\treturn bool(rc)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef num_rings(self):\n\n\t\tnum = self._libinput.libinput_device_tablet_pad_get_num_rings(\n\t\t\tself._handle)\n\t\tif num < 0:\n\t\t\traise AttributeError('This device is not a tablet pad device')\n\t\treturn num", "response": "The number of rings a device with\n\tthe : attr : ~libinput. constant. DeviceCapability. TABLET_PAD_HAS_RINGS."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef num_strips(self):\n\n\t\tnum = self._libinput.libinput_device_tablet_pad_get_num_strips(\n\t\t\tself._handle)\n\t\tif num < 0:\n\t\t\traise AttributeError('This device is not a tablet pad device')\n\t\treturn num", "response": "The number of strips a device with\n\tthe : attr : ~libinput. DeviceCapability. TABLET_PAD_HAS_STRIPS."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef num_mode_groups(self):\n\n\t\tnum = self._libinput.libinput_device_tablet_pad_get_num_mode_groups(\n\t\t\tself._handle)\n\t\tif num < 0:\n\t\t\traise AttributeError('This device is not a tablet pad device')\n\t\treturn num", "response": "Returns the number of mode groups available on this device."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef get_mode_group(self, group):\n\n\t\thmodegroup = self._libinput.libinput_device_tablet_pad_get_mode_group(\n\t\t\tself._handle, group)\n\t\tif hmodegroup:\n\t\t\treturn TabletPadModeGroup(hmodegroup, self._libinput)\n\t\treturn None", "response": "Returns the mode group with the given index or None if the group is invalid."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nreturning the logical name of the seat.", "response": "def logical_name(self):\n\t\t\"\"\"The logical name of the seat.\n\n\t\tThis is an identifier to group sets of devices within the compositor.\n\n\t\tReturns:\n\t\t\tstr: The logical name of this seat.\n\t\t\"\"\"\n\n\t\tpchar = self._libinput.libinput_seat_get_logical_name(self._handle)\n\t\treturn string_at(pchar).decode()"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef configure():\n log_levels = {\n 5: logging.NOTSET,\n 4: logging.DEBUG,\n 3: logging.INFO,\n 2: logging.WARNING,\n 1: logging.ERROR,\n 0: logging.CRITICAL\n }\n\n logging.captureWarnings(True)\n root_logger = logging.getLogger()\n if settings.CFG[\"debug\"]:\n details_format = logging.Formatter(\n '%(name)s (%(filename)s:%(lineno)s) [%(levelname)s] %(message)s')\n details_hdl = logging.StreamHandler()\n details_hdl.setFormatter(details_format)\n root_logger.addHandler(details_hdl)\n else:\n brief_format = logging.Formatter('%(message)s')\n console_hdl = logging.StreamHandler()\n console_hdl.setFormatter(brief_format)\n root_logger.addHandler(console_hdl)\n root_logger.setLevel(log_levels[int(settings.CFG[\"verbosity\"])])\n\n configure_plumbum_log()\n configure_migrate_log()\n configure_parse_log()", "response": "Load logging configuration from our own defaults."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef get_number(self):\n '''\n .. versionchanged:: 0.9.2\n Add support for float exponent strings (e.g., ``3.435e-7``).\n\n Fixes `issue #4 `.\n '''\n number = None\n start = self.get_char('0123456789.-')\n if start:\n number = start\n finish = self.get_chars('-e0123456789.')\n if finish:\n number += finish\n if any(c in number for c in '.e'):\n return float(number)\n else:\n return int(number)", "response": "Get the number of the ISO - 8601 ISO - 8601 ISO - 8601 ISO - 8601."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef to_tuples(self, data):\n '''\n path_data : string, from an svg path tag's 'd' attribute, eg:\n 'M 46,74 L 35,12 l 53,-13 z'\n returns the same data collected in a list of tuples, eg:\n [ ('M', 46, 74), ('L', 35, 12), ('l', 53, -13), ('z') ],\n The input data may have floats instead of ints, this will be reflected\n in the output. The input may have its whitespace stripped out, or its\n commas replaced by whitespace.\n '''\n self.data = data\n self.pos = 0\n parsed = []\n command = []\n\n while self.pos < len(self.data):\n indicator = self.data[self.pos]\n if indicator == ' ':\n self.pos += 1\n elif indicator == ',':\n if len(command) >= 2:\n self.pos += 1\n else:\n msg = 'unexpected comma at %d in %r' % (self.pos, self.data)\n raise ParseError(msg)\n elif indicator in '0123456789.-':\n if command:\n command.append(self.get_number())\n else:\n msg = 'missing command at %d in %r' % (self.pos, self.data)\n raise ParseError(msg)\n else:\n if command:\n parsed.append(tuple(command))\n command = [indicator]\n self.pos += 1\n\n if command:\n parsed.append(tuple(command))\n\n if parsed[0][0] == 'M' and parsed[-1][0] == 'L'\\\n and parsed[0][1:] == parsed[-1][1:]:\n parsed[-1] = ('z',)\n return parsed", "response": "Parse the data into a list of tuples."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nconvert a list of command characters to a list of loops.", "response": "def to_loops(self, commands):\n '''\n commands : list of tuples, as output from to_tuples() method, eg:\n [('M', 1, 2), ('L', 3, 4), ('L', 5, 6), ('z')]\n Interprets the command characters at the start of each tuple to return\n a list of loops, where each loop is a closed list of verts, and each\n vert is a pair of ints or floats, eg:\n [[1, 2, 3, 4, 5, 6]]\n Note that the final point of each loop is eliminated if it is equal to\n the first.\n SVG defines commands:\n M x,y: move, start a new loop\n L x,y: line, draw boundary\n H x: move horizontal\n V y: move vertical\n Z: close current loop - join to start point\n Lower-case command letters (eg 'm') indicate a relative offset.\n See http://www.w3.org/TR/SVG11/paths.html\n '''\n lookup = {\n 'M': self.onMove,\n 'L': self.onLine,\n 'H': self.onHorizontalMove,\n 'h': self.onHorizontalMove,\n 'V': self.onVerticalMove,\n 'v': self.onVerticalMove,\n 'Z': self.onClose,\n 'z': self.onClose,\n }\n self.loops = []\n self.current_loop = None\n\n for command in commands:\n action = command[0]\n if action in lookup:\n lookup[action](command)\n else:\n self.onBadCommand(action)\n return self.loops"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nparse a color string into a tuple of integers.", "response": "def parse_color(self, color):\n '''\n color : string, eg: '#rrggbb' or 'none'\n (where rr, gg, bb are hex digits from 00 to ff)\n returns a triple of unsigned bytes, eg: (0, 128, 255)\n '''\n if color == 'none':\n return None\n return (\n int(color[1:3], 16),\n int(color[3:5], 16),\n int(color[5:7], 16))"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef parse_style(self, style):\n '''\n style : string, eg:\n fill:#ff2a2a;fill-rule:evenodd;stroke:none;stroke-width:1px;\n stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1\n returns color as a triple of unsigned bytes: (r, g, b), or None\n '''\n style_elements = style.split(';')\n while style_elements:\n element = style_elements.pop()\n if element.startswith('fill:'):\n return self.parse_color(element[5:])\n return None", "response": "Parses a style string into a color tuple."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nparse the XML tag and returns the id attribute and path attribute of the current level.", "response": "def parse(self, tag):\n '''\n returns (id, path)\n where: 'id' is the path tag's id attribute\n 'path' is a populated instance of SvgPath\n\n >>> from lxml import etree\n >>> from lxml.builder import E\n >>> path_tag = etree.XML(\"\"\"\n ... \n ... \"\"\")\n >>> path_parser = PathParser()\n >>> id, svg_path = path_parser.parse(path_tag)\n >>> id\n 'path0'\n >>> svg_path.color\n (0, 0, 255)\n >>> len(svg_path.loops)\n 1\n >>> svg_path.loops[0].verts\n [(534.07239, 261.47322), (534.07239, 269.65826), (525.933, 85), (525.93385, 261.47322)]\n\n Note that only absolute commands (i.e., uppercase) are currently supported. For example:\n paths will throw a ParseError exception. For example:\n\n >>> path_tag = E.path(id=\"path0\", d=\"M 636.0331,256.9345 l 636.0331,256.9345\")\n >>> print etree.tostring(path_tag)\n \n >>> path_parser.parse(path_tag)\n Traceback (most recent call last):\n ...\n ParseError: unsupported svg path command: l\n >>> \n '''\n id = self.get_id(tag.attrib)\n \n parser = PathDataParser()\n path_data = tag.attrib['d']\n path_tuple = parser.to_tuples(path_data)\n\n tracer = LoopTracer()\n loops = tracer.to_loops(path_tuple)\n path = ColoredPath(loops)\n\n if 'style' in list(tag.attrib.keys()):\n style_data = tag.attrib['style']\n path.color = self.parse_style(style_data)\n\n return id, path"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef _validate_response(response):\n try:\n response.raise_for_status()\n except requests.RequestException:\n raise RequestException(\n code=response.status_code,\n url=response.url,\n description=response.text)\n else:\n return response", "response": "Validate the response and raise RequestException if not ok"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef _validate_request(endpoint, file_type='json', data=None, params=None):\n if not isinstance(endpoint, string_types) or endpoint.strip() == '':\n raise ClientException(\"Must submit `endpoint` for DHIS2 API\")\n if not isinstance(file_type, string_types) or file_type.lower() not in ('json', 'csv', 'xml', 'pdf', 'xlsx'):\n raise ClientException(\"Invalid file_type: {}\".format(file_type))\n if params:\n if not isinstance(params, (dict, list)):\n raise ClientException(\"`params` must be a dict or list of tuples, not {}\".format(params.__class__.__name__))\n if isinstance(params, list) and not all([isinstance(elem, tuple) for elem in params]):\n raise ClientException(\"`params` list must all be tuples\")\n if data and not isinstance(data, dict):\n raise ClientException(\"`data` must be a dict, not {}\".format(data.__class__.__name__))", "response": "Validate request before calling API\n "} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nmake a request to DHIS2 API.", "response": "def _make_request(self, method, endpoint, **kwargs):\n \"\"\"\n Do the actual request with supplied HTTP method\n :param method: HTTP method\n :param endpoint: DHIS2 API endpoint\n :param kwargs: keyword args\n :return: response if ok, RequestException if not\n \"\"\"\n if isinstance(kwargs.get('file_type'), string_types):\n file_type = kwargs['file_type'].lower()\n else:\n file_type = 'json'\n params = kwargs.get('params')\n\n data = kwargs.get('data', kwargs.get('json', None))\n url = '{}/{}'.format(self.api_url, endpoint)\n self._validate_request(endpoint, file_type, data, params)\n\n if method == 'get':\n stream = kwargs.get('stream', False)\n url = '{}.{}'.format(url, file_type)\n r = self.session.get(url, params=params, stream=stream)\n\n elif method == 'post':\n r = self.session.post(url=url, json=data, params=params)\n\n elif method == 'put':\n r = self.session.put(url=url, json=data, params=params)\n\n elif method == 'patch':\n r = self.session.patch(url=url, json=data, params=params)\n\n elif method == 'delete':\n r = self.session.delete(url=url, params=params)\n\n else:\n raise ClientException(\"Non-supported HTTP method: {}\".format(method))\n\n return self._validate_response(r)"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef get(self, endpoint, file_type='json', params=None, stream=False):\n return self._make_request('get', endpoint, params=params, file_type=file_type, stream=stream)", "response": "Get from DHIS2 API"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef post(self, endpoint, json=None, params=None, **kwargs):\n json = kwargs['data'] if 'data' in kwargs else json\n return self._make_request('post', endpoint, data=json, params=params)", "response": "POST to DHIS2 API endpoint"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef put(self, endpoint, json=None, params=None, **kwargs):\n json = kwargs['data'] if 'data' in kwargs else json\n return self._make_request('put', endpoint, data=json, params=params)", "response": "This method is used to put a set of unique identifiers."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef patch(self, endpoint, json=None, params=None, **kwargs):\n json = kwargs['data'] if 'data' in kwargs else json\n return self._make_request('patch', endpoint, data=json, params=params)", "response": "PATCH to DHIS2 API endpoint"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef delete(self, endpoint, json=None, params=None, **kwargs):\n json = kwargs['data'] if 'data' in kwargs else json\n return self._make_request('delete', endpoint, data=json, params=params)", "response": "DELETE from DHIS2\n :param endpoint: DHIS2 API endpoint\n :param json: HTTP payload\n :param params: HTTP parameters (dict)\n :return: requests.Response object"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef get_paged(self, endpoint, params=None, page_size=50, merge=False):\n try:\n if not isinstance(page_size, (string_types, int)) or int(page_size) < 1:\n raise ValueError\n except ValueError:\n raise ClientException(\"page_size must be > 1\")\n\n params = {} if not params else params\n if 'paging' in params:\n raise ClientException(\"Can't set paging manually in `params` when using `get_paged`\")\n params['pageSize'] = page_size\n params['page'] = 1\n params['totalPages'] = True\n\n collection = endpoint.split('/')[0] # only use e.g. events when submitting events/query as endpoint\n\n def page_generator():\n \"\"\"Yield pages\"\"\"\n page = self.get(endpoint=endpoint, file_type='json', params=params).json()\n page_count = page['pager']['pageCount']\n yield page\n\n while page['pager']['page'] < page_count:\n params['page'] += 1\n page = self.get(endpoint=endpoint, file_type='json', params=params).json()\n yield page\n\n if not merge:\n return page_generator()\n else:\n data = []\n for p in page_generator():\n data.append(p[collection])\n return {collection: list(chain.from_iterable(data))}", "response": "Get all DHIS2 objects in a single page."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef get_sqlview(self, uid, execute=False, var=None, criteria=None, merge=False):\n params = {}\n sqlview_type = self.get('sqlViews/{}'.format(uid), params={'fields': 'type'}).json().get('type')\n if sqlview_type == 'QUERY':\n if not isinstance(var, dict):\n raise ClientException(\"Use a dict to submit variables: e.g. var={'key1': 'value1', 'key2': 'value2'}\")\n var = ['{}:{}'.format(k, v) for k, v in var.items()]\n params['var'] = var\n if execute:\n raise ClientException(\"SQL view of type QUERY, no view to create (no execute=True)\")\n\n else: # MATERIALIZED_VIEW / VIEW\n if criteria:\n if not isinstance(criteria, dict):\n raise ClientException(\"Use a dict to submit criteria: { 'col1': 'value1', 'col2': 'value2' }\")\n criteria = ['{}:{}'.format(k, v) for k, v in criteria.items()]\n params['criteria'] = criteria\n\n if execute: # materialize\n self.post('sqlViews/{}/execute'.format(uid))\n\n def page_generator():\n with closing(self.get('sqlViews/{}/data'.format(uid), file_type='csv', params=params, stream=True)) as r:\n # do not need to use unicodecsv.DictReader as data comes in bytes already\n reader = DictReader(codecs.iterdecode(r.iter_lines(), 'utf-8'), delimiter=',', quotechar='\"')\n for row in reader:\n yield row\n\n if not merge:\n return page_generator()\n else:\n return list(page_generator())", "response": "GET SQL View data"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef post_partitioned(self, endpoint, json, params=None, thresh=1000):\n\n if not isinstance(json, dict):\n raise ClientException('Parameter `json` must be a dict')\n if not isinstance(thresh, int) or thresh < 2:\n raise ClientException(\"`thresh` must be integer of 2 or larger\")\n\n try:\n key = next(iter(json)) # the (only) key in the payload\n except StopIteration:\n raise ClientException(\"`json` is empty\")\n else:\n if len(json.keys()) != 1:\n raise ClientException('Must submit exactly one key in payload - e.g. json={\"dataElements\": [...]\"}')\n if not json.get(key):\n raise ClientException(\"payload for key '{}' is empty\".format(key))\n else:\n for data in partition_payload(data=json, key=key, thresh=thresh):\n yield self.post(endpoint, json=data, params=params)", "response": "Post a payload in chunks to prevent Request Entity Too Large Timeout errors"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nreturns a DataFrame with the 3x3 transformation matrix resulting in specified x y and scale.", "response": "def get_transform(offset, scale):\n '''\n Parameters\n ----------\n offset : pandas.Series\n Cartesian ``(x, y)`` coordinate of offset origin.\n scale : pandas.Series\n Scaling factor for ``x`` and ``y`` dimensions.\n\n Returns\n -------\n pandas.DataFrame\n 3x3 transformation matrix resulting in specified `x/y` offset and\n scale. **Note that third row label is ``w`` and not ``z``).**\n '''\n return pd.DataFrame([[scale, 0, offset.x], [0, scale, offset.y],\n [0, 0, 1]], index=['x', 'y', 'w'])"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nlook up shape based on canvas coordinates.", "response": "def find_shape(self, canvas_x, canvas_y):\n '''\n Look up shape based on canvas coordinates.\n '''\n shape_x, shape_y, w = self.canvas_to_shapes_transform.dot([canvas_x,\n canvas_y,\n 1])\n if hasattr(self.space, 'point_query_first'):\n # Assume `pymunk<5.0`.\n shape = self.space.point_query_first((shape_x, shape_y))\n else:\n # Assume `pymunk>=5.0`, where `point_query_first` method has been\n # deprecated.\n info = self.space.point_query_nearest((shape_x, shape_y), 0,\n [pymunk.ShapeFilter\n .ALL_CATEGORIES])\n shape = info.shape if info else None\n\n if shape:\n return self.bodies[shape.body]\n return None"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nreturn a pandas. Series indexed by shape_i_columns containing the area of each shape.", "response": "def get_shape_areas(df_shapes, shape_i_columns, signed=False):\n '''\n Return a `pandas.Series` indexed by `shape_i_columns` (i.e., each entry\n corresponds to a single shape/polygon), containing the following columns\n the area of each shape.\n\n If `signed=True`, a positive area value corresponds to a clockwise loop,\n whereas a negative area value corresponds to a counter-clockwise loop.\n '''\n # Make a copy of the SVG data frame since we need to add columns to it.\n df_i = df_shapes.copy()\n df_i['vertex_count'] = (df_i.groupby(shape_i_columns)['x']\n .transform('count'))\n df_i['area_a'] = df_i.x\n df_i['area_b'] = df_i.y\n\n # Vector form of [Shoelace formula][1].\n #\n # [1]: http://en.wikipedia.org/wiki/Shoelace_formula\n df_i.loc[df_i.vertex_i == df_i.vertex_count - 1, 'area_a'] *= df_i.loc[df_i.vertex_i == 0, 'y'].values\n df_i.loc[df_i.vertex_i < df_i.vertex_count - 1, 'area_a'] *= df_i.loc[df_i.vertex_i > 0, 'y'].values\n\n df_i.loc[df_i.vertex_i == df_i.vertex_count - 1, 'area_b'] *= df_i.loc[df_i.vertex_i == 0, 'x'].values\n df_i.loc[df_i.vertex_i < df_i.vertex_count - 1, 'area_b'] *= df_i.loc[df_i.vertex_i > 0, 'x'].values\n\n area_components = df_i.groupby(shape_i_columns)[['area_a', 'area_b']].sum()\n shape_areas = .5 * (area_components['area_b'] - area_components['area_a'])\n\n if not signed:\n shape_areas.name = 'area'\n return shape_areas.abs()\n else:\n shape_areas.name = 'signed_area'\n return shape_areas"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef get_bounding_boxes(df_shapes, shape_i_columns):\n '''\n Return a `pandas.DataFrame` indexed by `shape_i_columns` (i.e., each row\n corresponds to a single shape/polygon), containing the following columns:\n\n - `width`: The width of the widest part of the shape.\n - `height`: The height of the tallest part of the shape.\n '''\n xy_groups = df_shapes.groupby(shape_i_columns)[['x', 'y']]\n xy_min = xy_groups.agg('min')\n xy_max = xy_groups.agg('max')\n\n shapes = (xy_max - xy_min).rename(columns={'x': 'width', 'y': 'height'})\n return xy_min.join(shapes)", "response": "Return a pandas. DataFrame indexed by shape_i_columns containing the minimum and maximum bounding boxes of the shape."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nreturn a pandas. DataFrame indexed by shape_i_columns containing the information about the shape.", "response": "def get_shape_infos(df_shapes, shape_i_columns):\n '''\n Return a `pandas.DataFrame` indexed by `shape_i_columns` (i.e., each row\n corresponds to a single shape/polygon), containing the following columns:\n\n - `area`: The area of the shape.\n - `width`: The width of the widest part of the shape.\n - `height`: The height of the tallest part of the shape.\n '''\n shape_areas = get_shape_areas(df_shapes, shape_i_columns)\n bboxes = get_bounding_boxes(df_shapes, shape_i_columns)\n return bboxes.join(pd.DataFrame(shape_areas))"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\ncalculates the bounding box of all points in a data frame.", "response": "def get_bounding_box(df_points):\n '''\n Calculate the bounding box of all points in a data frame.\n '''\n xy_min = df_points[['x', 'y']].min()\n xy_max = df_points[['x', 'y']].max()\n\n wh = xy_max - xy_min\n wh.index = 'width', 'height'\n bbox = pd.concat([xy_min, wh])\n bbox.name = 'bounding_box'\n return bbox"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef json_decoder_hook(dct, str_decoders=STRING_DECODERS,\n converters=MappingProxyType(dict())) -> dict:\n \"\"\"Decoder for parsing typical objects like uuid's and dates.\"\"\"\n\n for k, v in dct.items():\n if k in converters:\n parse_func = converters[k]\n dct[k] = parse_func(v)\n\n elif isinstance(v, str):\n for decode_func in str_decoders:\n v = decode_func(v)\n\n if not isinstance(v, str):\n break\n\n dct[k] = v\n elif isinstance(v, collections.Mapping):\n dct[k] = json_decoder_hook(v, str_decoders, converters)\n\n return dct", "response": "Decoder for parsing typical objects like uuid s and dates."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef make_json_decoder_hook(str_decoders=STRING_DECODERS,\n extra_str_decoders=tuple(),\n converters=MappingProxyType(dict())) -> Callable:\n \"\"\"Customize JSON string decoder hooks.\n\n Object hook for typical deserialization scenarios.\n\n Notes\n -----\n Specifying a field in converters will ensure custom decoding/passthrough.\n\n Parameters\n ----------\n str_decoders: functions for decoding strings to objects.\n extra_str_decoders: appends additional string decoders to str_decoders.\n converters: field / parser function mapping.\n \"\"\"\n\n str_decoders = tuple(chain(str_decoders, extra_str_decoders))\n object_hook = partial(json_decoder_hook, str_decoders=str_decoders,\n converters=converters)\n\n return object_hook", "response": "Create a JSON decoder hook for typical deserialization scenarios."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef wait_for_completion(report, interval=10):\n for jobid in report.collect('jobid'):\n try:\n if not Job.finished(jobid):\n logging.info('waiting for SLURM job %s', jobid)\n time.sleep(interval)\n while not Job.finished(jobid):\n time.sleep(interval)\n yield Job.fromid(jobid)._asdict()\n except OSError as e:\n if e.errno == errno.ENOENT:\n yield dict(id=str(jobid))\n else:\n raise e", "response": "Wait for asynchronous jobs to complete in the given campaign."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nreturn a list of colors with one main color and two opposite color evenly spread apart from the main one.", "response": "def getColors_Triad(hue=None, sat = 1, val = 1, spread = 60):\n\t\"\"\"\n\tCreate a palette with one main color and two opposite color evenly spread apart from the main one. \n\n\t:param hue: A 0-1 float with the starting hue value.\n\t:param sat: A 0-1 float with the palette saturation.\n\t:param val: A 0-1 float with the palette value.\n\t:param val: An int with the spread in degrees from the opposite color.\n\t:rtype: A list of :py:class:`Color` objects.\n\t\"\"\"\n\tpalette = list()\n\tif hue==None:\n\t\tleadHue = randFloat(0, 1)\n\telse:\n\t\tleadHue = hue\n\tpalette.append(Color(0,0,0,1).set_HSV(leadHue, sat, val))\n\tpalette.append(Color(0,0,0,1).set_HSV((leadHue + 0.5 + spread/360) % 1, sat, val))\n\tpalette.append(Color(0,0,0,1).set_HSV((leadHue + 0.5 - spread/360) % 1, sat, val))\t\n\treturn palette"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef default_job_name(self):\n name = ''\n if not self.root.existing_campaign:\n campaign_file = osp.basename(self.root.campaign_file)\n campaign = osp.splitext(campaign_file)[0]\n name += campaign + '/'\n name += self.tag\n return name", "response": "Returns the default job name if not already specified in the batch section."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nreturn the Jinja sbatch template for the current tag", "response": "def sbatch_template(self):\n \"\"\":return Jinja sbatch template for the current tag\"\"\"\n template = self.sbatch_template_str\n if template.startswith('#!'):\n # script is embedded in YAML\n return jinja_environment.from_string(template)\n return jinja_environment.get_template(template)"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nreturns Jinja sbatch template for the current tag as string", "response": "def sbatch_template_str(self):\n \"\"\":return Jinja sbatch template for the current tag as string\"\"\"\n templates = self.campaign.process.sbatch_template\n if isinstance(templates, Mapping):\n # find proper template according to the tag\n template = templates.get(self.tag)\n if template is None:\n template = templates.get('*')\n if template is None:\n template = SBATCH_JINJA_TEMPLATE\n else:\n template = templates\n return template"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef _create_sbatch(self, ostr):\n properties = dict(\n sbatch_arguments=self.sbatch_args, hpcbench_command=self.hpcbench_cmd\n )\n try:\n self.sbatch_template.stream(**properties).dump(ostr)\n except jinja2.exceptions.UndefinedError:\n self.logger.error('Error while generating SBATCH template:')\n self.logger.error('%%<--------' * 5)\n for line in self.sbatch_template_str.splitlines():\n self.logger.error(line)\n self.logger.error('%%<--------' * 5)\n self.logger.error('Template properties: %s', properties)\n raise", "response": "Write the SBATCH template to the output stream"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _execute_sbatch(self):\n commands = self.campaign.process.get('commands', {})\n sbatch = find_executable(commands.get('sbatch', 'sbatch'))\n sbatch_command = [sbatch, '--parsable', self.sbatch_filename]\n try:\n self.logger.debug(\n 'Executing command: %s',\n ' '.join(map(six.moves.shlex_quote, sbatch_command)),\n )\n sbatch_out = subprocess.check_output(\n sbatch_command, universal_newlines=True\n )\n except subprocess.CalledProcessError as cpe:\n self.logger.error(\n \"SBATCH return non-zero exit\" \"status %d for tag %s\",\n cpe.returncode,\n self.tag,\n )\n sbatch_out = cpe.output\n jobidre = re.compile(r'^([\\d]+)(?:;\\S*)?$')\n jobid = None\n for line in sbatch_out.splitlines():\n res = jobidre.match(line)\n if res is not None:\n jobid = res.group(1)\n self.logger.info(\"Submitted SBATCH job %s for tag %s\", jobid, self.tag)\n elif line:\n self.logger.warning(\"SBATCH: %s\", line)\n if jobid is None:\n self.logger.error(\"SBATCH submission failed for tag %s\", self.tag)\n return -1\n else:\n return int(jobid)", "response": "Execute the SBATCH command and return the job id"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\npartition ordered sequence by key.", "response": "def partition_ordered(sequence, key=None):\n \"\"\"Partition ordered sequence by key.\n\n Sequence is expected to already be ordered.\n\n Parameters\n ----------\n sequence: iterable data.\n key: partition key function\n\n Yields\n -------\n iterable tuple(s) of partition key, data list pairs.\n\n Examples\n --------\n 1. By object attributes.\n\n Partition sequence of objects by a height and weight attributes\n into an ordered dict.\n\n >> attributes = ('height', 'weight')\n >> OrderedDict(partition_ordered(sequence, attrgetter(*attributes)))\n\n 2. By index items.\n\n Partition sequence by the first character index of each element.\n\n >> index = 0\n >> sequence = ['112', '124', '289', '220', 'Z23']\n >> list(partition_ordered(sequence, itemgetter(index)))\n \"\"\"\n\n yield from ((k, list(g)) for k, g in groupby(sequence, key=key))"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef partition(predicate, iterable):\n\n t1, t2 = tee(iterable)\n\n return filterfalse(predicate, t1), filter(predicate, t2)", "response": "Use a predicate to partition true and false entries."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\ncreate a new dynamic portage project.", "response": "def PortageFactory(name, NAME, DOMAIN, BaseClass=autoportage.AutoPortage):\n \"\"\"\n Create a new dynamic portage project.\n\n Auto-Generated projects can only be used for compilie-time experiments,\n because there simply is no run-time test defined for it. Therefore,\n we implement the run symbol as a noop (with minor logging).\n\n This way we avoid the default implementation for run() that all projects\n inherit.\n\n Args:\n name: Name of the dynamic class.\n NAME: NAME property of the dynamic class.\n DOMAIN: DOMAIN property of the dynamic class.\n BaseClass: Base class to use for the dynamic class.\n\n Returns:\n A new class with NAME,DOMAIN properties set, unable to perform\n run-time tests.\n\n Examples:\n >>> from benchbuild.projects.gentoo.portage_gen import PortageFactory\n >>> from benchbuild.experiments.empty import Empty\n >>> c = PortageFactory(\"test\", \"NAME\", \"DOMAIN\")\n >>> c\n \n >>> i = c(Empty())\n >>> i.NAME\n 'NAME'\n >>> i.DOMAIN\n 'DOMAIN'\n \"\"\"\n\n def run_not_supported(self, *args, **kwargs):\n \"\"\"Dynamic projects don't support a run() test.\"\"\"\n del args, kwargs # Unused\n\n LOG.warning(\n \"Runtime testing not supported on auto-generated projects.\")\n return\n\n newclass = type(\n name, (BaseClass, ), {\n \"NAME\": NAME,\n \"DOMAIN\": DOMAIN,\n \"SRC_FILE\": \"none\",\n \"VERSION\": BaseClass.VERSION,\n \"GROUP\": \"auto-gentoo\",\n \"run\": run_not_supported,\n \"__module__\": \"__main__\"\n })\n return newclass"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\napplies the units schema to the data and return the result as a pint. Quantity object", "response": "def apply_unitschema(data, uschema, as_quantity=True,\n raise_outerr=False, convert_base=False,\n use_wildcards=False, list_of_dicts=False):\n \"\"\" apply the unit schema to the data\n\n Parameters\n ----------\n data : dict\n uschema : dict\n units schema to apply\n as_quantity : bool\n if true, return values as pint.Quantity objects\n raise_outerr : bool\n raise error if a unit cannot be found in the outschema\n convert_to_base : bool\n rescale units to base units\n use_wildcards : bool\n if true, can use * (matches everything) and ? (matches any single character)\n list_of_dicts: bool\n treat list of dicts as additional branches\n\n Examples\n --------\n >>> from pprint import pprint\n\n >>> data = {'energy':1,'x':[1,2],'other':{'y':[4,5]},'y':[4,5],'meta':None}\n >>> uschema = {'energy':'eV','x':'nm','other':{'y':'m'},'y':'cm'}\n >>> data_units = apply_unitschema(data,uschema)\n >>> pprint(data_units)\n {'energy': ,\n 'meta': None,\n 'other': {'y': },\n 'x': ,\n 'y': }\n\n >>> newschema = {'energy':'kJ','other':{'y':'nm'},'y':'m'}\n >>> new_data = apply_unitschema(data_units,newschema)\n >>> str(new_data[\"energy\"])\n '1.60217653e-22 kilojoule'\n >>> new_data[\"other\"][\"y\"].magnitude.round(3).tolist(), str(new_data[\"other\"][\"y\"].units)\n ([4000000000.0, 5000000000.0], 'nanometer')\n\n >>> old_data = apply_unitschema(new_data,uschema,as_quantity=False)\n >>> old_data[\"energy\"]\n 1.0\n >>> old_data[\"other\"][\"y\"].round(3).tolist()\n [4.0, 5.0]\n\n \"\"\" # noqa: E501\n try:\n from pint import UnitRegistry\n ureg = UnitRegistry()\n from pint.quantity import _Quantity\n except ImportError:\n raise ImportError('please install pint to use this module')\n list_of_dicts = '__list__' if list_of_dicts else None\n\n # flatten edict\n uschema_flat = flatten(uschema, key_as_tuple=True)\n # sorted by longest key size, to get best match first\n uschema_keys = sorted(uschema_flat, key=len, reverse=True)\n data_flat = flatten(data, key_as_tuple=True, list_of_dicts=list_of_dicts)\n\n for dkey, dvalue in data_flat.items():\n converted = False\n for ukey in uschema_keys:\n\n if not len(ukey) == len(dkey[-len(ukey):]):\n continue\n\n if use_wildcards:\n match = all(\n [fnmatch(d, u) for u, d in zip(ukey, dkey[-len(ukey):])])\n else:\n match = ukey == dkey[-len(ukey):]\n\n if match:\n # handle that it return an numpy object type if list of floats\n if isinstance(dvalue, (list, tuple)):\n dvalue = np.array(dvalue)\n if dvalue.dtype == np.object:\n dvalue = dvalue.astype(float)\n\n if isinstance(dvalue, _Quantity):\n quantity = dvalue.to(uschema_flat[ukey])\n else:\n quantity = ureg.Quantity(dvalue, uschema_flat[ukey])\n\n if convert_base:\n quantity = quantity.to_base_units()\n\n if as_quantity:\n data_flat[dkey] = quantity\n else:\n data_flat[dkey] = quantity.magnitude\n break\n\n if not converted and raise_outerr:\n raise KeyError('could not find units for {}'.format(dkey))\n\n return unflatten(data_flat, list_of_dicts=list_of_dicts)"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nsplits quantities into a single object", "response": "def split_quantities(data, units='units', magnitude='magnitude',\n list_of_dicts=False):\n \"\"\" split pint.Quantity objects into pairs\n\n Parameters\n ----------\n data : dict\n units : str\n name for units key\n magnitude : str\n name for magnitude key\n list_of_dicts: bool\n treat list of dicts as additional branches\n\n Examples\n --------\n >>> from pprint import pprint\n\n >>> from pint import UnitRegistry\n >>> ureg = UnitRegistry()\n >>> Q = ureg.Quantity\n\n >>> qdata = {'energy': Q(1.602e-22, 'kilojoule'),\n ... 'meta': None,\n ... 'other': {'y': Q([4,5,6], 'nanometer')},\n ... 'x': Q([1,2,3], 'nanometer'),\n ... 'y': Q([8,9,10], 'meter')}\n ...\n >>> split_data = split_quantities(qdata)\n >>> pprint(split_data)\n {'energy': {'magnitude': 1.602e-22, 'units': 'kilojoule'},\n 'meta': None,\n 'other': {'y': {'magnitude': array([4, 5, 6]), 'units': 'nanometer'}},\n 'x': {'magnitude': array([1, 2, 3]), 'units': 'nanometer'},\n 'y': {'magnitude': array([ 8, 9, 10]), 'units': 'meter'}}\n\n \"\"\"\n try:\n from pint.quantity import _Quantity\n except ImportError:\n raise ImportError('please install pint to use this module')\n list_of_dicts = '__list__' if list_of_dicts else None\n data_flatten = flatten(data, list_of_dicts=list_of_dicts)\n for key, val in data_flatten.items():\n if isinstance(val, _Quantity):\n data_flatten[key] = {units: str(val.units),\n magnitude: val.magnitude}\n return unflatten(data_flatten, list_of_dicts=list_of_dicts)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef combine_quantities(data, units='units', magnitude='magnitude',\n list_of_dicts=False):\n \"\"\" combine pairs into pint.Quantity objects\n\n Parameters\n ----------\n data : dict\n units : str\n name of units key\n magnitude : str\n name of magnitude key\n list_of_dicts: bool\n treat list of dicts as additional branches\n\n Examples\n --------\n >>> from pprint import pprint\n\n >>> sdata = {'energy': {'magnitude': 1.602e-22, 'units': 'kilojoule'},\n ... 'meta': None,\n ... 'other': {'y': {'magnitude': [4, 5, 6], 'units': 'nanometer'}},\n ... 'x': {'magnitude': [1, 2, 3], 'units': 'nanometer'},\n ... 'y': {'magnitude': [8,9,10], 'units': 'meter'}}\n ...\n >>> combined_data = combine_quantities(sdata)\n >>> pprint(combined_data)\n {'energy': ,\n 'meta': None,\n 'other': {'y': },\n 'x': ,\n 'y': }\n\n \"\"\" # noqa: E501\n try:\n from pint import UnitRegistry\n ureg = UnitRegistry()\n except ImportError:\n raise ImportError('please install pint to use this module')\n list_of_dicts = '__list__' if list_of_dicts else None\n\n data_flatten2d = flatten2d(data, list_of_dicts=list_of_dicts)\n new_dict = {}\n for key, val in list(data_flatten2d.items()):\n if units in val and magnitude in val:\n quantity = ureg.Quantity(val.pop(magnitude), val.pop(units))\n if not val:\n data_flatten2d.pop(key)\n new_dict[key] = quantity\n final_dict = merge([data_flatten2d, new_dict])\n # olddict = unflatten(data_flatten2d,list_of_dicts=list_of_dicts)\n # new_dict = unflatten(new_dict,list_of_dicts=list_of_dicts)\n return unflatten(\n final_dict, list_of_dicts=list_of_dicts)", "response": "Combine quantities into single object."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef get_version_from_cache_dir(src_file):\n if src_file is None:\n return None\n\n tmp_dir = local.path(str(CFG[\"tmp_dir\"]))\n if tmp_dir.exists():\n cache_file = tmp_dir / src_file\n dir_hash = get_hash_of_dirs(cache_file)\n if dir_hash is None:\n return None\n if len(str(dir_hash)) <= 7:\n return str(dir_hash)\n return str(dir_hash)[:7]\n return None", "response": "Returns a version of a project from the cache directory."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef get_git_hash(from_url):\n from benchbuild.utils.cmd import git\n if from_url is None:\n return \"\"\n\n if not path.exists(from_url):\n return \"\"\n\n with local.cwd(from_url):\n return git(\"rev-parse\", \"HEAD\", retcode=None)", "response": "Get the git commit hash of HEAD from a file system url."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef fetch_time_output(marker, format_s, ins):\n from parse import parse\n\n timings = [x for x in ins if marker in x]\n res = [parse(format_s, t) for t in timings]\n return [_f for _f in res if _f]", "response": "Fetch the output of a.\n "} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef path_to_list(pathstr):\n return [elem for elem in pathstr.split(os.path.pathsep) if elem]", "response": "Conver a path string to a list of path elements."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nborrow from wxglade. py", "response": "def determine_path():\n \"\"\"Borrowed from wxglade.py\"\"\"\n root = __file__\n if os.path.islink(root):\n root = os.path.realpath(root)\n return os.path.dirname(os.path.abspath(root))"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nreturning a list of filenames found at path.", "response": "def template_files(path, exts=None):\n \"\"\"\n Return a list of filenames found at @path.\n\n The list of filenames can be filtered by extensions.\n\n Arguments:\n path: Existing filepath we want to list.\n exts: List of extensions to filter by.\n\n Returns:\n A list of filenames found in the path.\n \"\"\"\n if not os.path.isabs(path):\n _path = os.path.join(determine_path(), path)\n if not (os.path.exists(_path) and os.path.isdir(_path)):\n return []\n if not exts:\n exts = []\n files = os.listdir(_path)\n files = [f for f in files if os.path.splitext(f)[-1] in exts]\n files = [os.path.join(path, f) for f in files]\n return files"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nread a template file from the resources and return it as str.", "response": "def template_str(template):\n \"\"\"Read a template file from the resources and return it as str.\"\"\"\n tmpl_file = os.path.join(determine_path(), template)\n with open(tmpl_file, mode='r') as tmpl_strm:\n return \"\".join(tmpl_strm.readlines())"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef mkfile_uchroot(filepath, root=\".\"):\n from benchbuild.utils.uchroot import no_args, uretry\n\n uchroot = no_args()\n uchroot = uchroot[\"-E\", \"-A\", \"-C\", \"-w\", \"/\", \"-r\"]\n uchroot = uchroot[os.path.abspath(root)]\n uretry(uchroot[\"--\", \"/bin/touch\", filepath])", "response": "Create a file inside a uchroot container."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\ncreate a file inside a uchroot container.", "response": "def mkdir_uchroot(dirpath, root=\".\"):\n \"\"\"\n Create a file inside a uchroot env.\n\n You will want to use this when you need to create a file with apropriate\n rights inside a uchroot container with subuid/subgid handling enabled.\n\n Args:\n dirpath:\n The dirpath that should be created. Absolute inside the\n uchroot container.\n root:\n The root PATH of the container filesystem as seen outside of\n the container.\n \"\"\"\n from benchbuild.utils.uchroot import no_args, uretry\n\n uchroot = no_args()\n uchroot = uchroot[\"-E\", \"-A\", \"-C\", \"-w\", \"/\", \"-r\"]\n uchroot = uchroot[os.path.abspath(root)]\n uretry(uchroot[\"--\", \"/bin/mkdir\", \"-p\", dirpath])"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\ncreates a directory if required.", "response": "def mkdir_interactive(dirpath):\n \"\"\"\n Create a directory if required.\n\n This will query the user for a confirmation.\n\n Args:\n dirname: The path to create.\n \"\"\"\n from benchbuild.utils.cmd import mkdir\n if os.path.exists(dirpath):\n return\n\n response = ui.ask(\n \"The directory {dirname} does not exist yet. \"\n \"Should I create it?\".format(dirname=dirpath),\n default_answer=True,\n default_answer_str=\"yes\")\n\n if response:\n mkdir(\"-p\", dirpath)\n print(\"Created directory {0}.\".format(dirpath))"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef get_val_by_text(root,search):\n\n found_flag = False\n for el in root.iter():\n if found_flag:\n return(el)\n if el.text == search:\n # We want the next el\n found_flag = True", "response": "From MeasYaps XML root find next sibling of node matching search."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nread raw data from Siemens MRI scanners with IDEA VB15. Will return an array of measured k-space data from raw data from Siemens MRI scanners using IDEA VB15 (single value). If the option '-I' is used, then image space data will be returned instead. Usage: readMeasDataVB15 filename [ -t ] [ -rfft ] [ -r1 ] [ -rp ] [ -rn ] [ -skipts ] [ -nnavek ] [ -ros ] [ -rosa ] [ -I ] [ -w ] [-npz] Examples: python3 -m rawdatarinator.readMeasDataVB15 raw.dat -w or using the shortned alias... python3 -m rawdatarinator.raw raw.dat -w Command-line Options: filename Filename of file containing raw measurements. -rfft (resetFFTscale) Resets FFTscale and DataCorrection for each coil to 1. -r1 (readOneCoil) Read measurement data from from individual coil. -rp (readPhaseCorInfo) _ -rn (readNavigator) _ -skipts (skip readTimeStamp) _ -nnavek (nNavEK) _ -ros (removeOS) Flag to remove oversampling (OS) in the x direction. removeOS=True is more efficient as it processes each readout line independently, reducing the required memory space to keep all measured data. -rosa (removeOSafter) Flag to remove oversampling (OS) in the x direction. This works in image space, cutting FOV. Not likely a good idea for radial. -I (transformToImageSpace) Produce image space representation. Note that there is no correction for partial Fourier or parallel imaging k-space undersampling. The given version of code only uses numpy's FFT operation. -w (writeToFile) Save k-space or image space volume. Currently the output filename is auto generated. -npz (npz) Save k-space or image space volume using the .npz file extension. Default is to use hdf5 file standard. -h (help) Displays this documentation.", "response": "def readMeasDataVB15(filename,\n resetFFTscale=False,\n readOneCoil=False,\n readPhaseCorInfo=False,\n readNavigator=False,\n readTimeStamp=True,\n nNavEK=False,\n removeOS=False,\n removeOSafter=False,\n transformToImageSpace=False,\n writeToFile=False,\n npz=False):\n \"\"\"Read raw data from Siemens MRI scanners with IDEA VB15.\n\n Will return an array of measured k-space data from raw data from\n Siemens MRI scanners using IDEA VB15 (single value). If the option\n '-I' is used, then image space data will be returned instead.\n\n Usage:\n readMeasDataVB15 filename [ -t ] [ -rfft ] [ -r1 ] [ -rp ] [ -rn ]\n [ -skipts ] [ -nnavek ] [ -ros ]\n [ -rosa ] [ -I ] [ -w ] [-npz]\n\n Examples:\n python3 -m rawdatarinator.readMeasDataVB15 raw.dat -w\n\n or using the shortned alias...\n \n python3 -m rawdatarinator.raw raw.dat -w\n\n Command-line Options:\n filename Filename of file containing raw measurements.\n\n -rfft (resetFFTscale)\n Resets FFTscale and DataCorrection for each coil\n to 1.\n\n -r1 (readOneCoil)\n Read measurement data from from individual coil.\n\n -rp (readPhaseCorInfo)\n _\n\n -rn (readNavigator)\n _\n\n -skipts (skip readTimeStamp)\n _\n\n -nnavek (nNavEK)\n _\n\n -ros (removeOS)\n Flag to remove oversampling (OS) in the x\n direction. removeOS=True is more efficient as it\n processes each readout line independently,\n reducing the required memory space to keep all\n measured data.\n\n -rosa (removeOSafter)\n Flag to remove oversampling (OS) in the x\n direction. This works in image space, cutting FOV.\n Not likely a good idea for radial.\n\n -I (transformToImageSpace)\n Produce image space representation. Note that\n there is no correction for partial Fourier or\n parallel imaging k-space undersampling. The given\n version of code only uses numpy's FFT operation.\n\n -w (writeToFile)\n Save k-space or image space volume. Currently the\n output filename is auto generated.\n\n -npz (npz)\n Save k-space or image space volume using the .npz\n file extension. Default is to use hdf5 file\n standard.\n\n -h (help)\n Displays this documentation.\n \"\"\"\n\n filename_temp = os.path.splitext(filename)[0]\n if transformToImageSpace is False:\n filenameOut = '%s_Kspace' % filename_temp\n else:\n filenameOut = '%s_imageSpace' % filename_temp\n\n # Useful Parameters\n globalHeader = 32\n localHeader = 128\n\n infoParser = InfoParser()\n xmlstr = infoParser.raw2xml(filename)\n\n # Start in MeasYaps. MeasYaps starts with and ends at\n # the next XProtocol mod='Phoenix'\n startIdx = xmlstr.find('')\n endIdx = xmlstr.find('')\n my_xmlstr = '' + xmlstr[startIdx:endIdx] + '' # add root\n\n # Parse into XML\n root = ET.fromstring(my_xmlstr)\n\n # Used to decode Partial Fourier fractions (poor man's switch)\n fractionDict = { 10: 1.0,\n 8 : 7./8.,\n 4 : 0.75,\n 2 : 5./8.,\n 1 : 0.5 }\n\n # vals are tuples: (key,search,default,lambda)\n # key => key to the dictionary entry\n # search => string to search the xml document with\n # default => if not found in xml doc, use this value\n # lambda => whatever value you end up with, operate on it with this\n # anonymous function.\n vals = [\n ('ScanDimension','sKSpace.ucDimension',None,lambda x:int(x,16)),\n ('flag3D',None,None,lambda _:True if data['ScanDimension'] is 4 else False),\n ('NxAll','sKSpace.lBaseResolution',None,lambda x:int(x)),\n ('NyAll','sKSpace.lPhaseEncodingLines',None,lambda x:int(x)),\n ('OSfactorPE','sKSpace.dPhaseOversamplingForDialog',None,lambda x:1.0 if my_xmlstr.find('sKSpace.dPhaseOversamplingForDialog') < 0 else 1.0+float(x)),\n ('NzAll','sKSpace.lPartitions',None,lambda x:int(x)),\n ('OSfactor3D','sKSpace.dSliceOversamplingForDialog',None,lambda x:1.0 if my_xmlstr.find('sKSpace.dSliceOversamplingForDialog') < 0 else 1.0+float(x)),\n ('phaseResolution','sKSpace.dPhaseResolution',None,lambda x:float(x)),\n ('sliceResolution','sKSpace.dSliceResolution',None,lambda x:float(x)),\n ('Nsl','sSliceArray.lSize',None,lambda x:int(x)),\n ('Nconc','sSliceArray.lConc',None,lambda x:int(x)),\n ('Nc',None,None,lambda _:len(re.findall(r'\\.lRxChannelConnected',my_xmlstr))),\n ('Nc',None,None,lambda _:data['Nc']-1 if my_xmlstr.find('AdjustSeq%/AdjCoilSensSeq') > 0 else data['Nc']),\n ('nContrast','lContrasts',1,lambda x:int(x)),\n ('nSet','lSets',1,lambda x:int(x)),\n ('nAverage','lAverages',1,lambda x:int(x)),\n ('nRepetition','lRepetitions',None,lambda x:1 if my_xmlstr.find('lRepetitions') < 0 else 1+int(x)),\n ('nPhase','sPhysioImaging.lPhases',1,lambda x:int(x)),\n ('fractionY','sKSpace.PhasePartialFourier',10,lambda x: fractionDict[int(x)]),\n ('fractionZ','sKSpace.SlicePartialFourier',10,lambda x: fractionDict[int(x)]),\n ('phasePartialFourierForSNR','sKSpace.dSeqPhasePartialFourierForSNR',1.0,lambda x:float(x)),\n ('EPIFactor','sFastImaging.lEPIFactor',None,lambda x:int(x)),\n ('turboFactor','sFastImaging.lTurboFactor',1,lambda x:int(x)),\n ('PATMode','sPat.ucPATMode',None,lambda x:int(x,16)),\n ('PATRefScanMode','sPat.ucRefScanMode',None,lambda x:int(x,16)),\n ('AccelFactorPE','sPat.lAccelFactPE',None,lambda x:int(x)),\n ('AccelFactor3D','sPat.lAccelFact3D',None,lambda x:int(x)),\n ('nRefLinesPE','sPat.lRefLinesPE',None,lambda x:0 if data['AccelFactorPE'] is 1 else int(x)),\n ('nRefLines3D','sPat.lRefLines3D',None,lambda x:0 if data['AccelFactor3D'] is 1 else int(x)) ]\n\n # Evaluate all tuples\n data = dict() # dictionary to store all values we want to compre with MATLAB\n for tup in vals:\n if tup[1] is not None:\n idx = my_xmlstr.find(tup[1])\n else:\n idx = -1\n\n if idx < 0:\n val = tup[2] # Take the default if we can't find it\n else:\n val = get_val_by_text(root,tup[1]).text\n\n afun = tup[3]\n if afun is not None:\n val = afun(val) # Evaluate anonymous function if provided\n data[tup[0]] = val # Store the value in the dictionary\n\n ## Now use the whole xml document\n root = ET.fromstring(xmlstr)\n\n # Enforce a max value for Nc\n Nct = get_yaps_by_name(root,'iMaxNoOfRxChannels',lambda x:int(x))\n if Nct is not None:\n data['Nct'] = Nct\n if Nct < data['Nc']:\n data['Nc'] = Nct\n\n # If this exists, then we'll need it\n nPhCorrScan = get_yaps_by_name(root,'lNoOfPhaseCorrScans',lambda x:int(x))\n if nPhCorrScan is not None:\n data['nPhCorrScan'] = nPhCorrScan\n\n # Define some more variables\n if data['turboFactor'] > 1:\n data['nPhCorEcho'] = 1\n data['nPhCorScan'] = 1\n if data['EPIFactor'] > 1:\n data['nPhCorScan'] = 1\n data['nPhCorEcho'] = 3\n\n if data['AccelFactorPE'] is 1:\n data['FirstFourierLine'] = 1\n data['FirstRefLine'] = 1\n else:\n data['FirstFourierLine'] = get_yaps_by_name(root,'lFirstFourierLine',lambda x:1+int(x),1)\n data['FirstRefLine'] = get_yaps_by_name(root,'lFirstRefLine',lambda x:1+int(x),1)\n\n if data['AccelFactor3D'] is 1:\n data['FirstFourierPar'] = 1\n data['FirstRefPartition'] = 1\n else:\n data['FirstFourierPartition'] = get_yaps_by_name(root,'lFirstFourierPartition',lambda x:1+int(x),1)\n data['FirstRefPartition'] = get_yaps_by_name(root,'lFirstRefPartition',lambda x:1+int(x),1)\n\n data['NxOS'] = get_yaps_by_name(root,'iNoOfFourierColumns',lambda x:int(x))\n data['OSfactorRO'] = 2 # we can actually find this in YAPS, but MATLAB impl gives as magic num\n data['Nx'] = int(np.around(data['NxOS']/data['OSfactorRO']))\n data['Ny'] = int(root.find(\"ParamFunctor[@name='FillMiniHeaderData']/ParamLong[@name='NoOfFourierLines']/value\").text)\n data['Nz'] = get_yaps_by_name(root,'iNoOfFourierPartitions',lambda x:int(x))\n data['NxRecon'] = get_yaps_by_name(root,'iRoFTLength',lambda x:int(x))\n data['NyRecon'] = get_yaps_by_name(root,'iPEFTLength',lambda x:int(x))\n data['NzRecon'] = 1 if data['Nz'] is 1 else get_yaps_by_name(root,'i3DFTLength',lambda x:int(x))\n data['NslicePerConcat'] = get_yaps_by_name(root,'ushSlicePerConcat',lambda x:int(x))\n\n ## Partial Fourier Mode and Parameters\n data['PCAlgorithm'] = get_yaps_by_name(root,'lPCAlgorithm',lambda x:int(x))\n data['NoOfPhaseCorrColumns'] = get_yaps_by_name(root,'lNoOfPhaseCorrColumns',lambda x:int(x),data['Nx']/2)\n data['NoOfPhaseCorrLines'] = get_yaps_by_name(root,'lNoOfPhaseCorrLines',lambda x:int(x),data['NyAll']/2)\n data['NoOfPhaseCorrPartitions'] = get_yaps_by_name(root,'lNoOfPhaseCorrPartitions',lambda x:int(x),data['NzAll']/2)\n data['ColSlopeLength'] = get_yaps_by_name(root,'lColSlopeLength',lambda x:int(x),data['Nx']/4)\n # I think there's a mistake here in the MATLAB code: ColSlopeLength defined again instead of LinSlopeLength.\n # Commented is it how I think it should be and we'll go from there. The value is not actually used.\n # data['LinSlopeLength'] = get_yaps_by_name(root,'lLinSlopeLength',lambda x:int(x),data['NyAll']/4)\n data['ColSlopeLength'] = get_yaps_by_name(root,'lLinSlopeLength',lambda x:int(x),data['NyAll']/4)\n data['ParSlopeLength'] = get_yaps_by_name(root,'lParSlopeLength',lambda x:int(x),data['NzAll']/4)\n\n ## Raw data correction factors, use the MeasYaps portion of the xml document\n root = ET.fromstring(my_xmlstr)\n data['CorrFactor'] = np.ones(data['Nc'])\n for c in range(data['Nc']):\n text = 'axRawDataCorrectionFactor[0][%d].dRe' % c\n data['CorrFactor'][c] = 1 if my_xmlstr.find(text) < 0 else float(get_val_by_text(root,text).text)\n\n text = 'axRawDataCorrectionFactor[0][%d].dIm' % c\n if my_xmlstr.find(text) >= 0:\n data['CorrFactor'][c] = data['CorrFactor'][c] + 1j*float(get_val_by_text(root,text).text)\n\n ## FFT Correction Factors\n data['FFTCorrFactor'] = np.ones(data['Nc'])\n if resetFFTscale is False:\n data['FFTCorrFactor'] = np.ones(data['Nc'])\n for c in range(data['Nc']):\n text = 'asCoilSelectMeas[0].aFFT_SCALE[%d].flFactor' % c\n data['FFTCorrFactor'][c] = float(get_val_by_text(root,text).text)\n\n ## For PC Angio\n data['Nset'] = 1\n text = 'sAngio.ucPCFlowMode'\n data['PCMRAFlag'] = int(get_val_by_text(root,text).text,16) if my_xmlstr.find(text) > 0 else 0\n\n if data['PCMRAFlag'] is 1:\n text = 'sAngio.sFlowArray.lSize'\n if my_xmlstr.find(text) < 0:\n data['Nset'] = int(get_val_by_text(root,text).text)\n\n ## Recalculation of partial Fourier factors and EPI/turbo factor for EPI and TSE\n data['fractionPE'] = float(data['Ny'])/float(data['NyAll'])\n data['fraction3D'] = float(data['Nz'])/float(data['NzAll'])\n data['EPIFactor'] = np.around(data['fractionPE']*data['EPIFactor'])\n data['nEPITrain'] = data['Ny']/data['EPIFactor']\n data['turboFactor'] = np.around(data['fractionPE']*data['turboFactor'])\n data['nTSETrain'] = data['Ny']/data['turboFactor']\n data['Nc0'] = data['Nc']\n data['Nc'] = 1 if readOneCoil is True else data['Nc0']\n\n ## Calculation of the number of valid k-space readouts and k-space data matrix dimensions\n if data['PATMode'] is 1:\n data['nReadout'] = (\n data['nAverage']* \\\n data['nPhase']* \\\n data['nRepetition']* \\\n data['nContrast']* \\\n data['Nsl']* \\\n data['Nz']* \\\n data['Nc']* \\\n data['Ny'])\n\n elif (data['PATMode'] is 2) and (data['PATRefScanMode'] is 2):\n if (data['Ny'] % 2) is 1:\n data['NyPAT'] = (data['Ny'] - 1 + data['nRefLinesPE']*(data['AccelFactorPE'] - 1))/data['AccelFactorPE']\n else:\n data['NyPAT'] = np.floor((data['Ny'] + data['nRefLinesPE']*(data['AccelFactorPE'] - 1))/data['AccelFactorPE'])\n\n data['nReadout'] = (\n data['nAverage']* \\\n data['nPhase']* \\\n data['nRepetition']* \\\n data['nContrast']* \\\n data['Nsl']* \\\n data['Nz']* \\\n data['Nc']* \\\n data['NyPAT'])\n\n if removeOS is True:\n data['kSpace'] = np.zeros((\n data['nAverage'],\n data['nPhase'],\n data['nRepetition'],\n data['nContrast'],\n data['Nsl'],\n data['Nc'],\n data['Nz'],\n data['Nx'],\n data['Ny']), dtype=np.complex64)\n else:\n data['kSpace'] = np.zeros((\n data['nAverage'],\n data['nPhase'],\n data['nRepetition'],\n data['nContrast'],\n data['Nsl'],\n data['Nc'],\n data['Nz'],\n data['NxOS'],\n data['Ny']), dtype=np.complex64)\n\n if (readPhaseCorInfo is True) and (data['nPhCorScan'] > 0):\n data['kPhaseCor'] = np.zeros((\n data['nPhCorScan'],\n data['nPhCorEcho'],\n data['Nsl'],\n data['nRepetition'],\n data['Nc'],\n data['NxOS']), dtype=np.float32)\n\n if readNavigator is True:\n data['nNavigator'] = (\n data['nAverage']* \\\n data['nPhase']* \\\n data['nRepetition']* \\\n data['nContrast']* \\\n data['Nsl']* \\\n data['Nz']* \\\n data['Nc']* \\\n data['nEPITrain'] \\\n *data['nNavEK'])\n\n data['kNavigator'] = np.zeros((\n data['nAverage'],\n data['nPhase'],\n data['nRepetition'],\n data['nContrast']*nNavEK,\n data['Nsl'],\n data['Nc'],\n data['Nz'],\n data['nEPITrain'],\n data['NxOS']), dtype=np.float32)\n\n if readTimeStamp is True:\n data['nTimeStamp'] = (\n data['nAverage']* \\\n data['nPhase']* \\\n data['nRepetition']* \\\n data['nContrast']* \\\n data['Nz'])\n\n data['timeStamp'] = np.zeros((\n data['nAverage'],\n data['nPhase'],\n data['nRepetition'],\n data['nContrast']*nNavEK,\n data['Nz']), dtype=np.float32)\n\n ## Data Readout and Reordering\n # Read k-space data\n data['noiseMeasCounter'] = 0\n data['noiseMeas'] = np.zeros((data['NxOS'],data['Nc']),dtype=np.complex64)\n data['navigatorPrep'] = 0\n data['LineN'] = -1\n data['Ndr'] = 1\n\n for r in range(data['Ndr']):\n data['xCoil'] = r\n with open(filename,'rb') as f:\n readFlag = True\n skipField = 0\n countNavigator = 0\n navigatorDataON = False\n\n temp1 = np.zeros(data['NxOS'])\n data['dataField'] = np.fromfile(f,dtype=np.int32,count=1)[0]\n f.seek(data['dataField'],os.SEEK_SET)\n\n while readFlag is True:\n if readTimeStamp is True:\n f.seek(12,os.SEEK_CUR)\n data['timeS'] = np.fromfile(f,dtype=np.uint32,count=1)[0]\n f.seek(4,os.SEEK_CUR)\n else:\n f.seek(20,os.SEEK_CUR)\n\n data['evalMask1'] = np.fromfile(f,dtype=np.uint32,count=1)[0]\n data['evalMask2'] = np.fromfile(f,dtype=np.uint32,count=1)[0]\n flag = [(32 - m.start()) for m in re.finditer('1',np.binary_repr(data['evalMask1'],32))]\n\n # Tuples: (key,dtype,afun)\n vals = [ ('Nxr',None,None),\n ('Ncr',None,None),\n ('Line',None,None),\n ('Acquisition',None,None),\n ('Slice',None,None),\n ('Partition',None,None),\n ('Echo',None,None),\n ('Phase',None,None),\n ('Repetition',None,None),\n ('Set',None,None),\n ('CutOffDataPre',None,lambda:f.seek(12,os.SEEK_CUR)),\n ('CutOffDataPost',None,None),\n ('KSpaceCentreColumn',None,None),\n ('CoilMode',None,None),\n ('ReadOutOffCentre',np.float32,None),\n ('KSpaceCentreLineNo',None,lambda:f.seek(4,os.SEEK_CUR)),\n ('KSpaceCentrePartitionNo',None,None),\n ('Channel',None,lambda:f.seek(44,os.SEEK_CUR)) ]\n for tup in vals:\n t = np.uint16 if tup[1] is None else tup[1]\n if hasattr(tup[2], '__call__'):\n tup[2]()\n data[tup[0]] = np.fromfile(f,dtype=t,count=1)[0]\n\n f.seek(2,os.SEEK_CUR)\n\n if 1 in flag:\n break\n\n if any([k for k in [2,22,26] if k in flag]):\n if (22 in flag) and (readPhaseCorInfo is False):\n skipField = data['nPhCorScan']*data['nPhCorEcho']*data['Ncr']*(localHeader + 8*data['Nxr']) - localHeader\n\n if (22 in flag) and (readPhaseCorInfo is True):\n skipField = -localHeader\n f.seek(skipField,os.SEEK_CUR)\n skipField = 0\n\n for m in range(data['nPhCorScan']*data['nPhCorEcho']*data['Ncr']):\n infoMDH_TimeStamp = readMDH_TimeStamp_VB13(f)\n temp = np.fromfile(f,dtype=np.float32,count=2*data['Nxr'])\n\n if 25 in flag:\n temp[0::2] = np.flipud(temp[0::2])\n temp[1::2] = np.flipud(temp[1::2])\n if data['CutOffDataPre'] > 0:\n temp[0:2*data['CutOffDataPre']] = 0\n if data['CutOffDataPost'] > 0:\n temp[len(temp) - 2*data['CutOffDataPost']:] = 0\n\n data['kPhaseCor'][data['Echo'],np.ceil(m/data['Ncr']),data['Slice'],data['Repetition'],data['Channel'],:] = (temp[0::2] + 1j*temp[1::2]).astype(np.float32)\n\n if (2 in flag) and (readNavigator is False):\n skipField = data['Ncr']*(localHeader + 8*data['Nxr']) - localHeader\n if (2 in flag) and (readNavigator is True):\n if (countNavigator is False) and (navigatorPrep is False):\n kNavigator = np.zeros((\n data['Nxr'],\n data['Ncr'],\n data['nContrast']*nNavEK,\n data['nEPITrain'],\n data['Nz'],\n data['Nsl'],\n data['nAverage'],\n data['nPhase'],\n data['nRepetition']),dtype=np.float32)\n kNavigatorTemp = np.zeros((\n data['Nxr'],\n data['Ncr'],\n data['nContrast']*nNavEK),dtype=float32)\n navigatorPrep = 1\n\n skipField = -localHeader\n f.seek(skipField,os.SEEK_CUR)\n skipField = 0\n\n for m in range(nNavEK*data['Ncr']):\n infoMDH_TimeStamp = readMDH_TimeStamp_VB13(f)\n temp = np.fromfile(f,dtype=np.float32,count=2*data['Nxr'])\n\n if 25 in flag:\n temp[0::2] = np.flipud(temp[0::2])\n temp[1::2] = np.flipud(temp[1::2])\n if data['CutOffDataPre'] > 0:\n temp[0:2*data['CutOffDataPre']] = 0\n if data['CutOffDataPost'] > 0:\n temp[len(temp) - 2*data['CutOffDataPost']-1:] = 0;\n kNavigatorTemp[:,data['Channel'],data['Echo']] = (temp[0::2] + 1j*temp[1::2]).astype(np.complex64)\n\n navigatorDataON = True\n\n if 26 in flag:\n temp = np.fromfile(f,dtype=np.float32,count=2*data['Nxr'])\n\n if 25 in flag:\n temp[0::2] = np.flipud(temp[0::2])\n temp[1::2] = np.flipud(temp[1::2])\n\n data['noiseMeas'][:,data['Channel']] = temp[0::2] + 1j*temp[1::2]\n skipField = 0\n\n f.seek(skipField,os.SEEK_CUR)\n\n else:\n temp = np.fromfile(f,dtype=np.float32,count=2*data['Nxr'])\n if 25 in flag:\n temp[0::2] = np.flipud(temp[0::2])\n temp[1::2] = np.flipud(temp[1::2])\n if data['CutOffDataPre'] > 0:\n temp[0:2*data['CutOffDataPre']-1] = 0\n if data['CutOffDataPost'] > 0:\n temp[len(temp) - 2*data['CutOffDataPost']-1] = 0\n if 11 in flag:\n temp = data['CorrFactor'][data['Channel']]*temp\n temp = data['FFTCorrFactor'][data['Channel']]*temp\n\n if readOneCoil is False:\n if removeOS is True:\n temp1[len(temp1) - data['Nxr']:] = temp[0::2] + 1j*temp[1::2]\n tempX = np.fftshift(np.fft(np.fftshift(temp1)))\n tempK = np.fftshift(np.ifftshift(np.fftshift(tempX[np.around((data['NxOS'] - data['Nx'])/2):data['Nx'] + np.around((data['NxOS'] - data['Nx'])/2)])))\n data['kSpace'][data['Acquisition'],data['Phase'],data['Repetition'],data['Echo'],data['Slice'],data['Channel'],data['Partition'],:,data['Line']] = tempK.astype(np.complex64)\n else:\n data['kSpace'][data['Acquisition'],data['Phase'],data['Repetition'],data['Echo'],data['Slice'],data['Channel'],data['Partition'],data['kSpace'].shape[7] - data['Nxr']:data['kSpace'].shape[7],data['Line']] = (temp[0::2] + 1j*temp[1::2]).astype(np.complex64)\n\n elif (readOneCoil is True) and (data['Channel']+1 == coilIndex):\n if removeOS is True:\n temp1[len(temp1) - data['Nxr']:] = temp[0::2] + 1j*temp[1::2]\n tempx = np.fftshift(np.fft(np.fftshift(temp1)))\n tempK = np.fftshift(np.fft(np.fftshift(tempX[np.around((data['NxOS'] - data['Nx'])/2):data['Nx'] + np.around((data['NxOS'] - data['Nx'])/2)])))\n data['kSpace'][data['Acquisition'],data['Phase'],data['Repetition'],data['Echo'],data['Slice'],0,data['Partition'],:,data['Line']] = tempK.astype(np.complex64)\n else:\n data['kSpace'][data['Acquisition'],data['Phase'],data['Repetition'],data['Echo'],data['Slice'],0,data['Partition'],data['kSpace'].shape[7] - data['Nxr']:,data['Line']] = (temp[0::2] + 1j*temp[1::2]).astype(np.complex64)\n\n if (readTimeStamp is True) and (data['Channel'] == 0) and (navigatorDataON is True):\n data['EPITrain'] = countNavigator % data['nEPITrain']\n data['timeStamp'][data['Echo'],data['EPITrain'],data['Partition'],data['Slice'],data['Acquisition'],data['Phase'],data['Repetition']] = (0.0025*timeS).astype(np.complex64)\n\n if (readNavigator is True) and (data['Channel'] == 0) and (navigatorDataON is True):\n data['EPITrain'] = countNavigator % data['nEPITrain']\n kNavigator[:,:,:,data['EPITrain'],data['Partition'],data['Slice'],data['Acquisition'],data['Phase'],data[Repetition]] = kNavigatorTemp.astype(np.complex64)\n navigatorDataON = False\n countNavigator += 1\n\n if 1 in flag:\n break\n\n data['kSpace'] = np.squeeze(data['kSpace'])\n\n if len(data['kSpace'].shape) == 3:\n data['kSpace'] = np.transpose(data['kSpace'],[1,2,0])\n elif len(data['kSpace'].shape) == 4:\n if data['flag3D'] is False:\n data['kSpace'] = np.transpose(data['kSpace'],[2,3,0,1])\n else:\n data['kSpace'] = np.transpose(data['kSpace'],[2,3,1,0])\n elif len(data['kSpace'].shape) == 5:\n data['kSpace'] = np.transpose(data['kSpace'],[3,4,2,0,1])\n\n if transformToImageSpace is True:\n if data['flag3D'] is True:\n data['imSpace'] = np.fft.ifftshift(np.fft.ifftn(np.fft.fftshift(data['kSpace'],axes=(0,1,2)),axes=(0,1,2)),axes=(0,1,2))\n data['imSpace'][2] *= data['Nz']\n else:\n data['imSpace'] = np.fft.ifftshift(np.fft.ifft2(np.fft.fftshift(data['kSpace'],axes=(0,1)),axes=(0,1)),axes=(0,1))\n\n data['imSpace'][0] *= data['NxOS']\n data['imSpace'][1] *= data['Ny']\n\n if (removeOSafter is True) and (removeOS is False):\n if len(data['imSpace'].shape) == 2:\n data['imSpace'][0:data['NxOS']/4,:] = []\n data['imSpace'][data['imSpace'].shape[0] - data['NxOS']/4 :,:] = []\n elif len(data['imSpace'].shape) == 3:\n data['imSpace'][0:data['NxOS']/4,:,:] = []\n data['imSpace'][data['imSpace'].shape[0] - data['NxOS']/4:,:,:] = []\n elif len(data['imSpace'].shape) == 4:\n data['imSpace'][0:data['NxOS']/4,:,:,:] = []\n data['imSpace'][data['imSpace'].shape[0] - data['NxOS']/4:,:,:,:] = []\n elif len(data['imSpace'].shape) == 5:\n data['imSpace'][0:data['NxOS']/4,:,:,:,:] = []\n data['imSpace'][data['imSpace'].shape[0] - data['NxOS']/4:,:,:,:,:] = []\n\n if writeToFile is True:\n if transformToImageSpace is True:\n if npz:\n np.savez_compressed(filenameOut,imSpace=data['imSpace'],timeStamp=data['timeStamp'])\n else:\n with h5py.File('%s.hdf5' % filenameOut,'w') as f:\n dset = f.create_dataset('kSpace',data=data['imSpace'])\n else:\n if npz:\n np.savez_compressed(filenameOut,kSpace=data['kSpace'],timeStamp=data['timeStamp'])\n else:\n with h5py.File('%s.hdf5' % filenameOut,'w') as f:\n dset = f.create_dataset('kSpace',data=data['kSpace'])\n\n return(data)"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nfunctions run when called from command line.", "response": "def main(args):\n '''Function run when called from command line.'''\n\n options = { '-rfft': ['resetFFTscale',False],\n '-r1': ['readOneCoil',False],\n '-rp': ['readPhaseCorInfo',False],\n '-rn': ['readNavigator',False],\n '-skipts': ['readTimeStamp',True],\n '-nnavek': ['nNavEK',False],\n '-ros': ['removeOS',False],\n '-rosa': ['removeOSafter',False],\n '-I': ['transformToImageSpace',False],\n '-w': ['writeToFile',False],\n '-npz': ['npz',False] }\n\n decode_simple_opts(options,args,readMeasDataVB15)"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\ngetting json keys from directory structure", "response": "def _get_keys_folder(jdir, key_path=None, in_memory=True,\n ignore_prefix=('.', '_')):\n \"\"\" get json keys from directory structure\n\n e.g.\n\n jdir\n sub_dir1\n data1.json\n data2.json\n sub_dir2\n data.json\n\n _get_keys_folder(jdir)\n => ['sub_dir1', 'sub_dir2']\n _get_keys_folder(jdir,['sub_dir1'])\n => ['data1', 'data2']\n\n NB: json files are identified with .json extension\n files/directories beginning with '.' are ignored\n\n \"\"\"\n if not hasattr(jdir, 'iterdir'):\n raise ValueError('jdir is not a path object; {}'.format(jdir))\n\n key_path = [] if key_path is None else key_path\n\n keys = []\n\n key_found = False if key_path else True\n search_key = key_path[0] if len(key_path) > 0 else None\n\n for jsub in jdir.iterdir():\n if jsub.is_file() and jsub.name[-5:] == '.json':\n\n name, ext = os.path.splitext(jsub.name)\n if name == search_key or not key_path:\n key_found = True\n if key_path:\n return jkeys(jsub, key_path[1:], in_memory, ignore_prefix)\n else:\n keys.append(name)\n\n elif (jsub.is_dir()\n and not jsub.name.startswith(ignore_prefix)\n and (jsub.name == search_key or not key_path)):\n\n key_found = True\n if jsub.name in keys:\n raise IOError(\n 'directory has a sub-dir and file with same name: '\n '{1} and {1}.json in {0}'.format(jdir, jsub.name))\n if key_path:\n return jkeys(jsub, key_path[1:], in_memory, ignore_prefix)\n else:\n keys.append(jsub.name)\n\n if not key_found:\n raise KeyError('key not found: {0}'.format(search_key))\n\n return sorted(keys)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\ngetting keys for initial json level or at level after following key_path", "response": "def jkeys(jfile, key_path=None, in_memory=True, ignore_prefix=('.', '_')):\n \"\"\" get keys for initial json level, or at level after following key_path\n\n Parameters\n ----------\n jfile : str, file_like or path_like\n if str, must be existing file or folder,\n if file_like, must have 'read' method\n if path_like, must have 'iterdir' method (see pathlib.Path)\n key_path : list[str]\n a list of keys to index into the json before returning keys\n in_memory : bool\n if true reads json into memory before finding keys\n (this is faster but uses more memory)\n ignore_prefix : list[str]\n ignore folders beginning with these prefixes\n\n Examples\n --------\n\n >>> from jsonextended.utils import MockPath\n >>> file_obj = MockPath('test.json',is_file=True,\n ... content='''\n ... {\n ... \"a\": 1,\n ... \"b\": [1.1,2.1],\n ... \"c\": {\"d\":\"e\",\"f\":\"g\"}\n ... }\n ... ''')\n ...\n >>> jkeys(file_obj)\n ['a', 'b', 'c']\n\n >>> jkeys(file_obj,[\"c\"])\n ['d', 'f']\n\n >>> from jsonextended.utils import get_test_path\n >>> path = get_test_path()\n >>> jkeys(path)\n ['dir1', 'dir2', 'dir3']\n\n >>> path = get_test_path()\n >>> jkeys(path, ['dir1','file1'], in_memory=True)\n ['initial', 'meta', 'optimised', 'units']\n\n \"\"\"\n key_path = [] if key_path is None else key_path\n\n def eval_file(file_obj):\n if not in_memory:\n return _get_keys_ijson(file_obj, key_path)\n else:\n return _get_keys(file_obj, key_path)\n\n if isinstance(jfile, basestring):\n if not os.path.exists(jfile):\n raise IOError('jfile does not exist: {}'.format(jfile))\n if os.path.isdir(jfile):\n jpath = pathlib.Path(jfile)\n return _get_keys_folder(jpath, key_path, in_memory, ignore_prefix)\n else:\n with open(jfile, 'r') as file_obj:\n return eval_file(file_obj)\n elif hasattr(jfile, 'read'):\n return eval_file(jfile)\n elif hasattr(jfile, 'iterdir'):\n if jfile.is_file():\n with jfile.open('r') as file_obj:\n return eval_file(file_obj)\n else:\n return _get_keys_folder(jfile, key_path, in_memory, ignore_prefix)\n else:\n raise ValueError(\n 'jfile should be a str, '\n 'file_like or path_like object: {}'.format(jfile))"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nreads json with keys", "response": "def _file_with_keys(file_obj, key_path=None, parse_decimal=False):\n \"\"\"read json with keys\n\n Parameters\n ----------\n file_obj : object\n object with read method\n key_path : list[str]\n key to index befor parsing\n parse_decimal : bool\n whether to parse numbers as Decimal instances (retains exact precision)\n\n Notes\n -----\n ijson outputs decimals as Decimal class (for arbitrary precision)\n\n \"\"\"\n key_path = [] if key_path is None else key_path\n\n try:\n objs = ijson.items(file_obj, '.'.join(key_path))\n except NameError:\n warnings.warn('ijson package not found in environment, \\\n please install for on-disk key indexing', ImportWarning)\n data = json.load(\n file_obj, parse_float=Decimal if parse_decimal else float,\n object_hook=decode)\n return indexes(data, key_path)\n try:\n data = next(objs) # .next()\n except StopIteration:\n raise KeyError('key path not available in json: {}'.format(key_path))\n\n # by default ijson parses Decimal values\n if not parse_decimal:\n convert_type(data, Decimal, float, in_place=True)\n\n datastr = json.dumps(data)\n data = json.loads(datastr, object_hook=decode)\n\n return data"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nread in folder structure as json", "response": "def _folder_to_json(jdir, key_path=None, in_memory=True,\n ignore_prefix=('.', '_'), dic={}, parse_decimal=False):\n \"\"\" read in folder structure as json\n\n e.g.\n\n jdir\n sub_dir1\n data.json\n sub_dir2\n data.json\n\n _folder_to_json(jdir)\n => {'sub_dir1':{'data':{...}},\n 'sub_dir2':{'data':{...}}}\n\n NB: json files are identified with .json extension\n\n \"\"\"\n key_path = [] if key_path is None else key_path\n\n if not hasattr(jdir, 'iterdir'):\n raise ValueError('jdir is not a path object; {}'.format(jdir))\n\n key_found = False if key_path else True\n search_key = key_path[0] if len(key_path) > 0 else None\n\n for jsub in jdir.iterdir():\n if jsub.is_file() and jsub.name.endswith('.json'):\n\n name, ext = os.path.splitext(jsub.name)\n if name == search_key or not key_path:\n key_found = True\n if key_path:\n data = to_dict(jsub, key_path[1:], in_memory,\n ignore_prefix, parse_decimal)\n if isinstance(data, dict):\n dic.update(data)\n else:\n dic.update({_Terminus(): data})\n else:\n dic[name] = to_dict(jsub, key_path[1:], in_memory,\n ignore_prefix, parse_decimal)\n\n elif (jsub.is_dir()\n and not jsub.name.startswith(ignore_prefix)\n and (jsub.name == search_key or not key_path)):\n\n key_found = True\n if jsub.name in dic.keys():\n raise IOError(\n 'directory has a sub-dir and file with same name: '\n '{1} and {1}.json in {0}'.format(jdir, jsub.name))\n if key_path:\n sub_d = dic\n else:\n dic[jsub.name] = {}\n sub_d = dic[jsub.name]\n _folder_to_json(jsub, key_path[1:], in_memory, ignore_prefix,\n sub_d, parse_decimal)\n\n if not key_found:\n raise KeyError('key not found: {0}'.format(search_key))"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\ninput json to dict", "response": "def to_dict(jfile, key_path=None, in_memory=True,\n ignore_prefix=('.', '_'), parse_decimal=False):\n \"\"\" input json to dict\n\n Parameters\n ----------\n jfile : str, file_like or path_like\n if str, must be existing file or folder,\n if file_like, must have 'read' method\n if path_like, must have 'iterdir' method (see pathlib.Path)\n key_path : list[str]\n a list of keys to index into the json before parsing it\n in_memory : bool\n if true reads full json into memory before filtering keys\n (this is faster but uses more memory)\n ignore_prefix : list[str]\n ignore folders beginning with these prefixes\n parse_decimal : bool\n whether to parse numbers as Decimal instances (retains exact precision)\n\n Examples\n --------\n\n >>> from pprint import pformat\n\n >>> from jsonextended.utils import MockPath\n >>> file_obj = MockPath('test.json',is_file=True,\n ... content='''\n ... {\n ... \"a\": 1,\n ... \"b\": [1.1,2.1],\n ... \"c\": {\"d\":\"e\"}\n ... }\n ... ''')\n ...\n\n >>> dstr = pformat(to_dict(file_obj))\n >>> print(dstr.replace(\"u'\",\"'\"))\n {'a': 1, 'b': [1.1, 2.1], 'c': {'d': 'e'}}\n\n >>> dstr = pformat(to_dict(file_obj,parse_decimal=True))\n >>> print(dstr.replace(\"u'\",\"'\"))\n {'a': 1, 'b': [Decimal('1.1'), Decimal('2.1')], 'c': {'d': 'e'}}\n\n >>> str(to_dict(file_obj,[\"c\",\"d\"]))\n 'e'\n\n >>> from jsonextended.utils import get_test_path\n >>> path = get_test_path()\n >>> jdict1 = to_dict(path)\n >>> pprint(jdict1,depth=2)\n dir1:\n dir1_1: {...}\n file1: {...}\n file2: {...}\n dir2:\n file1: {...}\n dir3:\n\n >>> jdict2 = to_dict(path,['dir1','file1','initial'],in_memory=False)\n >>> pprint(jdict2,depth=1)\n crystallographic: {...}\n primitive: {...}\n\n \"\"\"\n key_path = [] if key_path is None else key_path\n\n if isinstance(jfile, basestring):\n if not os.path.exists(jfile):\n raise IOError('jfile does not exist: {}'.format(jfile))\n if os.path.isdir(jfile):\n data = {}\n jpath = pathlib.Path(jfile)\n _folder_to_json(jpath, key_path[:], in_memory, ignore_prefix,\n data, parse_decimal)\n if isinstance(list(data.keys())[0], _Terminus):\n data = data.values()[0]\n else:\n with open(jfile, 'r') as file_obj:\n if key_path and not in_memory:\n data = _file_with_keys(file_obj, key_path, parse_decimal)\n elif key_path:\n data = json.load(\n file_obj, object_hook=decode,\n parse_float=Decimal if parse_decimal else float)\n data = indexes(data, key_path)\n else:\n data = json.load(\n file_obj, object_hook=decode,\n parse_float=Decimal if parse_decimal else float)\n elif hasattr(jfile, 'read'):\n if key_path and not in_memory:\n data = _file_with_keys(jfile, key_path, parse_decimal)\n elif key_path:\n data = json.load(\n jfile, object_hook=decode,\n parse_float=Decimal if parse_decimal else float)\n data = indexes(data, key_path)\n else:\n data = json.load(\n jfile, object_hook=decode,\n parse_float=Decimal if parse_decimal else float)\n elif hasattr(jfile, 'iterdir'):\n if jfile.is_file():\n with jfile.open() as file_obj:\n if key_path and not in_memory:\n data = _file_with_keys(file_obj, key_path, parse_decimal)\n elif key_path:\n data = json.load(\n file_obj, object_hook=decode,\n parse_float=Decimal if parse_decimal else float)\n data = indexes(data, key_path)\n else:\n data = json.load(\n file_obj, object_hook=decode,\n parse_float=Decimal if parse_decimal else float)\n else:\n data = {}\n _folder_to_json(jfile, key_path[:], in_memory, ignore_prefix,\n data, parse_decimal)\n if isinstance(list(data.keys())[0], _Terminus):\n data = data.values()[0]\n else:\n raise ValueError(\n 'jfile should be a str, '\n 'file_like or path_like object: {}'.format(jfile))\n\n return data"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nturn generator sequence into sequence of chunks.", "response": "def chunks(items, chunksize):\n \"\"\"Turn generator sequence into sequence of chunks.\"\"\"\n\n items = iter(items)\n for first in items:\n chunk = chain((first,), islice(items, chunksize - 1))\n yield chunk\n deque(chunk, 0)"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef unionfs(rw='rw', ro=None, union='union'):\n from functools import wraps\n\n def wrap_in_union_fs(func):\n \"\"\"\n Function that wraps a given function inside the file system.\n\n Args:\n func: The function that needs to be wrapped inside the unions fs.\n Return:\n The file system with the function wrapped inside.\n \"\"\"\n\n @wraps(func)\n def wrap_in_union_fs_func(project, *args, **kwargs):\n \"\"\"\n Wrap the func in the UnionFS mount stack.\n\n We make sure that the mount points all exist and stack up the\n directories for the unionfs. All directories outside of the default\n build environment are tracked for deletion.\n \"\"\"\n container = project.container\n if container is None or in_container():\n return func(project, *args, **kwargs)\n\n build_dir = local.path(project.builddir)\n LOG.debug(\"UnionFS - Project builddir: %s\", project.builddir)\n if __unionfs_is_active(root=build_dir):\n LOG.debug(\n \"UnionFS already active in %s, nesting not supported.\",\n build_dir)\n return func(project, *args, **kwargs)\n\n ro_dir = local.path(container.local)\n rw_dir = build_dir / rw\n un_dir = build_dir / union\n LOG.debug(\"UnionFS - RW: %s\", rw_dir)\n\n unionfs_cmd = __unionfs_set_up(ro_dir, rw_dir, un_dir)\n project_builddir_bak = project.builddir\n project.builddir = un_dir\n\n proc = unionfs_cmd.popen()\n while (not __unionfs_is_active(root=un_dir)) and \\\n (proc.poll() is None):\n pass\n\n ret = None\n if proc.poll() is None:\n try:\n with local.cwd(un_dir):\n ret = func(project, *args, **kwargs)\n finally:\n project.builddir = project_builddir_bak\n\n from signal import SIGINT\n is_running = proc.poll() is None\n while __unionfs_is_active(root=un_dir) and is_running:\n try:\n proc.send_signal(SIGINT)\n proc.wait(timeout=3)\n except subprocess.TimeoutExpired:\n proc.kill()\n is_running = False\n LOG.debug(\"Unionfs shut down.\")\n\n if __unionfs_is_active(root=un_dir):\n raise UnmountError()\n\n return ret\n\n return wrap_in_union_fs_func\n\n return wrap_in_union_fs", "response": "This function is used to configure the UnionsFS feature."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef __update_cleanup_paths(new_path):\n cleanup_dirs = settings.CFG[\"cleanup_paths\"].value\n cleanup_dirs = set(cleanup_dirs)\n cleanup_dirs.add(new_path)\n cleanup_dirs = list(cleanup_dirs)\n settings.CFG[\"cleanup_paths\"] = cleanup_dirs", "response": "Updates the list of paths to clean up afterwards."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nchecking if a project lies outside of its expected directory.", "response": "def __is_outside_of_builddir(project, path_to_check):\n \"\"\"Check if a project lies outside of its expected directory.\"\"\"\n bdir = project.builddir\n cprefix = os.path.commonprefix([path_to_check, bdir])\n return cprefix != bdir"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef __unionfs_set_up(ro_dir, rw_dir, mount_dir):\n mount_dir.mkdir()\n rw_dir.mkdir()\n if not ro_dir.exists():\n LOG.error(\"Base dir does not exist: '%s'\", ro_dir)\n raise ValueError(\"Base directory does not exist\")\n\n from benchbuild.utils.cmd import unionfs as unionfs_cmd\n LOG.debug(\"Mounting UnionFS on %s with RO:%s RW:%s\", mount_dir, ro_dir,\n rw_dir)\n return unionfs_cmd[\"-f\", \"-o\", \"auto_unmount,allow_other,cow\", rw_dir +\n \"=RW:\" + ro_dir + \"=RO\", mount_dir]", "response": "Setup a unionfs via unionfs - fuse."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef get_precision_regex():\n expr = re.escape(PRECISION_FORMULA)\n expr += r'=\\s*(\\S*)\\s.*\\s([A-Z]*)'\n return re.compile(expr)", "response": "Build a regular expression used to extract precision\n metric from command output"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nbuilds HPL data from basic parameters", "response": "def _build_data(self):\n \"\"\"Build HPL data from basic parameters\"\"\"\n\n def baseN(nodes, mpn):\n return int(math.sqrt(mpn * 0.80 * nodes * 1024 * 1024 / 8))\n\n def nFromNb(baseN, nb):\n factor = int(baseN / nb)\n if factor % 2 != 0:\n factor -= 1\n return nb * factor\n\n def get_grid(nodes, ppn):\n cores = nodes * ppn\n sqrt = math.sqrt(cores)\n factors = [\n num for num in range(2, int(math.floor(sqrt) + 1)) if cores % num == 0\n ]\n if len(factors) == 0:\n factors = [1]\n\n diff = 0\n keep = 0\n for factor in factors:\n if diff == 0:\n diff = cores - factor\n if keep == 0:\n keep = factor\n tmp_diff = cores - factor\n if tmp_diff < diff:\n diff = tmp_diff\n keep = factor\n return [keep, int(cores / keep)]\n\n properties = dict(\n realN=nFromNb(baseN(self.nodes, self.memory_per_node), self.block_size),\n nb=self.block_size,\n pQ=get_grid(self.nodes, self.cores_per_node),\n )\n return self._data_from_jinja(**properties)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef mpirun(self):\n cmd = self.attributes['mpirun']\n if cmd and cmd[0] != 'mpirun':\n cmd = ['mpirun']\n return [str(e) for e in cmd]", "response": "Additional options passed as a list to the mpirun command"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nperforms variable substitution on the given string s.", "response": "def expandvars(s, vars=None):\n \"\"\"Perform variable substitution on the given string\n\n Supported syntax:\n * $VARIABLE\n * ${VARIABLE}\n * ${#VARIABLE}\n * ${VARIABLE:-default}\n\n :param s: message to expand\n :type s: str\n\n :param vars: dictionary of variables. Default is ``os.environ``\n :type vars: dict\n\n :return: expanded string\n :rtype: str\n \"\"\"\n tpl = TemplateWithDefaults(s)\n return tpl.substitute(vars or os.environ)"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nqueries an object and return True if all kwargs are True False otherwise.", "response": "def and_(cls, obj, **kwargs):\n \"\"\"Query an object\n\n :param obj:\n object to test\n\n :param kwargs: query specified in kwargssql\n\n :return:\n `True` if all `kwargs` expression are `True`, `False` otherwise.\n :rtype: bool\n \"\"\"\n return cls.__eval_seqexp(obj, operator.and_, **kwargs)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef or_(cls, obj, **kwargs):\n return cls.__eval_seqexp(obj, operator.or_, **kwargs)", "response": "Query an object\n\n :param obj:\n object to test\n\n :param kwargs: query specified in kwargssql\n\n :return:\n `True` if at leat one `kwargs` expression is `True`,\n `False` otherwise.\n :rtype: bool"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef xor(cls, obj, **kwargs):\n return cls.__eval_seqexp(obj, operator.xor, **kwargs)", "response": "Query an object.\n returns True if exactly one kwargssql\n is True otherwise False."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef __resolve_path(cls, obj, path):\n path = [p for p in path if p]\n if any(path):\n pathes = len(path)\n i = 0\n while i < pathes:\n # _get_obj_attr can supersede `i` because it might\n # evaluate the entire expression by itself.\n obj, i = cls._get_obj_attr(obj, path, i)\n i += 1\n else:\n raise Exception(\"Nothing to do\")\n return obj", "response": "Follow a kwargsql expression starting from a given object\n and return the deduced object."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef _sequence_map(cls, seq, path):\n if not any(path):\n # There is no further kwargsql expression\n return seq\n result = []\n for item in seq:\n try:\n result.append(cls.__resolve_path(item, path))\n except (KeyError, IndexError):\n pass\n return result", "response": "Apply a kwargsql expression to every item of a sequence and returns it."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nreturn a function that returns True if the given operation is not met.", "response": "def _not(cls, operation):\n \"\"\"not operation\"\"\"\n\n def _wrap(*args, **kwargs):\n return not operation(*args, **kwargs)\n\n return _wrap"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef populate(projects_to_filter=None, group=None):\n if projects_to_filter is None:\n projects_to_filter = []\n\n import benchbuild.projects as all_projects\n all_projects.discover()\n\n prjs = ProjectRegistry.projects\n if projects_to_filter:\n prjs = {}\n for filter_project in set(projects_to_filter):\n try:\n prjs.update({\n x: y\n for x, y in ProjectRegistry.projects.items(\n prefix=filter_project)\n })\n except KeyError:\n pass\n\n if group:\n groupkeys = set(group)\n prjs = {\n name: cls\n for name, cls in prjs.items() if cls.GROUP in groupkeys\n }\n\n return {\n x: prjs[x]\n for x in prjs if prjs[x].DOMAIN != \"debug\" or x in projects_to_filter\n }", "response": "Populate the list of projects that belong to this experiment."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef cc(project, detect_project=False):\n from benchbuild.utils import cmd\n\n cc_name = str(CFG[\"compiler\"][\"c\"])\n wrap_cc(cc_name, compiler(cc_name), project, detect_project=detect_project)\n return cmd[\"./{}\".format(cc_name)]", "response": "Returns a clang that hides CFLAGS and LDFLAGS and returns a complete plumbum command to it."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nreturn a clang ++ that hides CFLAGS and LDFLAGS and returns a complete plumbum command to it.", "response": "def cxx(project, detect_project=False):\n \"\"\"\n Return a clang++ that hides CFLAGS and LDFLAGS.\n\n This will generate a wrapper script in the current directory\n and return a complete plumbum command to it.\n\n Args:\n cflags: The CFLAGS we want to hide.\n ldflags: The LDFLAGS we want to hide.\n func (optional): A function that will be pickled alongside the compiler.\n It will be called before the actual compilation took place. This\n way you can intercept the compilation process with arbitrary python\n code.\n\n Returns (benchbuild.utils.cmd):\n Path to the new clang command.\n \"\"\"\n from benchbuild.utils import cmd\n\n cxx_name = str(CFG[\"compiler\"][\"cxx\"])\n wrap_cc(\n cxx_name, compiler(cxx_name), project, detect_project=detect_project)\n return cmd[\"./{name}\".format(name=cxx_name)]"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef compiler(name):\n pinfo = __get_paths()\n _compiler = local[name]\n _compiler = _compiler.setenv(\n PATH=pinfo[\"path\"], LD_LIBRARY_PATH=pinfo[\"ld_library_path\"])\n return _compiler", "response": "Returns a usable clang ++ plumbum command that executes the specified clang ++ library"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef strip_path_prefix(ipath, prefix):\n if prefix is None:\n return ipath\n\n return ipath[len(prefix):] if ipath.startswith(prefix) else ipath", "response": "Strip prefix from path."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef unpickle(pickle_file):\n pickle = None\n with open(pickle_file, \"rb\") as pickle_f:\n pickle = dill.load(pickle_f)\n if not pickle:\n LOG.error(\"Could not load python object from file\")\n return pickle", "response": "Unpickle a python object from the given path."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nwraps the binary with the runtime extension of the project.", "response": "def wrap(name, project, sprefix=None, python=sys.executable):\n \"\"\" Wrap the binary :name: with the runtime extension of the project.\n\n This module generates a python tool that replaces :name:\n The function in runner only accepts the replaced binaries\n name as argument. We use the cloudpickle package to\n perform the serialization, make sure :runner: can be serialized\n with it and you're fine.\n\n Args:\n name: Binary we want to wrap\n project: The project that contains the runtime_extension we want\n to run instead of the binary.\n\n Returns:\n A plumbum command, ready to launch.\n \"\"\"\n env = __create_jinja_env()\n template = env.get_template('run_static.py.inc')\n\n name_absolute = os.path.abspath(name)\n real_f = name_absolute + PROJECT_BIN_F_EXT\n if sprefix:\n run(uchroot()[\"/bin/mv\",\n strip_path_prefix(name_absolute, sprefix),\n strip_path_prefix(real_f, sprefix)])\n else:\n run(mv[name_absolute, real_f])\n\n project_file = persist(project, suffix=\".project\")\n\n env = CFG['env'].value\n\n bin_path = list_to_path(env.get('PATH', []))\n bin_path = list_to_path([bin_path, os.environ[\"PATH\"]])\n\n bin_lib_path = list_to_path(env.get('LD_LIBRARY_PATH', []))\n bin_lib_path = list_to_path([bin_lib_path, os.environ[\"LD_LIBRARY_PATH\"]])\n\n with open(name_absolute, 'w') as wrapper:\n wrapper.write(\n template.render(\n runf=strip_path_prefix(real_f, sprefix),\n project_file=strip_path_prefix(project_file, sprefix),\n path=str(bin_path),\n ld_library_path=str(bin_lib_path),\n python=python,\n ))\n\n run(chmod[\"+x\", name_absolute])\n return local[name_absolute]"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef wrap_cc(filepath,\n compiler,\n project,\n python=sys.executable,\n detect_project=False):\n \"\"\"\n Substitute a compiler with a script that hides CFLAGS & LDFLAGS.\n\n This will generate a wrapper script in the current directory\n and return a complete plumbum command to it.\n\n Args:\n filepath (str): Path to the wrapper script.\n compiler (benchbuild.utils.cmd):\n Real compiler command we should call in the script.\n project (benchbuild.project.Project):\n The project this compiler will be for.\n python (str): Path to the python interpreter we should use.\n detect_project: Should we enable project detection or not.\n\n Returns (benchbuild.utils.cmd):\n Command of the new compiler we can call.\n \"\"\"\n env = __create_jinja_env()\n template = env.get_template('run_compiler.py.inc')\n\n cc_fname = local.path(filepath).with_suffix(\".benchbuild.cc\", depth=0)\n cc_f = persist(compiler, filename=cc_fname)\n\n project_file = persist(project, suffix=\".project\")\n\n with open(filepath, 'w') as wrapper:\n wrapper.write(\n template.render(\n cc_f=cc_f,\n project_file=project_file,\n python=python,\n detect_project=detect_project))\n\n chmod(\"+x\", filepath)\n LOG.debug(\"Placed wrapper in: %s for compiler %s\", local.path(filepath),\n str(compiler))\n LOG.debug(\"Placed project in: %s\", local.path(project_file))\n LOG.debug(\"Placed compiler command in: %s\", local.path(cc_f))\n return local[filepath]", "response": "Wrap a C ++ compiler command in a new file."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef persist(id_obj, filename=None, suffix=None):\n if suffix is None:\n suffix = \".pickle\"\n if hasattr(id_obj, 'id'):\n ident = id_obj.id\n else:\n ident = str(id(id_obj))\n\n if filename is None:\n filename = \"{obj_id}{suffix}\".format(obj_id=ident, suffix=suffix)\n\n with open(filename, 'wb') as obj_file:\n dill.dump(id_obj, obj_file)\n return os.path.abspath(filename)", "response": "Persist an object in the filesystem."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef load(filename):\n if not os.path.exists(filename):\n LOG.error(\"load object - File '%s' does not exist.\", filename)\n return None\n\n obj = None\n with open(filename, 'rb') as obj_file:\n obj = dill.load(obj_file)\n return obj", "response": "Load a pickled object from the filesystem."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef extend_shapes(df_shapes, axis, distance):\n '''\n Extend shape/polygon outline away from polygon center point by absolute\n distance.\n '''\n df_shapes_i = df_shapes.copy()\n offsets = df_shapes_i[axis + '_center_offset'].copy()\n offsets[offsets < 0] -= distance\n offsets[offsets >= 0] += distance\n df_shapes_i[axis] = df_shapes_i[axis + '_center'] + offsets\n return df_shapes_i", "response": "Extend shape or polygon outline away from polygon center point by absolute distance."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nreturning a list of lists of adjacent electrodes in a single table.", "response": "def extract_adjacent_shapes(df_shapes, shape_i_column, extend=.5):\n '''\n Generate list of connections between \"adjacent\" polygon shapes based on\n geometrical \"closeness\".\n\n Parameters\n ----------\n df_shapes : pandas.DataFrame\n Table of polygon shape vertices (one row per vertex).\n\n Table rows with the same value in the :data:`shape_i_column` column\n are grouped together as a polygon.\n shape_i_column : str or list[str]\n Column name(s) that identify the polygon each row belongs to.\n extend : float, optional\n Extend ``x``/``y`` coords by the specified number of absolute units\n from the center point of each polygon.\n Each polygon is stretched independently in the ``x`` and ``y`` direction.\n In each direction, a polygon considered adjacent to all polygons that\n are overlapped by the extended shape.\n\n Returns\n -------\n pandas.DataFrame\n Adjacency list as a frame containing the columns ``source`` and\n ``target``.\n\n The ``source`` and ``target`` of each adjacency connection is ordered\n such that the ``source`` is less than the ``target``.\n '''\n # Find corners of each solid shape outline.\n # Extend x coords by abs units\n df_scaled_x = extend_shapes(df_shapes, 'x', extend)\n # Extend y coords by abs units\n df_scaled_y = extend_shapes(df_shapes, 'y', extend)\n\n df_corners = df_shapes.groupby(shape_i_column).agg({'x': ['min', 'max'],\n 'y': ['min', 'max']})\n\n # Find adjacent electrodes\n row_list = []\n\n for shapeNumber in df_shapes[shape_i_column].drop_duplicates():\n df_stretched = df_scaled_x[df_scaled_x[shape_i_column]\n .isin([shapeNumber])]\n xmin_x, xmax_x, ymin_x, ymax_x = (df_stretched.x.min(),\n df_stretched.x.max(),\n df_stretched.y.min(),\n df_stretched.y.max())\n df_stretched = df_scaled_y[df_scaled_y[shape_i_column]\n .isin([shapeNumber])]\n xmin_y, xmax_y, ymin_y, ymax_y = (df_stretched.x.min(),\n df_stretched.x.max(),\n df_stretched.y.min(),\n df_stretched.y.max())\n\n #Some conditions unnecessary if it is assumed that electrodes don't overlap\n adjacent = df_corners[\n ((df_corners.x['min'] < xmax_x) & (df_corners.x['max'] >= xmax_x)\n # Check in x stretched direction\n |(df_corners.x['min'] < xmin_x) & (df_corners.x['max'] >= xmin_x))\n # Check if y is within bounds\n & (df_corners.y['min'] < ymax_x) & (df_corners.y['max'] > ymin_x) |\n\n #maybe do ymax_x - df_corners.y['min'] > threshold &\n # df_corners.y['max'] - ymin_x > threshold\n\n ((df_corners.y['min'] < ymax_y) & (df_corners.y['max'] >= ymax_y)\n # Checks in y stretched direction\n |(df_corners.y['min'] < ymin_y) & (df_corners.y['max'] >= ymin_y))\n # Check if x in within bounds\n & ((df_corners.x['min'] < xmax_y) & (df_corners.x['max'] > xmin_y))\n ].index.values\n\n for shape in adjacent:\n temp_dict = {}\n reverse_dict = {}\n\n temp_dict ['source'] = shapeNumber\n reverse_dict['source'] = shape\n temp_dict ['target'] = shape\n reverse_dict['target'] = shapeNumber\n\n if(reverse_dict not in row_list):\n row_list.append(temp_dict)\n\n df_connected = (pd.DataFrame(row_list)[['source', 'target']]\n .sort_index(axis=1, ascending=True)\n .sort_values(['source', 'target']))\n return df_connected"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef extract_connections(svg_source, shapes_canvas, line_layer='Connections',\n line_xpath=None, path_xpath=None, namespaces=None):\n '''\n Load all ```` elements and ```` elements from a layer\n of an SVG source. For each element, if endpoints overlap distinct shapes\n in :data:`shapes_canvas`, add connection between overlapped shapes.\n\n .. versionchanged:: 0.6.post1\n Allow both ```` *and* ```` instances to denote\n connected/adjacent shapes.\n\n .. versionadded:: 0.6.post1\n :data:`path_xpath`\n\n Parameters\n ----------\n svg_source : filepath\n Input SVG file containing connection lines.\n shapes_canvas : shapes_canvas.ShapesCanvas\n Shapes canvas containing shapes to compare against connection\n endpoints.\n line_layer : str\n Name of layer in SVG containing connection lines.\n line_xpath : str\n XPath string to iterate through connection lines.\n path_xpath : str\n XPath string to iterate through connection paths.\n namespaces : dict\n SVG namespaces (compatible with :func:`etree.parse`).\n\n Returns\n -------\n pandas.DataFrame\n Each row corresponds to connection between two shapes in\n :data:`shapes_canvas`, denoted ``source`` and ``target``.\n '''\n from lxml import etree\n\n if namespaces is None:\n # Inkscape namespace is required to match SVG elements as well as\n # Inkscape-specific SVG tags and attributes (e.g., `inkscape:label`).\n namespaces = INKSCAPE_NSMAP\n\n # Parse SVG source.\n e_root = etree.parse(svg_source)\n # List to hold records of form: `[, , , , ]`.\n frames = []\n\n if line_xpath is None:\n # Define query to look for `svg:line` elements in top level of layer of\n # SVG specified to contain connections.\n line_xpath = (\"//svg:g[@inkscape:label='%s']/svg:line\" % line_layer)\n coords_columns = ['x1', 'y1', 'x2', 'y2']\n\n for line_i in e_root.xpath(line_xpath, namespaces=namespaces):\n # Extract start and end coordinate from `svg:line` element.\n line_i_dict = dict(list(line_i.items()))\n values = ([line_i_dict.get('id', None)] +\n [float(line_i_dict[k]) for k in coords_columns])\n # Append record for end points of current line.\n frames.append(values)\n\n # Regular expression pattern to match start and end coordinates of\n # connection `svg:path` element.\n cre_path_ends = re.compile(r'^\\s*M\\s*(?P\\d+(\\.\\d+)?),\\s*'\n r'(?P\\d+(\\.\\d+)?).*'\n # Diagonal line...\n r'((L\\s*(?P\\d+(\\.\\d+)?),\\s*'\n r'(?P\\d+(\\.\\d+)?))|'\n # or Vertical line...\n r'(V\\s*(?P\\d+(\\.\\d+)?))|'\n # or Horizontal line\n r'(H\\s*(?P\\d+(\\.\\d+)?))'\n r')\\D*'\n r'$')\n\n if path_xpath is None:\n # Define query to look for `svg:path` elements in top level of layer of\n # SVG specified to contain connections.\n path_xpath = (\"//svg:g[@inkscape:label='%s']/svg:path\" % line_layer)\n\n for path_i in e_root.xpath(path_xpath, namespaces=namespaces):\n path_i_dict = dict(list(path_i.items()))\n match_i = cre_path_ends.match(path_i_dict['d'])\n if match_i:\n # Connection `svg:path` matched required format. Extract start and\n # end coordinates.\n match_dict_i = match_i.groupdict()\n if match_dict_i['end_vy']:\n # Path ended with vertical line\n match_dict_i['end_x'] = match_dict_i['start_x']\n match_dict_i['end_y'] = match_dict_i['end_vy']\n if match_dict_i['end_hx']:\n # Path ended with horizontal line\n match_dict_i['end_x'] = match_dict_i['end_hx']\n match_dict_i['end_y'] = match_dict_i['start_y']\n # Append record for end points of current path.\n frames.append([path_i_dict['id']] + list(map(float,\n (match_dict_i['start_x'],\n match_dict_i['start_y'],\n match_dict_i['end_x'],\n match_dict_i['end_y']))))\n\n if not frames:\n return pd.DataFrame(None, columns=['source', 'target'])\n\n df_connection_lines = pd.DataFrame(frames, columns=['id'] + coords_columns)\n\n # Use `shapes_canvas.find_shape` to determine shapes overlapped by end\n # points of each `svg:path` or `svg:line`.\n df_shape_connections_i = pd.DataFrame([[shapes_canvas.find_shape(x1, y1),\n shapes_canvas.find_shape(x2, y2)]\n for i, (x1, y1, x2, y2) in\n df_connection_lines[coords_columns]\n .iterrows()],\n columns=['source', 'target'])\n # Order the source and target of each row so the source shape identifier is\n # always the lowest.\n df_shape_connections_i.sort_index(axis=1, inplace=True)\n # Tag each shape connection with the corresponding `svg:line`/`svg:path`\n # identifier. May be useful, e.g., in debugging.\n df_shape_connections_i['line_id'] = df_connection_lines['id']\n # Remove connections where source or target shape was not matched (e.g., if\n # one or more end points does not overlap with a shape).\n return df_shape_connections_i.dropna()", "response": "Extract all connection lines from a layer in a SVG file."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef discover():\n if CFG[\"plugins\"][\"autoload\"]:\n report_plugins = CFG[\"plugins\"][\"reports\"].value\n for plugin in report_plugins:\n try:\n importlib.import_module(plugin)\n LOG.debug(\"Found report: %s\", plugin)\n except ImportError:\n LOG.error(\"Could not find '%s'\", plugin)", "response": "Discover all experiments listed in benchbuild. reports. raw\nAttributeNames"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef index_name(self):\n fmt = self.campaign.export.elasticsearch.index_name\n fields = dict(date=self.report['date'])\n return fmt.format(**fields).lower()", "response": "Get Elasticsearch index name associated to the campaign\n "} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nremoving Elasticsearch index associated to the campaign", "response": "def remove_index(self):\n \"\"\"Remove Elasticsearch index associated to the campaign\"\"\"\n self.index_client.close(self.index_name)\n self.index_client.delete(self.index_name)"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef parse_args(args=None):\n from argparse import ArgumentParser\n\n if args is None:\n args = sys.argv\n\n parser = ArgumentParser(description='Rename template project with'\n 'hyphen-separated (path names and in '\n 'files).')\n parser.add_argument('new_name', help='New project name (e.g., '\n ' `my-new-project`)')\n\n args = parser.parse_args()\n return args", "response": "Parses arguments returns options and args."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nextracting the build information from a given executable.", "response": "def extract_build_info(exe_path, elf_section=ELF_SECTION):\n \"\"\"Extracts the build information from a given executable.\n\n The build information is expected to be in json format, which is parsed\n and returned as a dictionary.\n If no build information is found an empty dictionary is returned.\n\n This assumes binutils 2.25 to work.\n\n Args:\n exe_path (str): The full path to the executable to be examined\n\n Returns:\n dict: A dictionary of the extracted information.\n \"\"\"\n build_info = {}\n with mkdtemp() as tempd, pushd(tempd):\n proc = subprocess.Popen(\n [\n OBJCOPY,\n DUMP_SECTION,\n \"{secn}={ofile}\".format(secn=elf_section, ofile=BUILDINFO_FILE),\n exe_path,\n ],\n stderr=subprocess.PIPE,\n )\n proc.wait()\n errno = proc.returncode\n stderr = proc.stderr.read()\n if errno or len(stderr): # just return the empty dict\n LOGGER.warning('objcopy failed with errno %s.', errno)\n if len(stderr):\n LOGGER.warning('objcopy failed with following msg:\\n%s', stderr)\n return build_info\n\n with open(BUILDINFO_FILE) as build_info_f:\n try:\n build_info = json.load(build_info_f, object_hook=byteify)\n except JSONDcdError as jsde:\n LOGGER.warning('benchmark executable build is not valid json:')\n LOGGER.warning(jsde.msg)\n LOGGER.warning('build info section content:')\n LOGGER.warning(jsde.doc)\n return build_info"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\ncreate a new entry in the cache.", "response": "def rk_create(self, im, im_type, timeout=60):\n \"\"\"\n im: \u56fe\u7247\u5b57\u8282\n im_type: \u9898\u76ee\u7c7b\u578b\n \"\"\"\n params = {\n 'typeid': im_type,\n 'timeout': timeout,\n }\n params.update(self.base_params)\n files = {'image': ('check_code.png', im)}\n r = requests.post('http://api.ruokuai.com/create.json', data=params, files=files, headers=self.headers)\n return r.json()"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nreporting an error to the user.", "response": "def rk_report_error(self, im_id):\n \"\"\"\n im_id:\u62a5\u9519\u9898\u76ee\u7684ID\n \"\"\"\n params = {\n 'id': im_id,\n }\n params.update(self.base_params)\n r = requests.post('http://api.ruokuai.com/reporterror.json', data=params, headers=self.headers)\n return r.json()"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef lc_score(value):\n rebased = 2 * (value - 0.5)\n\n if rebased == 0:\n return 0\n elif rebased > 0:\n compliment = 1.0 - rebased\n score = - np.log2(compliment)\n else:\n compliment = 1.0 + rebased\n score = np.log2(compliment)\n return score", "response": "Evaluates the accuracy of a predictive measure of a predictive measure."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nreturns a customizable uchroot command.", "response": "def uchroot(*args, **kwargs):\n \"\"\"\n Return a customizable uchroot command.\n\n Args:\n args: List of additional arguments for uchroot (typical: mounts)\n Return:\n chroot_cmd\n \"\"\"\n uchroot_cmd = with_mounts(*args, uchroot_cmd_fn=no_llvm, **kwargs)\n return uchroot_cmd[\"--\"]"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nreturning a customizable uchroot command.", "response": "def no_llvm(*args, uid=0, gid=0, **kwargs):\n \"\"\"\n Return a customizable uchroot command.\n\n The command will be executed inside a uchroot environment.\n\n Args:\n args: List of additional arguments for uchroot (typical: mounts)\n Return:\n chroot_cmd\n \"\"\"\n uchroot_cmd = no_args()\n uchroot_cmd = uchroot_cmd[__default_opts__(uid, gid)]\n return uchroot_cmd[args]"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nreturning the uchroot command without any customizations.", "response": "def no_args(**kwargs):\n \"\"\"Return the uchroot command without any customizations.\"\"\"\n from benchbuild.utils.cmd import uchroot as uchrt\n\n prefixes = CFG[\"container\"][\"prefixes\"].value\n p_paths, p_libs = env(prefixes)\n uchrt = run.with_env_recursive(\n uchrt,\n LD_LIBRARY_PATH=path.list_to_path(p_libs),\n PATH=path.list_to_path(p_paths))\n\n return uchrt"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nreturn a uchroot command with all mounts enabled.", "response": "def with_mounts(*args, uchroot_cmd_fn=no_args, **kwargs):\n \"\"\"Return a uchroot command with all mounts enabled.\"\"\"\n mounts = CFG[\"container\"][\"mounts\"].value\n prefixes = CFG[\"container\"][\"prefixes\"].value\n\n uchroot_opts, mounts = __mounts__(\"mnt\", mounts)\n uchroot_cmd = uchroot_cmd_fn(**kwargs)\n uchroot_cmd = uchroot_cmd[uchroot_opts]\n uchroot_cmd = uchroot_cmd[args]\n paths, libs = env(mounts)\n prefix_paths, prefix_libs = env(prefixes)\n\n uchroot_cmd = run.with_env_recursive(\n uchroot_cmd,\n LD_LIBRARY_PATH=path.list_to_path(libs + prefix_libs),\n PATH=path.list_to_path(paths + prefix_paths))\n return uchroot_cmd"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nreturns a uchroot cmd that runs inside a filtered environment.", "response": "def clean_env(uchroot_cmd, varnames):\n \"\"\"Returns a uchroot cmd that runs inside a filtered environment.\"\"\"\n env = uchroot_cmd[\"/usr/bin/env\"]\n __clean_env = env[\"-u\", \",\".join(varnames)]\n return __clean_env"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef mounts(prefix, __mounts):\n i = 0\n mntpoints = []\n for mount in __mounts:\n if not isinstance(mount, dict):\n mntpoint = \"{0}/{1}\".format(prefix, str(i))\n mntpoints.append(mntpoint)\n i = i + 1\n return mntpoints", "response": "Compute the mountpoints of the current user."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\ncompute the environment of the change root for the current user.", "response": "def env(mounts):\n \"\"\"\n Compute the environment of the change root for the user.\n\n Args:\n mounts: The mountpoints of the current user.\n Return:\n paths\n ld_libs\n \"\"\"\n f_mounts = [m.strip(\"/\") for m in mounts]\n\n root = local.path(\"/\")\n\n ld_libs = [root / m / \"lib\" for m in f_mounts]\n ld_libs.extend([root / m / \"lib64\" for m in f_mounts])\n\n paths = [root / m / \"bin\" for m in f_mounts]\n paths.extend([root / m / \"sbin\" for m in f_mounts])\n paths.extend([root / m for m in f_mounts])\n return paths, ld_libs"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef get_abbreviations(kb):\n return {\"%s$$n%i\" % (author.get_urn(), i): abbrev\n for author in kb.get_authors()\n for i, abbrev in enumerate(author.get_abbreviations())\n if author.get_urn() is not None}", "response": "Returns a list of all the abbreviations in the given kb."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nreturning a dictionary like this :", "response": "def author_names(self):\n \"\"\"\n Returns a dictionary like this:\n\n {\n \"urn:cts:greekLit:tlg0012$$n1\" : \"Homer\"\n , \"urn:cts:greekLit:tlg0012$$n2\" : \"Omero\"\n , ...\n }\n \"\"\"\n return {\"%s$$n%i\" % (author.get_urn(), i): name[1]\n for author in self.get_authors()\n for i, name in enumerate(author.get_names())\n if author.get_urn() is not None}"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef get_resource_by_urn(self, urn):\n search_query = \"\"\"\n PREFIX frbroo: \n PREFIX crm: \n PREFIX rdfs: \n\n SELECT ?resource_URI\n\n WHERE {\n ?resource_URI crm:P1_is_identified_by ?urn .\n ?urn a crm:E42_Identifier .\n ?urn rdfs:label \"%s\"\n }\n \"\"\" % urn\n # check type of the input URN\n try:\n assert isinstance(urn, CTS_URN)\n except Exception as e:\n # convert to pyCTS.CTS_URN if it's a string\n urn = CTS_URN(urn)\n logger.debug('Converted the input urn from string to %s' % type(\n CTS_URN\n ))\n\n if (urn.work is not None):\n Work = self._session.get_class(surf.ns.EFRBROO['F1_Work'])\n result = self._store.execute_sparql(search_query)\n if len(result['results']['bindings']) == 0:\n raise ResourceNotFound\n else:\n tmp = result['results']['bindings'][0]\n resource_uri = tmp['resource_URI']['value']\n return self._session.get_resource(resource_uri, Work)\n\n elif (urn.work is None and urn.textgroup is not None):\n Person = self._session.get_class(surf.ns.EFRBROO['F10_Person'])\n result = self._store.execute_sparql(search_query)\n if len(result['results']['bindings']) == 0:\n raise ResourceNotFound\n else:\n tmp = result['results']['bindings'][0]\n resource_uri = tmp['resource_URI']['value']\n return self._session.get_resource(resource_uri, Person)", "response": "Fetch the resource corresponding to the input URN."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nsearching for a given string through the resources and returns an instance of HucitAuthor or HucitWork.", "response": "def search(self, search_string):\n \"\"\"\n Searches for a given string through the resources' labels.\n\n :param search_string:\n :return: an instance of `HucitAuthor` or `HucitWork`.\n \"\"\"\n query = \"\"\"\n SELECT ?s ?label ?type\n WHERE {\n ?s a ?type .\n ?s rdfs:label ?label .\n ?label bif:contains \"'%s'\" .\n }\n \"\"\" % search_string\n response = self._session.default_store.execute_sparql(query)\n results = [(result['s']['value'], result['label']['value'], result['type']['value'])\n for result in response[\"results\"][\"bindings\"]]\n resources = [(label, self._session.get_resource(subject, self._session.get_class(type)))\n for subject, label, type in results]\n\n Name = self._session.get_class(surf.ns.EFRBROO['F12_Name'])\n Title = self._session.get_class(surf.ns.EFRBROO['E35_Title'])\n Work = self._session.get_class(surf.ns.EFRBROO['F1_Work'])\n Person = self._session.get_class(surf.ns.EFRBROO['F10_Person'])\n\n result = []\n for label, resource in resources:\n if resource.uri == surf.ns.EFRBROO['E35_Title']:\n work = Work.get_by(efrbroo_P102_has_title = resource).first()\n result.append((label, work))\n elif resource.uri == surf.ns.EFRBROO['F12_Name']:\n author = Person.get_by(ecrm_P1_is_identified_by = resource).first()\n result.append((label, author))\n elif resource.uri == surf.ns.ECRM['E41_Appellation']:\n try:\n name = Name.get_by(ecrm_P139_has_alternative_form = resource).first()\n assert name is not None\n author = Person.get_by(ecrm_P1_is_identified_by = name).first()\n result.append((label, author))\n except Exception as e:\n title = Title.get_by(ecrm_P139_has_alternative_form=resource).first()\n assert title is not None\n work = Work.get_by(efrbroo_P102_has_title = title).first()\n result.append((label, work))\n return result"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nreturn the authors in the Knowledge Base.", "response": "def get_authors(self):\n \"\"\"\n Returns the authors in the Knowledge Base.\n\n :return: a list of `HucitAuthor` instances.\n\n \"\"\"\n Person = self._session.get_class(surf.ns.EFRBROO['F10_Person'])\n return list(Person.all())"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nreturning the author s works.", "response": "def get_works(self):\n \"\"\"Return the author's works.\n\n :return: a list of `HucitWork` instances.\n\n \"\"\"\n Work = self._session.get_class(surf.ns.EFRBROO['F1_Work'])\n return list(Work.all())"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\ngets the label corresponding to the author identified by the CTS URN.", "response": "def get_author_label(self, urn):\n \"\"\"Get the label corresponding to the author identified by the CTS URN.\n\n try to get an lang=en label (if multiple labels in this lang pick the shortest)\n try to get a lang=la label (if multiple labels in this lang exist pick the shortest)\n try to get a lang=None label (if multiple labels in this lang exist pick the shortest)\n\n returns None if no name is found\n\n \"\"\"\n author = self.get_resource_by_urn(urn)\n names = author.get_names()\n en_names = sorted([name[1] for name in names if name[0] == \"en\"], key=len)\n try:\n assert len(en_names) > 0\n return en_names[0]\n except Exception as e:\n none_names = sorted([name[1] for name in names if name[0] == None], key=len)\n try:\n return none_names[0]\n except Exception as e:\n la_names = sorted([name[1] for name in names if name[0] == \"la\"], key=len)\n try:\n assert len(la_names) > 0\n return la_names[0]\n except Exception as e:\n return None"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\ngather basic stats about the Knowledge Base and its contents.", "response": "def get_statistics(self):\n \"\"\"\n Gather basic stats about the Knowledge Base and its contents.\n\n :return: a dictionary\n\n \"\"\"\n statistics = {\n \"number_authors\": 0,\n \"number_author_names\": 0,\n \"number_author_abbreviations\": 0,\n \"number_works\": 0,\n \"number_work_titles\": 0,\n \"number_title_abbreviations\": 0,\n \"number_opus_maximum\":0,\n }\n for author in self.get_authors():\n if author.get_urn() is not None:\n opmax = True if self.get_opus_maximum_of(author.get_urn())\\\n is not None else False\n if opmax:\n statistics[\"number_opus_maximum\"] += 1\n statistics[\"number_authors\"] += 1\n statistics[\"number_author_names\"] += len(author.get_names())\n statistics[\"number_author_abbreviations\"] += len(\n author.get_abbreviations()\n )\n for work in author.get_works():\n statistics[\"number_works\"] += 1\n statistics[\"number_work_titles\"] += len(work.get_titles())\n statistics[\"number_title_abbreviations\"] += len(\n work.get_abbreviations()\n )\n return statistics"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef get_opus_maximum_of(self, author_cts_urn):\n author = self.get_resource_by_urn(author_cts_urn)\n assert author is not None\n works = author.get_works()\n\n if len(works) > 1:\n for work in works:\n if work.is_opus_maximum():\n return work\n elif len(works) == 1:\n return works[0]\n else:\n return None", "response": "Returns the opus maximum of an author."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef to_json(self):\n return json.dumps({\n \"statistics\": self.get_statistics()\n , \"authors\": [json.loads(author.to_json()) for author in self.get_authors()]\n }, indent=2)", "response": "Serialises the content of the KnowledgeBase as JSON."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef write_yaml_report(func):\n\n @wraps(func)\n def _wrapper(*args, **kwargs):\n now = datetime.datetime.now()\n with Timer() as timer:\n data = func(*args, **kwargs)\n if isinstance(data, (SEQUENCES, types.GeneratorType)):\n report = dict(children=list(map(str, data)))\n elif isinstance(data, MAPPINGS):\n report = data\n else:\n raise Exception('Unexpected data type: %s', type(data))\n report['elapsed'] = timer.elapsed\n report['date'] = now.isoformat()\n if \"no_exec\" not in kwargs and report is not None:\n with open(YAML_REPORT_FILE, 'w') as ostr:\n yaml.dump(report, ostr, default_flow_style=False)\n return report\n\n return _wrapper", "response": "Decorator used in campaign node post - processing\n "} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nclasses function that MUST be specified as decorator to the `__call__` method overriden by sub-classes.", "response": "def call_decorator(cls, func):\n \"\"\"class function that MUST be specified as decorator\n to the `__call__` method overriden by sub-classes.\n \"\"\"\n\n @wraps(func)\n def _wrap(self, *args, **kwargs):\n try:\n return func(self, *args, **kwargs)\n except Exception:\n self.logger.exception('While executing benchmark')\n if not (self.catch_child_exception or False):\n raise\n\n return _wrap"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nenumerates children and build associated objects", "response": "def traverse(self):\n \"\"\"Enumerate children and build associated objects\n \"\"\"\n builder = self.child_builder\n for child in self._children:\n with pushd(str(child)):\n yield child, builder(child)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef i_ww(self):\n d_values = []\n for i in range(self.n_pads_l):\n d_values.append(self.pad_position_l(i))\n d_values = np.array(d_values) - self.length / 2\n area_d_sqrd = sum(self.pad_area * d_values ** 2) * self.n_pads_w\n i_second = self.pad_i_ww * self.n_pads\n return area_d_sqrd + i_second", "response": "Second moment of inertia around the width axis."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef i_ll(self):\n d_values = []\n for i in range(self.n_pads_w):\n d_values.append(self.pad_position_w(i))\n d_values = np.array(d_values) - self.width / 2\n area_d_sqrd = sum(self.pad_area * d_values ** 2) * self.n_pads_l\n i_second = self.pad_i_ll * self.n_pads\n return area_d_sqrd + i_second", "response": "Second moment of inertia around the length axis."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\ndetermines the position of the ith pad in the length direction.", "response": "def pad_position_l(self, i):\n \"\"\"\n Determines the position of the ith pad in the length direction.\n Assumes equally spaced pads.\n\n :param i: ith number of pad in length direction (0-indexed)\n :return:\n \"\"\"\n if i >= self.n_pads_l:\n raise ModelError(\"pad index out-of-bounds\")\n return (self.length - self.pad_length) / (self.n_pads_l - 1) * i + self.pad_length / 2"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\ndetermining the position of the ith pad in the width direction.", "response": "def pad_position_w(self, i):\n \"\"\"\n Determines the position of the ith pad in the width direction.\n Assumes equally spaced pads.\n\n :param i: ith number of pad in width direction (0-indexed)\n :return:\n \"\"\"\n if i >= self.n_pads_w:\n raise ModelError(\"pad index out-of-bounds\")\n return (self.width - self.pad_width) / (self.n_pads_w - 1) * i + self.pad_width / 2"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nadding paths to the given batch object.", "response": "def add_to_batch(self, batch):\n '''\n Adds paths to the given batch object. They are all added as\n GL_TRIANGLES, so the batch will aggregate them all into a single OpenGL\n primitive.\n '''\n for name in self.paths:\n svg_path = self.paths[name]\n svg_path.add_to_batch(batch)"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nparse all path elements from xml_root and return a dict of the attributes.", "response": "def parse(self, xml_root, on_error=None):\n '''\n Parse all elements from xml_root.\n\n Optional on_error arg specifies a callback function to be run when\n an error occurs during parsing.\n The specified on_error function must accept 3 arguments:\n , , \n An example on_error handler is provided as svg_load.svg_parser.parse_warning(),\n where all SvgParseErrors are converted to warning messages. See usage below:\n >>> import re\n >>> svg_parser = SvgParser()\n >>> path_tag = etree.XML(\"\"\"\n ... \"\"\")\n >>> with warnings.catch_warnings(record=True) as w:\n ... svg = svg_parser.parse(path_tag, on_error=parse_warning)\n >>> print w[-1].category\n \n >>> match = re.search(r'^Error parsing None:\\d+, unsupported svg path command: l', str(w[-1].message))\n >>> print match is None\n False\n >>> path_tag = etree.XML(\"\"\"\n ... \"\"\")\n >>> with warnings.catch_warnings(record=True) as w:\n ... svg = svg_parser.parse(path_tag, on_error=parse_warning)\n >>> print w[-1].category\n \n >>> match = re.search(r'^Error parsing None:\\d+, loop needs 3 or more verts', str(w[-1].message))\n >>> print match is None\n False\n '''\n svg = Svg()\n svg_namespace = {'svg': 'http://www.w3.org/2000/svg'}\n path_tags = xml_root.xpath('(/svg:svg|/svg:svg/svg:g)/svg:path',\n namespaces=svg_namespace)\n parser = PathParser()\n for path_tag in path_tags:\n try:\n id, svg_path = parser.parse(path_tag)\n if svg_path.loops:\n svg.add_path(id, svg_path)\n except (ParseError, ) as why:\n filename = getattr(self, 'filename', None)\n args = (filename, path_tag, why.message)\n if on_error:\n on_error(*args)\n else:\n raise SvgParseError(*args)\n\n if svg.paths:\n x, y = svg.get_boundary().get_center()\n for svg_path in svg.paths.values():\n svg_path.offset(-x, -y)\n return svg"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nturning a string into a list of string and number chunks.", "response": "def alphanum_key(s):\n \"\"\"Turn a string into a list of string and number chunks.\n\n \"z23a\" -> [\"z\", 23, \"a\"]\n \"\"\"\n\n return [int(c) if c.isdigit() else c for c in _RE_INT.split(s)]"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\ndiscovering all experiments listed in PLUGINS_EXPERIMENTS.", "response": "def discover():\n \"\"\"\n Import all experiments listed in PLUGINS_EXPERIMENTS.\n\n Tests:\n >>> from benchbuild.settings import CFG\n >>> from benchbuild.experiments import discover\n >>> import logging as lg\n >>> import sys\n >>> l = lg.getLogger('benchbuild')\n >>> lg.getLogger('benchbuild').setLevel(lg.DEBUG)\n >>> lg.getLogger('benchbuild').handlers = [lg.StreamHandler(stream=sys.stdout)]\n >>> CFG[\"plugins\"][\"experiments\"] = [\"benchbuild.non.existing\", \"benchbuild.experiments.raw\"]\n >>> discover()\n Could not find 'benchbuild.non.existing'\n ImportError: No module named 'benchbuild.non'\n \"\"\"\n if CFG[\"plugins\"][\"autoload\"]:\n experiment_plugins = CFG[\"plugins\"][\"experiments\"].value\n for exp_plugin in experiment_plugins:\n try:\n importlib.import_module(exp_plugin)\n except ImportError as import_error:\n LOG.error(\"Could not find '%s'\", exp_plugin)\n LOG.error(\"ImportError: %s\", import_error.msg)"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nprinting a list of projects registered for that experiment.", "response": "def print_projects(projects=None):\n \"\"\"\n Print a list of projects registered for that experiment.\n\n Args:\n exp: The experiment to print all projects for.\n\n \"\"\"\n grouped_by = {}\n if not projects:\n print(\n \"Your selection didn't include any projects for this experiment.\")\n return\n\n for name in projects:\n prj = projects[name]\n\n if prj.GROUP not in grouped_by:\n grouped_by[prj.GROUP] = []\n\n grouped_by[prj.GROUP].append(\"{name}/{group}\".format(\n name=prj.NAME, group=prj.GROUP))\n\n for name in grouped_by:\n print(\"group: {0}\".format(name))\n group_projects = sorted(grouped_by[name])\n for prj in group_projects:\n prj_cls = projects[prj]\n\n version_str = None\n if hasattr(prj_cls, 'versions'):\n version_str = \", \".join(prj_cls.versions())\n\n project_id = \"{0}/{1}\".format(prj_cls.NAME, prj_cls.GROUP)\n\n project_str = \\\n \" name: {id:<32} version: {version:<24} source: {src}\".format(\n id=str(project_id),\n version=str(prj_cls.VERSION),\n src=str(prj_cls.SRC_FILE))\n print(project_str)\n if prj_cls.__doc__:\n docstr = prj_cls.__doc__.strip(\"\\n \")\n print(\" description: {desc}\".format(desc=docstr))\n if version_str:\n print(\" versions: {versions}\".format(versions=version_str))\n print()"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nrun a CTS API from command line.", "response": "def _commandline(repositories,\n port=8000, host=\"127.0.0.1\", debug=False,\n cache=None, cache_path=\"./cache\", redis=None):\n \"\"\" Run a CTS API from command line.\n\n .. warning:: This function should not be used in the production context\n\n :param repositories:\n :param port:\n :param ip:\n :param debug:\n :param cache:\n :param cache_path:\n :return:\n \"\"\"\n\n if cache == \"redis\":\n nautilus_cache = RedisCache(redis)\n cache_type = \"redis\"\n elif cache == \"filesystem\":\n nautilus_cache = FileSystemCache(cache_path)\n cache_type = \"simple\"\n else:\n nautilus_cache = NullCache()\n cache_type = \"simple\"\n\n app = Flask(\"Nautilus\")\n if debug:\n app.logger.setLevel(logging.INFO)\n\n resolver = NautilusCtsResolver(resource=repositories)\n nautilus = FlaskNautilus(\n app=app,\n resolver=resolver\n #parser_cache=WerkzeugCacheWrapper(nautilus_cache),\n #logger=None\n )\n nautilus.resolver.parse()\n app.run(debug=debug, port=port, host=host)"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef tex_escape(text):\n conv = {\n '&': r'\\&',\n '%': r'\\%',\n '$': r'\\$',\n '#': r'\\#',\n '_': r'\\_',\n '{': r'\\{',\n '}': r'\\}',\n '~': r'\\textasciitilde{}',\n '^': r'\\^{}',\n '\\\\': r'\\textbackslash{}',\n '<': r'\\textless ',\n '>': r'\\textgreater ',\n }\n regex = re.compile(\n '|'.join(\n re.escape(six.text_type(key))\n for key in sorted(conv.keys(), key=lambda item: -len(item))\n )\n )\n return regex.sub(lambda match: conv[match.group()], text)", "response": "Escape string for LaTeX usage\n :param text: a plain text message\n :return: the message escaped to appear correctly in LaTeX"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\ngenerate a report from a campaign", "response": "def render(template=None, ostr=None, **kwargs):\n \"\"\"Generate report from a campaign\n\n :param template: Jinja template to use, ``DEFAULT_TEMPLATE`` is used\n if not specified\n :param ostr: output file or filename. Default is standard output\n \"\"\"\n jinja_environment.filters['texscape'] = tex_escape\n template = template or DEFAULT_TEMPLATE\n ostr = ostr or sys.stdout\n jinja_template = jinja_environment.get_template(template)\n jinja_template.stream(**kwargs).dump(ostr)"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef get_shapes_pymunk_space(df_convex_shapes, shape_i_columns):\n '''\n Return two-ple containing:\n\n - A `pymunk.Space` instance.\n - A `pandas.Series` mapping each `pymunk.Body` object in the `Space` to a\n shape index.\n\n The `Body` to shape index mapping makes it possible to, for example, look\n up the index of the convex shape associated with a `Body` returned by a\n `pymunk` point query in the `Space`.\n '''\n if isinstance(shape_i_columns, bytes):\n shape_i_columns = [shape_i_columns]\n\n space = pm.Space()\n\n bodies = []\n\n convex_groups = df_convex_shapes.groupby(shape_i_columns)\n\n for shape_i, df_i in convex_groups:\n if not isinstance(shape_i, (list, tuple)):\n shape_i = [shape_i]\n\n if hasattr(pm.Body, 'STATIC'):\n # Assume `pymunk>=5.0`, where static bodies must be declared\n # explicitly.\n body = pm.Body(body_type=pm.Body.STATIC)\n else:\n # Assume `pymunk<5.0`, where bodies are static unless otherwise\n # specified.\n body = pm.Body()\n # Using the code below is about 66% faster than:\n # `df_i[['x', 'y']].values`.\n points = [[x, y] for x, y in zip(df_i.x, df_i.y)]\n poly = pm.Poly(body, points)\n space.add(poly)\n bodies.append([body, shape_i[0]])\n bodies = None if not bodies else bodies\n return space, (pd.DataFrame(bodies, columns=['body',\n shape_i_columns[0]])\n .set_index('body')[shape_i_columns[0]])", "response": "Returns two -ple containing the two -ple shape space and a pandas. Series mapping each body to avec."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef load_json(ffp, custom=None, verbose=0):\n data = json.load(open(ffp))\n return ecp_dict_to_objects(data, custom, verbose=verbose)", "response": "Given a json file creates a dictionary of sfsi objects"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef loads_json(p_str, custom=None, meta=False, verbose=0):\n data = json.loads(p_str)\n if meta:\n md = {}\n for item in data:\n if item != \"models\":\n md[item] = data[item]\n return ecp_dict_to_objects(data, custom, verbose=verbose), md\n else:\n return ecp_dict_to_objects(data, custom, verbose=verbose)", "response": "Given a json string creates a dictionary of sfsi objects"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\ngives an ecp dictionary build a dictionary of sfsi objects.", "response": "def ecp_dict_to_objects(ecp_dict, custom_map=None, verbose=0):\n \"\"\"\n Given an ecp dictionary, build a dictionary of sfsi objects\n\n :param ecp_dict: dict, engineering consistency project dictionary\n :param custom: dict, used to load custom objects, {model type: custom object}\n :param verbose: int, console output\n :return: dict\n \"\"\"\n if custom_map is None:\n custom_map = {}\n\n obj_map = {\n \"soil-soil\": soils.Soil,\n \"soil-critical_soil\": soils.CriticalSoil,\n \"soil-soil_critical\": soils.CriticalSoil, # deprecated type\n \"soil-stress_dependent_soil\": soils.StressDependentSoil,\n \"soil-soil_stress_dependent\": soils.StressDependentSoil,\n \"soil_profile-soil_profile\": soils.SoilProfile,\n \"building-building\": buildings.Building,\n \"building-frame_building\": buildings.FrameBuilding,\n \"building-frame_building2D\": buildings.FrameBuilding2D,\n \"building-building_frame2D\": buildings.FrameBuilding2D, # deprecated type\n \"building-wall_building\": buildings.WallBuilding,\n \"building-building_wall\": buildings.WallBuilding, # deprecated type\n \"building-structure\": buildings.SDOFBuilding, # Deprecated type, remove in v1\n \"building-sdof\": buildings.SDOFBuilding,\n \"foundation-foundation\": foundations.Foundation,\n \"foundation-foundation_raft\": foundations.RaftFoundation, # deprecated type\n \"foundation-raft_foundation\": foundations.RaftFoundation,\n \"foundation-raft\": foundations.RaftFoundation, # Deprecated approach for type, remove in v1\n \"foundation-pad_foundation\": foundations.PadFoundation,\n \"foundation-foundation_pad\": foundations.PadFoundation, # deprecated type\n \"section-section\": buildings.Section,\n \"custom_object-custom_object\": abstract_models.CustomObject,\n \"system-system\": systems.SoilStructureSystem, # deprecated type\n \"system-sfs\": systems.SoilStructureSystem\n }\n\n # merge and overwrite the object map with custom maps\n # for item in custom_map:\n # obj_map[item] = custom_map[item]\n obj_map = {**obj_map, **custom_map}\n\n data_models = ecp_dict[\"models\"]\n\n exception_list = []\n objs = OrderedDict()\n collected = set([])\n # Set base type properly\n mtypes = list(data_models)\n for mtype in mtypes:\n base_type = mtype\n if base_type[:-1] in standard_types: # support the loading of old plural based ecp files\n base_type = base_type[:-1]\n data_models[base_type] = data_models[mtype]\n del data_models[mtype]\n for m_id in data_models[base_type]:\n data_models[base_type][m_id][\"base_type\"] = base_type\n\n for mtype in data_models:\n base_type = mtype\n if base_type in exception_list:\n continue\n collected.add(base_type)\n objs[base_type] = OrderedDict()\n for m_id in data_models[mtype]:\n obj = data_models[mtype][m_id]\n if \"type\" not in obj:\n obj[\"type\"] = base_type\n try:\n obj_class = obj_map[\"%s-%s\" % (base_type, obj[\"type\"])]\n except KeyError:\n if obj[\"type\"] in deprecated_types:\n try:\n obj_class = obj_map[\"%s-%s\" % (base_type, deprecated_types[obj[\"type\"]])]\n except KeyError:\n raise KeyError(\"Map for Model: '%s' index: '%s' and type: '%s' not available, \"\n \"add '%s-%s' to custom dict\" % (base_type, m_id, base_type, base_type, obj[\"type\"]))\n else:\n raise KeyError(\"Map for Model: '%s' index: '%s' and type: '%s' not available, \"\n \"add '%s-%s' to custom dict\" % (base_type, m_id, base_type, base_type, obj[\"type\"]))\n try:\n new_instance = obj_class()\n except TypeError as e:\n if \"required positional argument:\" in str(e):\n parameters = [str(e).split(\"argument: \")[-1]]\n elif \"required positional arguments:\" in str(e):\n p_str = str(e).split(\"arguments: \")[-1]\n if \", and \" in p_str: # if more than 2\n partial = p_str.split(\", and \")\n parameters = partial[0].split(\", \") + partial[-1:]\n else: # if one\n parameters = p_str.split(\" and \")\n else:\n raise TypeError(e)\n params = []\n for parameter in parameters:\n parameter = parameter[1:-1]\n try:\n params.append(data_models[mtype][m_id][parameter])\n except KeyError as e2: # To be removed and just raise exception\n deprecation(\"Your file is out of date, \"\n \"run sfsimodels.migrate_ecp(, ).\")\n if mtype == \"building\":\n params = [len(data_models[mtype][m_id][\"storey_masses\"])] # n_storeys\n if \"frame\" in data_models[mtype][m_id][\"type\"]:\n params.append(len(data_models[mtype][m_id][\"bay_lengths\"]))\n else:\n raise KeyError(\"Can't find required positional argument: {0} for {1} id: {2}\".format(\n parameter, mtype, m_id\n ))\n new_instance = obj_class(*params)\n\n add_to_obj(new_instance, data_models[mtype][m_id], objs=objs, verbose=verbose)\n # print(mtype, m_id)\n objs[base_type][int(data_models[mtype][m_id][\"id\"])] = new_instance\n\n # Deal with all the exceptions\n # for mtype in data_models:\n # base_type = mtype\n #\n # if base_type in collected:\n # continue\n # if base_type not in objs:\n # objs[base_type] = OrderedDict()\n\n all_bts = list(objs)\n for base_type in all_bts: # Support for old style ecp file\n if base_type in standard_types:\n objs[base_type + \"s\"] = objs[base_type]\n return objs"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef migrate_ecp(in_ffp, out_ffp):\n objs, meta_data = load_json_and_meta(in_ffp)\n ecp_output = Output()\n for m_type in objs:\n for instance in objs[m_type]:\n ecp_output.add_to_dict(objs[m_type][instance])\n ecp_output.name = meta_data[\"name\"]\n ecp_output.units = meta_data[\"units\"]\n ecp_output.comments = meta_data[\"comments\"]\n ecp_output.sfsimodels_version = meta_data[\"sfsimodels_version\"]\n p_str = json.dumps(ecp_output.to_dict(), skipkeys=[\"__repr__\"], indent=4)\n a = open(out_ffp, \"w\")\n a.write(p_str)\n a.close()", "response": "Migrates and ECP file to the current version of sfsimodels"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nconverting an object to a dictionary.", "response": "def add_to_dict(self, an_object, extras=None):\n \"\"\"\n Convert models to json serialisable output\n\n :param an_object: An instance of a model object\n :param extras: A dictionary of extra variables that should be\n :return:\n \"\"\"\n if an_object.id is None:\n raise ModelError(\"id must be set on object before adding to output.\")\n if hasattr(an_object, \"base_type\"):\n mtype = an_object.base_type\n elif hasattr(an_object, \"type\"):\n if an_object.type in standard_types:\n mtype = an_object.type\n else:\n mtype = \"custom_type\"\n else:\n raise ModelError(\"Object does not have attribute 'base_type' or 'type', cannot add to output.\")\n if mtype not in self.unordered_models: # Catch any custom objects\n self.unordered_models[mtype] = OrderedDict()\n\n if hasattr(an_object, \"add_to_dict\"):\n an_object.add_to_dict(self.unordered_models)\n\n elif hasattr(an_object, \"to_dict\"):\n self.unordered_models[mtype][an_object.unique_hash] = an_object.to_dict(compression=self.compression)\n else:\n raise ModelError(\"Object does not have method 'to_dict', cannot add to output.\")"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nadd additional objects or dictionaries to output file that don t conform to standard objects.", "response": "def add_to_output(self, mtype, m_id, serialisable_dict):\n \"\"\"\n Can add additional objects or dictionaries to output file that don't conform to standard objects.\n\n :param mtype:\n :param m_id:\n :param serialisable_dict:\n :return:\n \"\"\"\n if mtype not in self.unordered_models:\n self.unordered_models[mtype] = OrderedDict()\n self.unordered_models[mtype][m_id] = serialisable_dict"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nreturning a dictionary of all the unhashed models in the database.", "response": "def models(self):\n \"\"\"Unhashed\"\"\"\n models_dict = OrderedDict()\n collected = []\n for item in standard_types:\n if item in self.unordered_models:\n new_dict, replacement_dict = unhash_dict(self.unordered_models[item])\n models_dict[item] = new_dict\n collected.append(item)\n for item in self.unordered_models:\n # print(\"item: \", item)\n if item not in collected:\n new_dict, replacement_dict = unhash_dict(self.unordered_models[item])\n models_dict[item] = new_dict\n return models_dict"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef available_cpu_count() -> int:\n\n # cpuset\n # cpuset may restrict the number of *available* processors\n try:\n match = re.search(r'(?m)^Cpus_allowed:\\s*(.*)$',\n open('/proc/self/status').read())\n if match:\n res = bin(int(match.group(1).replace(',', ''), 16)).count('1')\n if res > 0:\n return res\n except IOError:\n LOG.debug(\"Could not get the number of allowed CPUs\")\n\n # http://code.google.com/p/psutil/\n try:\n import psutil\n return psutil.cpu_count() # psutil.NUM_CPUS on old versions\n except (ImportError, AttributeError):\n LOG.debug(\"Could not get the number of allowed CPUs\")\n\n # POSIX\n try:\n res = int(os.sysconf('SC_NPROCESSORS_ONLN'))\n\n if res > 0:\n return res\n except (AttributeError, ValueError):\n LOG.debug(\"Could not get the number of allowed CPUs\")\n\n # Linux\n try:\n res = open('/proc/cpuinfo').read().count('processor\\t:')\n\n if res > 0:\n return res\n except IOError:\n LOG.debug(\"Could not get the number of allowed CPUs\")\n\n raise Exception('Can not determine number of CPUs on this system')", "response": "Get the number of available CPUs on this system."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef escape_yaml(raw_str: str) -> str:\n escape_list = [char for char in raw_str if char in ['!', '{', '[']]\n if len(escape_list) == 0:\n return raw_str\n\n str_quotes = '\"'\n i_str_quotes = \"'\"\n if str_quotes in raw_str and str_quotes not in raw_str[1:-1]:\n return raw_str\n\n if str_quotes in raw_str[1:-1]:\n raw_str = i_str_quotes + raw_str + i_str_quotes\n else:\n raw_str = str_quotes + raw_str + str_quotes\n return raw_str", "response": "Shell - Escape a yaml input string."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nconverts a given value to a YAML string.", "response": "def to_yaml(value) -> str:\n \"\"\"Convert a given value to a YAML string.\"\"\"\n stream = yaml.io.StringIO()\n dumper = ConfigDumper(stream, default_flow_style=True, width=sys.maxsize)\n val = None\n try:\n dumper.open()\n dumper.represent(value)\n val = stream.getvalue().strip()\n dumper.close()\n finally:\n dumper.dispose()\n\n return val"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef to_env_var(env_var: str, value) -> str:\n val = to_yaml(value)\n ret_val = \"%s=%s\" % (env_var, escape_yaml(val))\n return ret_val", "response": "Converts a value into an environment variable."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef find_config(test_file=None, defaults=None, root=os.curdir):\n if defaults is None:\n defaults = [\".benchbuild.yml\", \".benchbuild.yaml\"]\n\n def walk_rec(cur_path, root):\n cur_path = local.path(root) / test_file\n if cur_path.exists():\n return cur_path\n\n new_root = local.path(root) / os.pardir\n return walk_rec(cur_path, new_root) if new_root != root else None\n\n if test_file is not None:\n return walk_rec(test_file, root)\n\n for test_file in defaults:\n ret = walk_rec(test_file, root)\n if ret is not None:\n return ret", "response": "Find the path to the default config file."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef setup_config(cfg, config_filenames=None, env_var_name=None):\n if env_var_name is None:\n env_var_name = \"BB_CONFIG_FILE\"\n\n config_path = os.getenv(env_var_name, None)\n if not config_path:\n config_path = find_config(defaults=config_filenames)\n\n if config_path:\n cfg.load(config_path)\n cfg[\"config_file\"] = os.path.abspath(config_path)\n cfg.init_from_env()", "response": "This function will initialize the given configuration object."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef upgrade(cfg):\n db_node = cfg[\"db\"]\n old_db_elems = [\"host\", \"name\", \"port\", \"pass\", \"user\", \"dialect\"]\n has_old_db_elems = [x in db_node for x in old_db_elems]\n\n if any(has_old_db_elems):\n print(\"Old database configuration found. \"\n \"Converting to new connect_string. \"\n \"This will *not* be stored in the configuration automatically.\")\n cfg[\"db\"][\"connect_string\"] = \\\n \"{dialect}://{user}:{password}@{host}:{port}/{name}\".format(\n dialect=cfg[\"db\"][\"dialect\"][\"value\"],\n user=cfg[\"db\"][\"user\"][\"value\"],\n password=cfg[\"db\"][\"pass\"][\"value\"],\n host=cfg[\"db\"][\"host\"][\"value\"],\n port=cfg[\"db\"][\"port\"][\"value\"],\n name=cfg[\"db\"][\"name\"][\"value\"])", "response": "Provide forward migration for configuration files."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nconstructs a uuid. UUID object form a scalar YAML node.", "response": "def uuid_constructor(loader, node):\n \"\"\"\"\n Construct a uuid.UUID object form a scalar YAML node.\n\n Tests:\n >>> yaml.add_constructor(\"!uuid\", uuid_constructor, Loader=yaml.SafeLoader)\n >>> yaml.safe_load(\"{'test': !uuid 'cc3702ca-699a-4aa6-8226-4c938f294d9b'}\")\n {'test': UUID('cc3702ca-699a-4aa6-8226-4c938f294d9b')}\n \"\"\"\n\n value = loader.construct_scalar(node)\n return uuid.UUID(value)"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nadd an implicit pattern resolver for UUID objects.", "response": "def uuid_add_implicit_resolver(Loader=ConfigLoader, Dumper=ConfigDumper):\n \"\"\"\n Attach an implicit pattern resolver for UUID objects.\n\n Tests:\n >>> class TestDumper(yaml.SafeDumper): pass\n >>> class TestLoader(yaml.SafeLoader): pass\n >>> TUUID = 'cc3702ca-699a-4aa6-8226-4c938f294d9b'\n >>> IN = {'test': uuid.UUID(TUUID)}\n >>> OUT = '{test: cc3702ca-699a-4aa6-8226-4c938f294d9b}'\n\n >>> yaml.add_representer(uuid.UUID, uuid_representer, Dumper=TestDumper)\n >>> yaml.add_constructor('!uuid', uuid_constructor, Loader=TestLoader)\n >>> uuid_add_implicit_resolver(Loader=TestLoader, Dumper=TestDumper)\n\n >>> yaml.dump(IN, Dumper=TestDumper)\n 'test: cc3702ca-699a-4aa6-8226-4c938f294d9b\\\\n'\n >>> yaml.load(OUT, Loader=TestLoader)\n {'test': UUID('cc3702ca-699a-4aa6-8226-4c938f294d9b')}\n \"\"\"\n uuid_regex = r'^\\b[a-f0-9]{8}-\\b[a-f0-9]{4}-\\b[a-f0-9]{4}-\\b[a-f0-9]{4}-\\b[a-f0-9]{12}$'\n pattern = re.compile(uuid_regex)\n yaml.add_implicit_resolver('!uuid', pattern, Loader=Loader, Dumper=Dumper)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nstoring the configuration dictionary to a file.", "response": "def store(self, config_file):\n \"\"\" Store the configuration dictionary to a file.\"\"\"\n\n selfcopy = copy.deepcopy(self)\n selfcopy.filter_exports()\n\n with open(config_file, 'w') as outf:\n yaml.dump(\n selfcopy.node,\n outf,\n width=80,\n indent=4,\n default_flow_style=False,\n Dumper=ConfigDumper)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef load(self, _from):\n\n def load_rec(inode, config):\n \"\"\"Recursive part of loading.\"\"\"\n for k in config:\n if isinstance(config[k], dict) and \\\n k not in ['value', 'default']:\n if k in inode:\n load_rec(inode[k], config[k])\n else:\n LOG.debug(\"+ config element: '%s'\", k)\n else:\n inode[k] = config[k]\n\n with open(_from, 'r') as infile:\n obj = yaml.load(infile, Loader=ConfigLoader)\n upgrade(obj)\n load_rec(self.node, obj)\n self['config_file'] = os.path.abspath(_from)", "response": "Load the configuration dictionary from file."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef init_from_env(self):\n\n if 'default' in self.node:\n env_var = self.__to_env_var__().upper()\n if self.has_value():\n env_val = self.node['value']\n else:\n env_val = self.node['default']\n env_val = os.getenv(env_var, to_yaml(env_val))\n try:\n self.node['value'] = yaml.load(\n str(env_val), Loader=ConfigLoader)\n except ValueError:\n self.node['value'] = env_val\n else:\n if isinstance(self.node, dict):\n for k in self.node:\n self[k].init_from_env()", "response": "Initialize this node from environment variables."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef value(self):\n\n def validate(node_value):\n if hasattr(node_value, 'validate'):\n node_value.validate()\n return node_value\n\n if 'value' in self.node:\n return validate(self.node['value'])\n return self", "response": "Return the value of the node."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nconverting configuration object to a flat dictionary.", "response": "def to_env_dict(self):\n \"\"\"Convert configuration object to a flat dictionary.\"\"\"\n entries = {}\n if self.has_value():\n return {self.__to_env_var__(): self.node['value']}\n if self.has_default():\n return {self.__to_env_var__(): self.node['default']}\n\n for k in self.node:\n entries.update(self[k].to_env_dict())\n\n return entries"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef rename_keys(record: Mapping, key_map: Mapping) -> dict:\n\n new_record = dict()\n\n for k, v in record.items():\n key = key_map[k] if k in key_map else k\n new_record[key] = v\n\n return new_record", "response": "Return a new record with same keys or renamed keys if key found in key_map."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nreturn a new record with renamed keys including keys only found in key_map.", "response": "def replace_keys(record: Mapping, key_map: Mapping) -> dict:\n \"\"\"New record with renamed keys including keys only found in key_map.\"\"\"\n\n return {key_map[k]: v for k, v in record.items() if k in key_map}"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\ninserting None as value for missing fields.", "response": "def inject_nulls(data: Mapping, field_names) -> dict:\n \"\"\"Insert None as value for missing fields.\"\"\"\n\n record = dict()\n\n for field in field_names:\n record[field] = data.get(field, None)\n\n return record"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nread a likwid struct from the text stream.", "response": "def read_struct(fstream):\n \"\"\"\n Read a likwid struct from the text stream.\n\n Args:\n fstream: Likwid's filestream.\n\n Returns (dict(str: str)):\n A dict containing all likwid's struct info as key/value pairs.\n \"\"\"\n line = fstream.readline().strip()\n fragments = line.split(\",\")\n fragments = [x for x in fragments if x is not None]\n partition = dict()\n if not len(fragments) >= 3:\n return None\n\n partition[\"struct\"] = fragments[0]\n partition[\"info\"] = fragments[1]\n partition[\"num_lines\"] = fragments[2]\n\n struct = None\n if partition is not None and partition[\"struct\"] == \"STRUCT\":\n num_lines = int(partition[\"num_lines\"].strip())\n struct = {}\n for _ in range(num_lines):\n cols = fetch_cols(fstream)\n struct.update({cols[0]: cols[1:]})\n return struct"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef read_table(fstream):\n pos = fstream.tell()\n line = fstream.readline().strip()\n fragments = line.split(\",\")\n fragments = [x for x in fragments if x is not None]\n partition = dict()\n if not len(fragments) >= 4:\n return None\n\n partition[\"table\"] = fragments[0]\n partition[\"group\"] = fragments[1]\n partition[\"set\"] = fragments[2]\n partition[\"num_lines\"] = fragments[3]\n\n struct = None\n if partition is not None and partition[\"table\"] == \"TABLE\":\n num_lines = int(partition[\"num_lines\"].strip())\n struct = {}\n header = fetch_cols(fstream)\n\n struct.update({header[0]: header[1:]})\n for _ in range(num_lines):\n cols = fetch_cols(fstream)\n struct.update({cols[0]: cols[1:]})\n else:\n fstream.seek(pos)\n\n return struct", "response": "Read a likwid table info from the text stream."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nreading all structs from likwid s output file stream.", "response": "def read_structs(fstream):\n \"\"\"\n Read all structs from likwid's file stream.\n\n Args:\n fstream: Likwid's output file stream.\n\n Returns:\n A generator that can be used to iterate over all structs in the\n fstream.\n \"\"\"\n struct = read_struct(fstream)\n while struct is not None:\n yield struct\n struct = read_struct(fstream)"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef read_tables(fstream):\n table = read_table(fstream)\n while table is not None:\n yield table\n table = read_table(fstream)", "response": "Read all tables from likwid s output file stream."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\ngetting the complete measurement info from likwid s region info.", "response": "def get_measurements(region, core_info, data, extra_offset=0):\n \"\"\"\n Get the complete measurement info from likwid's region info.\n\n Args:\n region: The region we took a measurement in.\n core_info: The core information.\n data: The raw data.\n extra_offset (int): default = 0\n\n Returns (list((region, metric, core, value))):\n A list of measurement tuples, a tuple contains the information about\n the region, the metric, the core and the actual value.\n \"\"\"\n measurements = []\n clean_core_info = [x for x in core_info if x]\n cores = len(clean_core_info)\n for k in data:\n if k not in [\"1\", \"Region Info\", \"Event\", \"Metric\", \"CPU clock\"]:\n slot = data[k]\n for i in range(cores):\n core = core_info[i]\n idx = extra_offset + i\n if core and slot[idx]:\n measurements.append((region, k, core, slot[idx]))\n\n return measurements"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef perfcounters(infile):\n measurements = []\n with open(infile, 'r') as in_file:\n read_struct(in_file)\n for region_struct in read_structs(in_file):\n region = region_struct[\"1\"][1]\n core_info = region_struct[\"Region Info\"]\n measurements += \\\n get_measurements(region, core_info, region_struct)\n\n for table_struct in read_tables(in_file):\n core_info = None\n if \"Event\" in table_struct:\n offset = 1\n core_info = table_struct[\"Event\"][offset:]\n measurements += get_measurements(region, core_info,\n table_struct, offset)\n elif \"Metric\" in table_struct:\n core_info = table_struct[\"Metric\"]\n measurements += get_measurements(region, core_info,\n table_struct)\n return measurements", "response": "Get a complete list of all measurements extracted from likwid s file stream."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef setup_logger(verbose, logfile):\n level = logging.WARNING\n if verbose == 1:\n level = logging.INFO\n elif verbose > 1:\n level = logging.DEBUG\n if logfile:\n logging.basicConfig(filename=logfile, level=level, format=LOGGING_FORMAT)\n else:\n logging.basicConfig(level=level, format=LOGGING_FORMAT)", "response": "Prepare root logger\n :param verbose: integer greater than 0 to indicate verbosity level"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nprograms initialization for all provided executables", "response": "def cli_common(doc, **kwargs):\n \"\"\"Program initialization for all provided executables\n \"\"\"\n arguments = docopt(doc, version='hpcbench ' + __version__, **kwargs)\n setup_logger(arguments['-v'], arguments['--log'])\n load_components()\n try:\n import matplotlib\n except ImportError:\n pass\n else:\n matplotlib.use('PS')\n return arguments"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nreturning a list of coordinates of points found in SVG path element.", "response": "def shape_path_points(svg_path_d):\n '''\n Parameters\n ----------\n svg_path_d : str\n ``\"d\"`` attribute of SVG ``path`` element.\n\n Returns\n -------\n list\n List of coordinates of points found in SVG path.\n\n Each point is represented by a dictionary with keys ``x`` and ``y``.\n '''\n # TODO Add support for relative commands, e.g., `l, h, v`.\n def _update_path_state(path_state, match):\n if match.group('xy_command'):\n for dim_j in 'xy':\n path_state[dim_j] = float(match.group(dim_j))\n if path_state.get('x0') is None:\n for dim_j in 'xy':\n path_state['%s0' % dim_j] = path_state[dim_j]\n elif match.group('x_command'):\n path_state['x'] = float(match.group('hx'))\n elif match.group('y_command'):\n path_state['y'] = float(match.group('vy'))\n elif match.group('command') == 'Z':\n for dim_j in 'xy':\n path_state[dim_j] = path_state['%s0' % dim_j]\n return path_state\n\n # Some commands in a SVG path element `\"d\"` attribute require previous state.\n #\n # For example, the `\"H\"` command is a horizontal move, so the previous\n # ``y`` position is required to resolve the new `(x, y)` position.\n #\n # Iterate through the commands in the `\"d\"` attribute in order and maintain\n # the current path position in the `path_state` dictionary.\n path_state = {'x': None, 'y': None}\n return [{k: v for k, v in six.iteritems(_update_path_state(path_state, match_i)) if k in 'xy'} for match_i in cre_path_command\n .finditer(svg_path_d)]"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef svg_shapes_to_df(svg_source, xpath='//svg:path | //svg:polygon',\n namespaces=INKSCAPE_NSMAP):\n '''\n Construct a data frame with one row per vertex for all shapes in\n :data:`svg_source``.\n\n Arguments\n ---------\n svg_source : str or file-like\n A file path, URI, or file-like object.\n xpath : str, optional\n XPath path expression to select shape nodes.\n\n By default, all ``svg:path`` and ``svg:polygon`` elements are selected.\n namespaces : dict, optional\n Key/value mapping of XML namespaces.\n\n Returns\n -------\n pandas.DataFrame\n Frame with one row per vertex for all shapes in :data:`svg_source`,\n with the following columns:\n - ``vertex_i``: The index of the vertex within the corresponding\n shape.\n - ``x``: The x-coordinate of the vertex.\n - ``y``: The y-coordinate of the vertex.\n - other: attributes of the SVG shape element (e.g., ``id``, ``fill``,\n etc.)\n '''\n from lxml import etree\n\n e_root = etree.parse(svg_source)\n frames = []\n attribs_set = set()\n\n # Get list of attributes that are set in any of the shapes (not including\n # the `svg:path` `\"d\"` attribute or the `svg:polygon` `\"points\"`\n # attribute).\n #\n # This, for example, collects attributes such as:\n #\n # - `fill`, `stroke` (as part of `\"style\"` attribute)\n # - `\"transform\"`: matrix, scale, etc.\n for shape_i in e_root.xpath(xpath, namespaces=namespaces):\n attribs_set.update(list(shape_i.attrib.keys()))\n\n for k in ('d', 'points'):\n if k in attribs_set:\n attribs_set.remove(k)\n\n attribs = list(sorted(attribs_set))\n\n # Always add 'id' attribute as first attribute.\n if 'id' in attribs:\n attribs.remove('id')\n attribs.insert(0, 'id')\n\n for shape_i in e_root.xpath(xpath, namespaces=namespaces):\n # Gather shape attributes from SVG element.\n base_fields = [shape_i.attrib.get(k, None) for k in attribs]\n\n if shape_i.tag == '{http://www.w3.org/2000/svg}path':\n # Decode `svg:path` vertices from [`\"d\"`][1] attribute.\n #\n # [1]: https://developer.mozilla.org/en-US/docs/Web/SVG/Attribute/d\n points_i = [base_fields + [i] + [point_i.get(k) for k in 'xy']\n for i, point_i in\n enumerate(shape_path_points(shape_i.attrib['d']))]\n elif shape_i.tag == '{http://www.w3.org/2000/svg}polygon':\n # Decode `svg:polygon` vertices from [`\"points\"`][2] attribute.\n #\n # [2]: https://developer.mozilla.org/en-US/docs/Web/SVG/Attribute/points\n points_i = [base_fields + [i] + list(map(float, v.split(',')))\n for i, v in enumerate(shape_i.attrib['points']\n .strip().split(' '))]\n else:\n warnings.warning('Unsupported shape tag type: %s' % shape_i.tag)\n continue\n frames.extend(points_i)\n if not frames:\n # There were no shapes found, so set `frames` list to `None` to allow\n # an empty data frame to be created.\n frames = None\n return pd.DataFrame(frames, columns=attribs + ['vertex_i', 'x', 'y'])", "response": "Convert a SVG file to a pandas. DataFrame."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef svg_polygons_to_df(svg_source, xpath='//svg:polygon',\n namespaces=INKSCAPE_NSMAP):\n '''\n Construct a data frame with one row per vertex for all shapes (e.g.,\n ``svg:path``, ``svg:polygon``) in :data:`svg_source``.\n\n Arguments\n ---------\n svg_source : str or file-like\n A file path, URI, or file-like object.\n xpath : str, optional\n XPath path expression to select shape nodes.\n namespaces : dict, optional\n Key/value mapping of XML namespaces.\n\n Returns\n -------\n pandas.DataFrame\n Frame with one row per vertex for all shapes in :data:`svg_source`,\n with the following columns:\n - ``path_id``: The ``id`` attribute of the corresponding shape.\n - ``vertex_i``: The index of the vertex within the corresponding\n shape.\n - ``x``: The x-coordinate of the vertex.\n - ``y``: The y-coordinate of the vertex.\n\n\n .. note:: Deprecated in :mod:`svg_model` 0.5.post10\n :func:`svg_polygons_to_df` will be removed in :mod:`svg_model` 1.0, it\n is replaced by :func:`svg_shapes_to_df` because the latter is more\n general and works with ``svg:path`` and ``svg:polygon`` elements.\n '''\n warnings.warn(\"The `svg_polygons_to_df` function is deprecated. Use \"\n \"`svg_shapes_to_df` instead.\")\n result = svg_shapes_to_df(svg_source, xpath=xpath, namespaces=namespaces)\n return result[['id', 'vertex_i', 'x', 'y']].rename(columns={'id':\n 'path_id'})", "response": "Convert a SVG file to a pandas. DataFrame."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef compute_shape_centers(df_shapes, shape_i_column, inplace=False):\n '''\n Compute the center point of each polygon shape, and the offset of each\n vertex to the corresponding polygon center point.\n\n Parameters\n ----------\n df_shapes : pandas.DataFrame\n Table of polygon shape vertices (one row per vertex).\n\n Must have at least the following columns:\n - ``vertex_i``: The index of the vertex within the corresponding\n shape.\n - ``x``: The x-coordinate of the vertex.\n - ``y``: The y-coordinate of the vertex.\n shape_i_column : str or list, optional\n Table rows with the same value in the :data:`shape_i_column` column are\n grouped together as a shape.\n in_place : bool, optional\n If ``True``, center coordinate columns are added directly to the input\n frame.\n\n Otherwise, center coordinate columns are added to copy of the input\n frame.\n\n Returns\n -------\n pandas.DataFrame\n Input frame with the following additional columns:\n - ``x_center``/``y_center``: Absolute coordinates of shape center.\n - ``x_center_offset``/``y_center_offset``:\n * Coordinates of each vertex coordinate relative to shape center.\n '''\n if not isinstance(shape_i_column, bytes):\n raise KeyError('Shape index must be a single column.')\n\n if not inplace:\n df_shapes = df_shapes.copy()\n\n # Get coordinates of center of each path.\n df_bounding_boxes = get_bounding_boxes(df_shapes, shape_i_column)\n path_centers = (df_bounding_boxes[['x', 'y']] + .5 *\n df_bounding_boxes[['width', 'height']].values)\n df_shapes['x_center'] = path_centers.x[df_shapes[shape_i_column]].values\n df_shapes['y_center'] = path_centers.y[df_shapes[shape_i_column]].values\n\n # Calculate coordinate of each path vertex relative to center point of\n # path.\n center_offset = (df_shapes[['x', 'y']] -\n df_shapes[['x_center', 'y_center']].values)\n return df_shapes.join(center_offset, rsuffix='_center_offset')", "response": "Compute the center point of each polygon shape and the offset of each polygon center point."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef scale_points(df_points, scale=INKSCAPE_PPmm.magnitude, inplace=False):\n '''\n Translate points such that bounding box is anchored at (0, 0) and scale\n ``x`` and ``y`` columns of input frame by specified :data:`scale`.\n\n Parameters\n ----------\n df_points : pandas.DataFrame\n Table of ``x``/``y`` point positions.\n\n Must have at least the following columns:\n - ``x``: x-coordinate\n - ``y``: y-coordinate\n scale : float, optional\n Factor to scale points by.\n\n By default, scale to millimeters based on Inkscape default of 90\n pixels-per-inch.\n scale : float, optional\n Factor to scale points by.\n in_place : bool, optional\n If ``True``, input frame will be modified.\n\n Otherwise, the scaled points are written to a new frame, leaving the\n input frame unmodified.\n\n Returns\n -------\n pandas.DataFrame\n Input frame with the points translated such that bounding box is\n anchored at (0, 0) and ``x`` and ``y`` values scaled by specified\n :data:`scale`.\n '''\n if not inplace:\n df_points = df_points.copy()\n\n # Offset device, such that all coordinates are >= 0.\n df_points.x -= df_points.x.min()\n df_points.y -= df_points.y.min()\n\n # Scale path coordinates.\n df_points.x /= scale\n df_points.y /= scale\n\n return df_points", "response": "Scale the points of the base - of - mass of the input frame by the specified scale."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef scale_to_fit_a_in_b(a_shape, b_shape):\n '''\n Return scale factor (scalar float) to fit `a_shape` into `b_shape` while\n maintaining aspect ratio.\n\n Arguments\n ---------\n a_shape, b_shape : pandas.Series\n Input shapes containing numeric `width` and `height` values.\n\n Returns\n -------\n float\n Scale factor to fit :data:`a_shape` into :data:`b_shape` while\n maintaining aspect ratio.\n '''\n # Normalize the shapes to allow comparison.\n a_shape_normal = a_shape / a_shape.max()\n b_shape_normal = b_shape / b_shape.max()\n\n if a_shape_normal.width > b_shape_normal.width:\n a_shape_normal *= b_shape_normal.width / a_shape_normal.width\n\n if a_shape_normal.height > b_shape_normal.height:\n a_shape_normal *= b_shape_normal.height / a_shape_normal.height\n\n return a_shape_normal.max() * b_shape.max() / a_shape.max()", "response": "Returns a scale factor to fit a_shape into b_shape while a_shape maintaining aspect ratio."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nfitting the points in a base - box to the given bounding box.", "response": "def fit_points_in_bounding_box(df_points, bounding_box, padding_fraction=0):\n '''\n Return data frame with ``x``, ``y`` columns scaled to fit points from\n :data:`df_points` to fill :data:`bounding_box` while maintaining aspect\n ratio.\n\n Arguments\n ---------\n df_points : pandas.DataFrame\n A frame with at least the columns ``x`` and ``y``, containing one row\n per point.\n bounding_box: pandas.Series\n A `pandas.Series` containing numeric `width` and `height` values.\n padding_fraction : float\n Fraction of padding to add around points.\n\n Returns\n -------\n pandas.DataFrame\n Input frame with the points with ``x`` and ``y`` values scaled to fill\n :data:`bounding_box` while maintaining aspect ratio.\n '''\n df_scaled_points = df_points.copy()\n offset, padded_scale = fit_points_in_bounding_box_params(df_points,\n bounding_box,\n padding_fraction)\n df_scaled_points[['x', 'y']] *= padded_scale\n df_scaled_points[['x', 'y']] += offset\n return df_scaled_points"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef fit_points_in_bounding_box_params(df_points, bounding_box,\n padding_fraction=0):\n '''\n Return offset and scale factor to scale ``x``, ``y`` columns of\n :data:`df_points` to fill :data:`bounding_box` while maintaining aspect\n ratio.\n\n Arguments\n ---------\n df_points : pandas.DataFrame\n A frame with at least the columns ``x`` and ``y``, containing one row\n per point.\n bounding_box: pandas.Series\n A `pandas.Series` containing numeric `width` and `height` values.\n padding_fraction : float\n Fraction of padding to add around points.\n\n Returns\n -------\n (offset, scale) : (pandas.Series, float)\n Offset translation and scale required to fit all points in\n :data:`df_points` to fill :data:`bounding_box` while maintaining aspect\n ratio.\n\n :data:`offset` contains ``x`` and ``y`` values for the offset.\n '''\n width = df_points.x.max()\n height = df_points.y.max()\n\n points_bbox = pd.Series([width, height], index=['width', 'height'])\n fill_scale = 1 - 2 * padding_fraction\n assert(fill_scale > 0)\n\n scale = scale_to_fit_a_in_b(points_bbox, bounding_box)\n\n padded_scale = scale * fill_scale\n offset = .5 * (bounding_box - points_bbox * padded_scale)\n offset.index = ['x', 'y']\n return offset, padded_scale", "response": "Fits all points in a base box to a bounding box."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nremoves a layer from SVG document.", "response": "def remove_layer(svg_source, layer_name):\n '''\n Remove layer(s) from SVG document.\n\n Arguments\n ---------\n svg_source : str or file-like\n A file path, URI, or file-like object.\n layer_name : str or list\n Layer name or list of layer names to remove from SVG document.\n\n Returns\n -------\n StringIO.StringIO\n File-like object containing XML document with layer(s) removed.\n '''\n # Parse input file.\n xml_root = lxml.etree.parse(svg_source)\n svg_root = xml_root.xpath('/svg:svg', namespaces=INKSCAPE_NSMAP)[0]\n\n if isinstance(layer_name, str):\n layer_name = [layer_name]\n\n for layer_name_i in layer_name:\n # Remove existing layer from source, in-memory XML (source file remains\n # unmodified).\n layer_xpath = '//svg:g[@inkscape:label=\"%s\"]' % layer_name_i\n layer_groups = svg_root.xpath(layer_xpath, namespaces=INKSCAPE_NSMAP)\n\n if layer_groups:\n for g in layer_groups:\n g.getparent().remove(g)\n\n # Write result to `StringIO`.\n output = StringIO.StringIO()\n xml_root.write(output)\n output.seek(0)\n return output"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef load_eggs(entry_point_name):\n\n def _load_eggs():\n distributions, errors = working_set.find_plugins(Environment())\n for dist in distributions:\n # pylint: disable=unsupported-membership-test\n if dist not in working_set:\n LOGGER.debug('Adding plugin %s from %s', dist, dist.location)\n working_set.add(dist)\n\n def _log_error(item, err):\n if isinstance(err, DistributionNotFound):\n LOGGER.debug('Skipping \"%s\": (\"%s\" not found)', item, err)\n elif isinstance(err, VersionConflict):\n LOGGER.error('Skipping \"%s\": (version conflict \"%s\")', item, err)\n elif isinstance(err, UnknownExtra):\n LOGGER.error('Skipping \"%s\": (unknown extra \"%s\")', item, err)\n else:\n LOGGER.error('Skipping \"%s\": %s', item, err)\n\n for dist, err in errors.items():\n _log_error(dist, err)\n\n for entry in sorted(\n working_set.iter_entry_points(entry_point_name),\n key=lambda entry: entry.name,\n ):\n try:\n entry.load(require=True)\n except Exception as exc: # pylint: disable=broad-except\n _log_error(entry, exc)\n\n return _load_eggs", "response": "Loader that loads any eggs in sys. path."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nreturn the ResfDecl expression of the citation scheme", "response": "def refsDecl(self):\n \"\"\" ResfDecl expression of the citation scheme\n\n :rtype: str\n :Example: /tei:TEI/tei:text/tei:body/tei:div//tei:l[@n='$1']\n \"\"\"\n for refsDecl in self.graph.objects(self.asNode(), RDF_NAMESPACES.TEI.replacementPattern):\n return str(refsDecl)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef file(self, value):\n\n if value is not None:\n assert type(value) is unicode, \"'{0}' attribute: '{1}' type is not 'unicode'!\".format(\"file\", value)\n self.__file = value", "response": "Sets the file attribute."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nsets the directory attribute.", "response": "def directory(self, value):\n \"\"\"\n Setter for **self.__directory** attribute.\n\n :param value: Attribute value.\n :type value: unicode\n \"\"\"\n\n if value is not None:\n assert type(value) is unicode, \"'{0}' attribute: '{1}' type is not 'unicode'!\".format(\n \"directory\", value)\n assert os.path.exists(value), \"'{0}' attribute: '{1}' directory doesn't exists!\".format(\"directory\", value)\n self.__directory = value"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef require(self, value):\n\n if value is not None:\n assert type(value) in (tuple, list), \"'{0}' attribute: '{1}' type is not 'tuple' or 'list'!\".format(\n \"require\", value)\n self.__require = value", "response": "Setter for **self. __require** attribute."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nsetting the module attribute.", "response": "def module(self, value):\n \"\"\"\n Setter for **self.__module** attribute.\n\n :param value: Attribute value.\n :type value: ModuleType\n \"\"\"\n\n if value is not None:\n assert type(value) is type(sys), \"'{0}' attribute: '{1}' type is not 'module'!\".format(\"module\", value)\n self.__module = value"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nsetting the category of the object.", "response": "def category(self, value):\n \"\"\"\n Setter for **self.__category** attribute.\n\n :param value: Attribute value.\n :type value: unicode\n \"\"\"\n\n if value is not None:\n assert type(value) is unicode, \"'{0}' attribute: '{1}' type is not 'unicode'!\".format(\n \"category\", value)\n self.__category = value"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nsets the title of the object.", "response": "def title(self, value):\n \"\"\"\n Setter for **self.__title** attribute.\n\n :param value: Attribute value.\n :type value: unicode\n \"\"\"\n\n if value is not None:\n assert type(value) is unicode, \"'{0}' attribute: '{1}' type is not 'unicode'!\".format(\n \"title\", value)\n self.__title = value"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nsetting the package of the resource.", "response": "def package(self, value):\n \"\"\"\n Setter for **self.__package** attribute.\n\n :param value: Attribute value.\n :type value: unicode\n \"\"\"\n\n if value is not None:\n assert type(value) is unicode, \"'{0}' attribute: '{1}' type is not 'unicode'!\".format(\n \"package\", value)\n self.__package = value"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef version(self, value):\n\n if value is not None:\n assert type(value) is unicode, \"'{0}' attribute: '{1}' type is not 'unicode'!\".format(\n \"version\", value)\n self.__version = value", "response": "Sets the version of the object."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nsetting the author of the object.", "response": "def author(self, value):\n \"\"\"\n Setter for **self.__author** attribute.\n\n :param value: Attribute value.\n :type value: unicode\n \"\"\"\n\n if value is not None:\n assert type(value) is unicode, \"'{0}' attribute: '{1}' type is not 'unicode'!\".format(\n \"author\", value)\n self.__author = value"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nset the email of the log entry.", "response": "def email(self, value):\n \"\"\"\n Setter for **self.__email** attribute.\n\n :param value: Attribute value.\n :type value: unicode\n \"\"\"\n\n if value is not None:\n assert type(value) is unicode, \"'{0}' attribute: '{1}' type is not 'unicode'!\".format(\n \"email\", value)\n self.__email = value"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef url(self, value):\n\n if value is not None:\n assert type(value) is unicode, \"'{0}' attribute: '{1}' type is not 'unicode'!\".format(\n \"url\", value)\n self.__url = value", "response": "Sets the url of the resource."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef description(self, value):\n\n if value is not None:\n assert type(value) is unicode, \"'{0}' attribute: '{1}' type is not 'unicode'!\".format(\n \"description\", value)\n self.__description = value", "response": "Setter for **self. __description** attribute."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef initializeProfile(self):\n\n LOGGER.debug(\"> Building '{0}' profile.\".format(self.__file))\n\n sections_file_parser = SectionsFileParser(self.__file)\n sections_file_parser.parse()\n\n if sections_file_parser.sections:\n fileStructureParsingError = lambda attribute: foundations.exceptions.FileStructureParsingError(\n \"{0} | No '{1}' attribute found, '{2}' file structure seems invalid!\".format(\n self.__class__.__name__, attribute, self.__file))\n\n self.__directory = os.path.dirname(self.__file)\n self.__name = sections_file_parser.get_value(\"Name\", \"Component\", default=None)\n if self.__name is None:\n raise fileStructureParsingError(\"Name\")\n\n self.__title = sections_file_parser.get_value(\"Title\", \"Component\", default=None)\n if self.__title is None:\n self.__title = self.__name\n\n self.__package = sections_file_parser.get_value(\"Module\", \"Component\", default=None)\n if self.__package is None:\n raise fileStructureParsingError(\"Module\")\n\n self.__attribute = sections_file_parser.get_value(\"Object\", \"Component\", default=None)\n if self.__attribute is None:\n raise fileStructureParsingError(\"Object\")\n\n self.__require = sections_file_parser.get_value(\"Require\", \"Component\", default=None)\n self.__require = list() if self.__require is None else self.__require.split(\"|\")\n\n self.__version = sections_file_parser.get_value(\"Version\", \"Component\", default=None)\n if self.__version is None:\n raise fileStructureParsingError(\"Version\")\n\n self.__author = sections_file_parser.get_value(\"Author\", \"Informations\", default=None)\n\n self.__email = sections_file_parser.get_value(\"Email\", \"Informations\", default=None)\n\n self.__url = sections_file_parser.get_value(\"Url\", \"Informations\", default=None)\n\n self.__description = sections_file_parser.get_value(\"Description\", \"Informations\", default=None)\n\n return True\n else:\n raise foundations.exceptions.FileStructureParsingError(\n \"{0} | No sections found, '{1}' file structure seems invalid!\".format(self.__class__.__name__,\n self.__file))", "response": "Initializes the Component Profile."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef paths(self, value):\n\n if value is not None:\n assert type(value) in (tuple, list), \"'{0}' attribute: '{1}' type is not 'tuple' or 'list'!\".format(\n \"paths\", value)\n for path in value:\n assert type(path) is unicode, \"'{0}' attribute: '{1}' type is not 'unicode'!\".format(\n \"paths\", path)\n assert os.path.exists(path), \"'{0}' attribute: '{1}' directory doesn't exists!\".format(\"paths\", path)\n self.__paths = value", "response": "Setter for **self. paths ** attribute."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef extension(self, value):\n\n if value is not None:\n assert type(value) is unicode, \"'{0}' attribute: '{1}' type is not 'unicode'!\".format(\n \"extension\", value)\n self.__extension = value", "response": "Setter for the extension attribute."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef categories(self, value):\n\n if value is not None:\n assert type(value) is dict, \"'{0}' attribute: '{1}' type is not 'dict'!\".format(\"categories\", value)\n for key in value:\n assert type(key) is unicode, \"'{0}' attribute: '{1}' type is not 'unicode'!\".format(\n \"categories\", key)\n self.__categories = value", "response": "Set the categories of the orphan in the database."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef register_component(self, path):\n\n component = foundations.strings.get_splitext_basename(path)\n LOGGER.debug(\"> Current Component: '{0}'.\".format(component))\n profile = Profile(file=path)\n if profile.initializeProfile():\n if os.path.isfile(os.path.join(profile.directory, profile.package) + \".py\") or \\\n os.path.isdir(os.path.join(profile.directory, profile.package)) or \\\n os.path.basename(profile.directory) == profile.package:\n self.__components[profile.name] = profile\n return True\n else:\n raise manager.exceptions.ComponentModuleError(\n \"{0} | '{1}' has no associated module and has been rejected!\".format(self.__class__.__name__,\n component))\n else:\n raise manager.exceptions.ComponentProfileError(\n \"{0} | '{1}' is not a valid Component and has been rejected!\".format(self.__class__.__name__,\n component))", "response": "Registers a Component using given path."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nregisters the Components. Usage:: >>> manager = Manager((\"./manager/tests/tests_manager/resources/components/core\",)) >>> manager.register_components() True >>> manager.components.keys() [u'core.tests_component_a', u'core.tests_component_b'] :return: Method success. :rtype: bool", "response": "def register_components(self):\n \"\"\"\n Registers the Components.\n\n Usage::\n\n >>> manager = Manager((\"./manager/tests/tests_manager/resources/components/core\",))\n >>> manager.register_components()\n True\n >>> manager.components.keys()\n [u'core.tests_component_a', u'core.tests_component_b']\n\n :return: Method success.\n :rtype: bool\n \"\"\"\n\n unregistered_components = []\n for path in self.paths:\n for file in foundations.walkers.files_walker(path, (\"\\.{0}$\".format(self.__extension),), (\"\\._\",)):\n if not self.register_component(file):\n unregistered_components.append(file)\n\n if not unregistered_components:\n return True\n else:\n raise manager.exceptions.ComponentRegistrationError(\n \"{0} | '{1}' Components failed to register!\".format(self.__class__.__name__,\n \", \".join(unregistered_components)))"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nreload given Component module.", "response": "def reload_component(self, component):\n \"\"\"\n Reload given Component module.\n\n Usage::\n\n >>> manager = Manager()\n >>> manager.register_component(\"tests_component_a.rc\")\n True\n >>> manager.instantiate_component(\"core.tests_component_a\")\n True\n >>> manager.get_interface(\"core.tests_component_a\")\n \n >>> manager.reload_component(\"core.tests_component_a\")\n True\n >>> manager.get_interface(\"core.tests_component_a\")\n \n\n :param component: Component name.\n :type component: unicode\n :return: Reload success.\n :rtype: bool\n \"\"\"\n\n dependents = list(reversed(self.list_dependents(component)))\n dependents.append(component)\n\n for dependent in dependents:\n profile = self.__components[dependent]\n module = __import__(profile.package)\n reload(module)\n object = profile.attribute in dir(module) and getattr(module, profile.attribute) or None\n if object and inspect.isclass(object):\n for type in self.__categories.itervalues():\n if type.__name__ in (base.__name__ for base in object.__bases__):\n instance = object(name=profile.name)\n profile.module = module\n profile.interface = instance\n LOGGER.info(\"{0} | '{1}' Component has been reloaded!\".format(\n self.__class__.__name__, profile.name))\n return True"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nlist the Components by dependency resolving.", "response": "def list_components(self, dependency_order=True):\n \"\"\"\n Lists the Components by dependency resolving.\n\n Usage::\n\n >>> manager = Manager((\"./manager/tests/tests_manager/resources/components/core\",))\n >>> manager.register_components()\n True\n >>> manager.list_components()\n [u'core.tests_component_a', u'core.tests_component_b']\n\n :param dependency_order: Components are returned by dependency order.\n :type dependency_order: bool\n \"\"\"\n\n if dependency_order:\n return list(itertools.chain.from_iterable([sorted(list(batch)) for batch in\n foundations.common.dependency_resolver(\n dict((key, value.require) for (key, value) in self))]))\n else:\n return [key for (key, value) in self]"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef list_dependents(self, component, dependents=None):\n\n dependents = set() if dependents is None else dependents\n for name, profile in self:\n if not component in profile.require:\n continue\n\n dependents.add(name)\n self.list_dependents(name, dependents)\n\n return sorted(list(dependents), key=(self.list_components()).index)", "response": "Lists given Component dependents Components."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nfilter the Components using given regex pattern.", "response": "def filter_components(self, pattern, category=None):\n \"\"\"\n Filters the Components using given regex pattern.\n\n Usage::\n\n >>> manager = Manager((\"./manager/tests/tests_manager/resources/components/core\",))\n >>> manager.register_components()\n True\n >>> manager.filter_components(\"\\w+A$\")\n [u'core.tests_component_a']\n\n :param pattern: Regex filtering pattern.\n :type pattern: unicode\n :param category: Category filter.\n :type category: unicode\n :return: Matching Components.\n :rtype: list\n \"\"\"\n\n filtered_components = []\n for component, profile in self:\n if category:\n if profile.category != category:\n continue\n\n if re.search(pattern, component):\n filtered_components.append(component)\n return filtered_components"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef get_profile(self, component):\n\n components = self.filter_components(r\"^{0}$\".format(component))\n if components != []:\n return self.__components[foundations.common.get_first_item(components)]", "response": "Gets given Component profile."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef get_interface(self, component):\n\n profile = self.get_profile(component)\n if profile:\n return profile.interface", "response": "Gets given Component interface."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nget given Component attribute name.", "response": "def get_component_attribute_name(component):\n \"\"\"\n Gets given Component attribute name.\n\n Usage::\n\n >>> Manager.get_component_attribute_name(\"factory.components_manager_ui\")\n u'factoryComponentsManagerUi'\n\n :param component: Component to get the attribute name.\n :type component: unicode\n :return: Component attribute name.\n :rtype: object\n \"\"\"\n\n search = re.search(r\"(?P\\w+)\\.(?P\\w+)\", component)\n if search:\n name = \"{0}{1}{2}\".format(\n search.group(\"category\"), search.group(\"name\")[0].upper(), search.group(\"name\")[1:])\n LOGGER.debug(\"> Component name: '{0}' to attribute name Active_QLabel: '{1}'.\".format(component, name))\n else:\n name = component\n return name"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\ncompiles the properties of the object to a table.", "response": "def output_to_table(obj, olist='inputs', oformat='latex', table_ends=False, prefix=\"\"):\n \"\"\"\n Compile the properties to a table.\n\n :param olist: list, Names of the parameters to be in the output table\n :param oformat: str, The type of table to be output\n :param table_ends: bool, Add ends to the table\n :param prefix: str, A string to be added to the start of each parameter name\n :return: para, str, table as a string\n \"\"\"\n para = \"\"\n property_list = []\n if olist == 'inputs':\n property_list = obj.inputs\n elif olist == 'all':\n for item in obj.__dict__:\n if \"_\" != item[0]:\n property_list.append(item)\n for item in property_list:\n if hasattr(obj, item):\n value = getattr(obj, item)\n value_str = format_value(value)\n if oformat == \"latex\":\n delimeter = \" & \"\n else:\n delimeter = \",\"\n para += \"{0}{1}{2}\\\\\\\\\\n\".format(prefix + format_name(item), delimeter, value_str)\n if table_ends:\n para = add_table_ends(para, oformat)\n return para"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef format_value(value, sf=3):\n if isinstance(value, str):\n return value\n\n elif isinstance(value, list) or isinstance(value, np.ndarray):\n value = list(value)\n for i in range(len(value)):\n vv = format_value(value[i])\n value[i] = vv\n return \"[\" + \", \".join(value) + \"]\"\n\n elif value is None:\n return \"N/A\"\n\n else:\n fmt_str = \"{0:.%ig}\" % sf\n return fmt_str.format(value)", "response": "convert a parameter value into a formatted string with certain significant figures\n "} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nadding the latex table ends to the end of the latex table.", "response": "def add_table_ends(para, oformat='latex', caption=\"caption-text\", label=\"table\"):\n \"\"\"\n Adds the latex table ends\n\n :param para:\n :param oformat:\n :param caption:\n :param label:\n :return:\n \"\"\"\n fpara = \"\"\n if oformat == 'latex':\n fpara += \"\\\\begin{table}[H]\\n\"\n fpara += \"\\\\centering\\n\"\n fpara += \"\\\\begin{tabular}{cc}\\n\"\n fpara += \"\\\\toprule\\n\"\n fpara += \"Parameter & Value \\\\\\\\\\n\"\n fpara += \"\\\\midrule\\n\"\n fpara += para\n fpara += \"\\\\bottomrule\\n\"\n fpara += \"\\\\end{tabular}\\n\"\n fpara += \"\\\\caption{%s \\label{tab:%s}}\\n\" % (caption, label)\n fpara += \"\\\\end{table}\\n\\n\"\n return fpara"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef draw_shapes_svg_layer(df_shapes, shape_i_columns, layer_name,\n layer_number=1, use_svg_path=True):\n '''\n Draw shapes as a layer in a SVG file.\n\n Args:\n\n df_shapes (pandas.DataFrame): Table of shape vertices (one row per\n vertex).\n shape_i_columns (str or list) : Either a single column name as a string\n or a list of column names in ``df_shapes``. Rows in ``df_shapes``\n with the same value in the ``shape_i_columns`` column(s) are\n grouped together as a shape.\n layer_name (str) : Name of Inkscape layer.\n layer_number (int, optional) : Z-order index of Inkscape layer.\n use_svg_path (bool, optional) : If ``True``, electrodes are drawn as\n ``svg:path`` elements. Otherwise, electrodes are drawn as\n ``svg:polygon`` elements.\n\n Returns\n -------\n StringIO.StringIO\n A file-like object containing SVG XML source.\n\n The XML contains a layer named according to :data:`layer_name`, which\n in turn contains ``svg:polygon`` or ``svg:path`` elements corresponding\n to the shapes in the input :data:`df_shapes` table.\n '''\n # Note that `svgwrite.Drawing` requires a filepath to be specified during\n # construction, *but* nothing is actually written to the path unless one of\n # the `save*` methods is called.\n #\n # In this function, we do *not* call any of the `save*` methods. Instead,\n # we use the `write` method to write to an in-memory file-like object.\n minx, miny = df_shapes[['x', 'y']].min().values\n maxx, maxy = df_shapes[['x', 'y']].max().values\n width = maxx - minx\n height = maxy - miny\n\n dwg = svgwrite.Drawing('should_not_exist.svg', size=(width, height),\n debug=False)\n\n nsmap = INKSCAPE_NSMAP\n\n dwg.attribs['xmlns:inkscape'] = nsmap['inkscape']\n\n svg_root = dwg.g(id='layer%d' % layer_number,\n **{'inkscape:label': layer_name,\n 'inkscape:groupmode': 'layer'})\n\n minx, miny = df_shapes[['x', 'y']].min().values\n\n for shape_i, df_shape_i in df_shapes.groupby(shape_i_columns):\n attr_columns = [c for c in df_shape_i.columns\n if c not in ('vertex_i', 'x', 'y')]\n attrs = df_shape_i.iloc[0][attr_columns].to_dict()\n vertices = df_shape_i[['x', 'y']].values.tolist()\n if not use_svg_path:\n # Draw electrode shape as an `svg:polygon` element.\n p = Polygon(vertices, debug=False, **attrs)\n else:\n # Draw electrode shape as an `svg:path` element.\n commands = ['M %s,%s' % tuple(vertices[0])]\n commands += ['L %s,%s' % tuple(v) for v in vertices[1:]]\n while vertices[0] == vertices[-1]:\n # Start is equal to end of path, but we will use the `'Z'`\n # command to close the path, so delete the last point in the\n # path.\n del vertices[-1]\n commands += ['Z']\n p = Path_(d=' '.join(commands), debug=False, **attrs)\n svg_root.add(p)\n dwg.add(svg_root)\n\n # Write result to `StringIO`.\n output = StringIO.StringIO()\n dwg.write(output)\n output.seek(0)\n\n return output", "response": "Draw shapes as a layer in a SVG file."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef draw_lines_svg_layer(df_endpoints, layer_name, layer_number=1):\n '''\n Draw lines defined by endpoint coordinates as a layer in a SVG file.\n\n Args:\n\n df_endpoints (pandas.DataFrame) : Each row corresponds to the endpoints\n of a single line, encoded through the columns: ``x_source``,\n ``y_source``, ``x_target``, and ``y_target``.\n layer_name (str) : Name of Inkscape layer.\n layer_number (int, optional) : Z-order index of Inkscape layer.\n\n Returns\n -------\n StringIO.StringIO\n A file-like object containing SVG XML source.\n\n The XML contains a layer named ``\"Connections\"``, which in turn\n contains one line per row in the input :data:`df_endpoints` table.\n '''\n # Note that `svgwrite.Drawing` requires a filepath to be specified during\n # construction, *but* nothing is actually written to the path unless one of\n # the `save*` methods is called.\n #\n # In this function, we do *not* call any of the `save*` methods. Instead,\n # we use the `write` method to write to an in-memory file-like object.\n dwg = svgwrite.Drawing('should_not_exist.svg', profile='tiny', debug=False)\n\n dwg.attribs['width'] = df_endpoints[['x_source', 'x_target']].values.max()\n dwg.attribs['height'] = df_endpoints[['y_source', 'y_target']].values.max()\n\n nsmap = INKSCAPE_NSMAP\n\n dwg.attribs['xmlns:inkscape'] = nsmap['inkscape']\n\n coord_columns = ['x_source', 'y_source', 'x_target', 'y_target']\n\n line_layer = dwg.g(id='layer%d' % layer_number,\n **{'inkscape:label': layer_name,\n 'inkscape:groupmode': 'layer'})\n\n for i, (x1, y1, x2, y2) in df_endpoints[coord_columns].iterrows():\n line_i = dwg.line((x1, y1), (x2, y2), id='line%d' % i,\n style='stroke:#000000; stroke-width:0.1;')\n line_layer.add(line_i)\n dwg.add(line_layer)\n\n output = StringIO.StringIO()\n dwg.write(output)\n # Rewind file.\n output.seek(0)\n return output", "response": "Draw lines defined by endpoint coordinates as a layer in a SVG file."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\ncreate a DTS Error Response with information about the error", "response": "def dts_error(self, error_name, message=None):\n \"\"\" Create a DTS Error reply\n\n :param error_name: Name of the error\n :param message: Message of the Error\n :return: DTS Error Response with information (JSON)\n \"\"\"\n self.nautilus_extension.logger.info(\"DTS error thrown {} for {} ({})\".format(\n error_name, request.path, message\n ))\n j = jsonify({\n \"error\": error_name,\n \"message\": message\n })\n j.status_code = 404\n return j"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nreturning DTS Collection Metadata reply for given objectId", "response": "def r_dts_collection(self, objectId=None):\n \"\"\" DTS Collection Metadata reply for given objectId\n\n :param objectId: Collection Identifier\n :return: JSON Format of DTS Collection\n \"\"\"\n try:\n j = self.resolver.getMetadata(objectId=objectId).export(Mimetypes.JSON.DTS.Std)\n j = jsonify(j)\n j.status_code = 200\n except NautilusError as E:\n return self.dts_error(error_name=E.__class__.__name__, message=E.__doc__)\n return j"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef create_run(cmd, project, exp, grp):\n from benchbuild.utils import schema as s\n\n session = s.Session()\n run = s.Run(\n command=str(cmd),\n project_name=project.name,\n project_group=project.group,\n experiment_name=exp,\n run_group=str(grp),\n experiment_group=project.experiment.id)\n session.add(run)\n session.commit()\n\n return (run, session)", "response": "Create a new run in the database and return it and the session opened."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\ncreating a new run_group in the database and return both the newly created run_group and the session object.", "response": "def create_run_group(prj):\n \"\"\"\n Create a new 'run_group' in the database.\n\n This creates a new transaction in the database and creates a new run_group\n within this transaction. Afterwards we return both the transaction as well\n as the run_group itself. The user is responsible for committing it when the\n time comes.\n\n Args:\n prj - The project for which we open the run_group.\n\n Returns:\n A tuple (group, session) containing both the newly created run_group and\n the transaction object.\n \"\"\"\n from benchbuild.utils import schema as s\n\n session = s.Session()\n experiment = prj.experiment\n group = s.RunGroup(id=prj.run_uuid, experiment=experiment.id)\n session.add(group)\n session.commit()\n\n return (group, session)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\npersisting this project in the benchbuild database.", "response": "def persist_project(project):\n \"\"\"\n Persist this project in the benchbuild database.\n\n Args:\n project: The project we want to persist.\n \"\"\"\n from benchbuild.utils.schema import Project, Session\n session = Session()\n projects = session.query(Project) \\\n .filter(Project.name == project.name) \\\n .filter(Project.group_name == project.group)\n\n name = project.name\n desc = project.__doc__\n domain = project.domain\n group_name = project.group\n version = project.version() \\\n if callable(project.version) else project.version\n try:\n src_url = project.src_uri\n except AttributeError:\n src_url = 'unknown'\n\n if projects.count() == 0:\n newp = Project()\n newp.name = name\n newp.description = desc\n newp.src_url = src_url\n newp.domain = domain\n newp.group_name = group_name\n newp.version = version\n session.add(newp)\n else:\n newp_value = {\n \"name\": name,\n \"description\": desc,\n \"src_url\": src_url,\n \"domain\": domain,\n \"group_name\": group_name,\n \"version\": version\n }\n projects.update(newp_value)\n\n session.commit()\n return (projects, session)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\npersist this experiment in the benchbuild database.", "response": "def persist_experiment(experiment):\n \"\"\"\n Persist this experiment in the benchbuild database.\n\n Args:\n experiment: The experiment we want to persist.\n \"\"\"\n from benchbuild.utils.schema import Experiment, Session\n\n session = Session()\n\n cfg_exp = experiment.id\n LOG.debug(\"Using experiment ID stored in config: %s\", cfg_exp)\n exps = session.query(Experiment).filter(Experiment.id == cfg_exp)\n desc = str(CFG[\"experiment_description\"])\n name = experiment.name\n\n if exps.count() == 0:\n newe = Experiment()\n newe.id = cfg_exp\n newe.name = name\n newe.description = desc\n session.add(newe)\n ret = newe\n else:\n exps.update({'name': name, 'description': desc})\n ret = exps.first()\n\n try:\n session.commit()\n except IntegrityError:\n session.rollback()\n persist_experiment(experiment)\n\n return (ret, session)"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\npersist the run results in the database.", "response": "def persist_time(run, session, timings):\n \"\"\"\n Persist the run results in the database.\n\n Args:\n run: The run we attach this timing results to.\n session: The db transaction we belong to.\n timings: The timing measurements we want to store.\n \"\"\"\n from benchbuild.utils import schema as s\n\n for timing in timings:\n session.add(\n s.Metric(name=\"time.user_s\", value=timing[0], run_id=run.id))\n session.add(\n s.Metric(name=\"time.system_s\", value=timing[1], run_id=run.id))\n session.add(\n s.Metric(name=\"time.real_s\", value=timing[2], run_id=run.id))"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\npersist the flamegraph in the database.", "response": "def persist_perf(run, session, svg_path):\n \"\"\"\n Persist the flamegraph in the database.\n\n The flamegraph exists as a SVG image on disk until we persist it in the\n database.\n\n Args:\n run: The run we attach these perf measurements to.\n session: The db transaction we belong to.\n svg_path: The path to the SVG file we want to store.\n \"\"\"\n from benchbuild.utils import schema as s\n\n with open(svg_path, 'r') as svg_file:\n svg_data = svg_file.read()\n session.add(\n s.Metadata(name=\"perf.flamegraph\", value=svg_data, run_id=run.id))"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef persist_compilestats(run, session, stats):\n for stat in stats:\n stat.run_id = run.id\n session.add(stat)", "response": "Persist the run results in the database."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\npersist the configuration in as key - value pairs.", "response": "def persist_config(run, session, cfg):\n \"\"\"\n Persist the configuration in as key-value pairs.\n\n Args:\n run: The run we attach the config to.\n session: The db transaction we belong to.\n cfg: The configuration we want to persist.\n \"\"\"\n from benchbuild.utils import schema as s\n\n for cfg_elem in cfg:\n session.add(\n s.Config(name=cfg_elem, value=cfg[cfg_elem], run_id=run.id))"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef parse_results_header(cls, header):\n header = IORMetricsExtractor.RE_MULTIPLE_SPACES.sub(' ', header)\n header = header.split(' ')\n return header", "response": "Extract columns from the line under Summary of all tests"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nextracting metrics from a result line", "response": "def parse_result_line(cls, columns, line, metrics):\n \"\"\"Extract metrics from a result line\n :param columns: list of columns specified under line\n \"Summary of all tests:\"\n :param line: string of results below the columns line\n :param metrics: output dict where metrics are written\n \"\"\"\n line = IORMetricsExtractor.RE_MULTIPLE_SPACES.sub(' ', line)\n line = line.split(' ')\n operation = line[0]\n assert len(line) == len(columns)\n for i, col in enumerate(line):\n col = columns[i]\n if col in cls.METAS_IGNORED:\n continue\n desc = cls.METAS.get(col)\n if desc is None:\n LOGGER.warn('Unrecognized column: %s', col)\n meta = cls.get_meta_name(operation, desc.get('name') or col)\n value = desc['metric'].type(line[i])\n metrics[meta] = value"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef apis(self):\n value = self.attributes['apis']\n if isinstance(value, six.string_types):\n value = shlex.split(value)\n return value", "response": "List of API to test"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nmake sure the named directory is created if possible", "response": "def pre_execute(self, execution, context):\n \"\"\"Make sure the named directory is created if possible\"\"\"\n path = self._fspath\n if path:\n path = path.format(\n benchmark=context.benchmark,\n api=execution['category'],\n **execution.get('metas', {})\n )\n if self.clean_path:\n shutil.rmtree(path, ignore_errors=True)\n if execution['metas']['file_mode'] == 'onefile':\n path = osp.dirname(path)\n if not osp.exists(path):\n os.makedirs(path)"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef sizes(self):\n if self.attributes.get('sizes'):\n for settings in self.attributes.get('sizes'):\n for pair in itertools.product(\n shlex.split(settings['block']), shlex.split(settings['transfer'])\n ):\n yield pair\n else:\n for pair in itertools.product(self.block_size, self.transfer_size):\n yield pair", "response": "Provides block_size and transfer_size through a list\n of dict for instance"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef file_mode(self):\n fms = self.attributes['file_mode']\n eax = set()\n if isinstance(fms, six.string_types):\n fms = shlex.split(fms)\n for fm in fms:\n if fm == 'both':\n eax.add('fpp')\n eax.add('onefile')\n elif fm in ['fpp', 'onefile']:\n eax.add(fm)\n else:\n raise Exception('Invalid IOR file mode: ' + fm)\n return eax", "response": "Returns a set of all file mode names that are set for this IOR."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef block_size(self):\n bs = self.attributes['block_size']\n if isinstance(bs, six.string_types):\n bs = shlex.split(bs)\n bs = [str(e) for e in bs]\n return bs", "response": "Contiguous bytes to write per task"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef transfer_size(self):\n ts = self.attributes['transfer_size']\n if isinstance(ts, six.string_types):\n ts = shlex.split(ts)\n ts = [str(e) for e in ts]\n return ts", "response": "Size of transfer in bytes"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef is_tablet_pad(self):\n\n\t\tif self in {type(self).TABLET_PAD_BUTTON, type(self).TABLET_PAD_RING,\n\t\t\ttype(self).TABLET_PAD_STRIP}:\n\t\t\treturn True\n\t\telse:\n\t\t\treturn False", "response": "Macro to check if this event is a tablet pad."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef is_gesture(self):\n\n\t\tif self in {type(self).GESTURE_SWIPE_BEGIN, type(self).GESTURE_SWIPE_END,\n\t\t\ttype(self).GESTURE_SWIPE_UPDATE, type(self).GESTURE_PINCH_BEGIN,\n\t\t\ttype(self).GESTURE_PINCH_UPDATE, type(self).GESTURE_PINCH_END}:\n\t\t\treturn True\n\t\telse:\n\t\t\treturn False", "response": "Macro to check if this event is a gesture event."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef delta_unaccelerated(self):\n\n\t\tif self.type != EventType.POINTER_MOTION:\n\t\t\traise AttributeError(_wrong_prop.format(self.type))\n\t\tdelta_x = self._libinput.libinput_event_pointer_get_dx_unaccelerated(\n\t\t\tself._handle)\n\t\tdelta_y = self._libinput.libinput_event_pointer_get_dy_unaccelerated(\n\t\t\tself._handle)\n\t\treturn delta_x, delta_y", "response": "The relative delta of the unaccelerated motion vector of the last pointer event."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef absolute_coords(self):\n\n\t\tif self.type != EventType.POINTER_MOTION_ABSOLUTE:\n\t\t\traise AttributeError(_wrong_prop.format(self.type))\n\t\tabs_x = self._libinput.libinput_event_pointer_get_absolute_x(\n\t\t\tself._handle)\n\t\tabs_y = self._libinput.libinput_event_pointer_get_absolute_y(\n\t\t\tself._handle)\n\t\treturn abs_x, abs_y", "response": "Returns the current absolute coordinates of the pointer event."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nreturn the current absolute coordinates of the pointer event and the current absolute coordinates transformed to screen coordinates.", "response": "def transform_absolute_coords(self, width, height):\n\t\t\"\"\"Return the current absolute coordinates of the pointer event,\n\t\ttransformed to screen coordinates.\n\n\t\tFor pointer events that are not of type\n\t\t:attr:`~libinput.constant.EventType.POINTER_MOTION_ABSOLUTE`,\n\t\tthis method raises :exc:`AttributeError`.\n\n\t\tArgs:\n\t\t\twidth (int): The current output screen width.\n\t\t\theight (int): The current output screen height.\n\t\tReturns:\n\t\t\t(float, float): The current absolute (x, y) coordinates transformed\n\t\t\tto a screen coordinates.\n\t\tRaises:\n\t\t\tAttributeError\n\t\t\"\"\"\n\n\t\tif self.type != EventType.POINTER_MOTION_ABSOLUTE:\n\t\t\traise AttributeError(_wrong_meth.format(self.type))\n\t\tabs_x = self._libinput \\\n\t\t\t.libinput_event_pointer_get_absolute_x_transformed(\n\t\t\t\tself._handle, width)\n\t\tabs_y = self._libinput \\\n\t\t\t.libinput_event_pointer_get_absolute_y_transformed(\n\t\t\t\tself._handle, height)\n\t\treturn abs_x, abs_y"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef button_state(self):\n\n\t\tif self.type != EventType.POINTER_BUTTON:\n\t\t\traise AttributeError(_wrong_prop.format(self.type))\n\t\treturn self._libinput.libinput_event_pointer_get_button_state(\n\t\t\tself._handle)", "response": "The button state that triggered this event."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\ncheck if the event contains a valid value for the given axis.", "response": "def has_axis(self, axis):\n\t\t\"\"\"Check if the event has a valid value for the given axis.\n\n\t\tIf this method returns True for an axis and :meth:`get_axis_value`\n\t\treturns a value of 0, the event is a scroll stop event.\n\n\t\tFor pointer events that are not of type\n\t\t:attr:`~libinput.constant.EventType.POINTER_AXIS`, this method raises\n\t\t:exc:`AttributeError`.\n\n\t\tArgs:\n\t\t\taxis (~libinput.constant.PointerAxis): The axis to check.\n\t\tReturns:\n\t\t\tbool: True if this event contains a value for this axis.\n\t\tRaises:\n\t\t\tAttributeError\n\t\t\"\"\"\n\n\t\tif self.type != EventType.POINTER_AXIS:\n\t\t\traise AttributeError(_wrong_meth.format(self.type))\n\t\treturn self._libinput.libinput_event_pointer_has_axis(\n\t\t\tself._handle, axis)"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nget the value of the given axis.", "response": "def get_axis_value(self, axis):\n\t\t\"\"\"Return the axis value of the given axis.\n\n\t\tThe interpretation of the value depends on the axis. For the two\n\t\tscrolling axes :attr:`~libinput.constant.PointerAxis.SCROLL_VERTICAL`\n\t\tand :attr:`~libinput.constant.PointerAxis.SCROLL_HORIZONTAL`, the value\n\t\tof the event is in relative scroll units, with the positive direction\n\t\tbeing down or right, respectively. For the interpretation of the value,\n\t\tsee :attr:`axis_source`.\n\n\t\tIf :meth:`has_axis` returns False for an axis, this method returns 0\n\t\tfor that axis.\n\n\t\tFor pointer events that are not of type\n\t\t:attr:`~libinput.constant.Event.POINTER_AXIS`, this method raises\n\t\t:exc:`AttributeError`.\n\n\t\tArgs:\n\t\t\taxis (~libinput.constant.PointerAxis): The axis who's value to get.\n\t\tReturns:\n\t\t\tfloat: The axis value of this event.\n\t\tRaises:\n\t\t\tAttributeError\n\t\t\"\"\"\n\n\t\tif self.type != EventType.POINTER_AXIS:\n\t\t\traise AttributeError(_wrong_meth.format(self.type))\n\t\treturn self._libinput.libinput_event_pointer_get_axis_value(\n\t\t\tself._handle, axis)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef get_axis_value_discrete(self, axis):\n\n\t\tif self.type != EventType.POINTER_AXIS:\n\t\t\traise AttributeError(_wrong_meth.format(self.type))\n\t\treturn self._libinput.libinput_event_pointer_get_axis_value_discrete(\n\t\t\tself._handle, axis)", "response": "Get the value of the given axis event in discrete steps."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef slot(self):\n\n\t\tif self.type == EventType.TOUCH_FRAME:\n\t\t\traise AttributeError(_wrong_prop.format(self.type))\n\t\treturn self._libinput.libinput_event_touch_get_slot(self._handle)", "response": "Returns the slot of this touch event."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nreturn the seat wide identifier of the touch event.", "response": "def seat_slot(self):\n\t\t\"\"\"The seat slot of the touch event.\n\n\t\tA seat slot is a non-negative seat wide unique identifier of an active\n\t\ttouch point.\n\n\t\tEvents from single touch devices will be represented as one individual\n\t\ttouch point per device.\n\n\t\tFor events not of type :attr:`~libinput.constant.EventType.TOUCH_DOWN`,\n\t\t:attr:`~libinput.constant.EventType.TOUCH_UP`,\n\t\t:attr:`~libinput.constant.EventType.TOUCH_MOTION` or\n\t\t:attr:`~libinput.constant.EventType.TOUCH_CANCEL`, this property\n\t\traises :exc:`AssertionError`.\n\n\t\tReturns:\n\t\t\tint: The seat slot of the touch event.\n\t\tRaises:\n\t\t\tAssertionError\n\t\t\"\"\"\n\n\t\tif self.type == EventType.TOUCH_FRAME:\n\t\t\traise AttributeError(_wrong_prop.format(self.type))\n\t\treturn self._libinput.libinput_event_touch_get_seat_slot(self._handle)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nreturn the current absolute coordinates of the touch event.", "response": "def coords(self):\n\t\t\"\"\"The current absolute coordinates of the touch event,\n\t\tin mm from the top left corner of the device.\n\n\t\tTo get the corresponding output screen coordinates, use\n\t\t:meth:`transform_coords`.\n\n\t\tFor events not of type :attr:`~libinput.constant.EventType.TOUCH_DOWN`,\n\t\t:attr:`~libinput.constant.EventType.TOUCH_MOTION`, this property\n\t\traises :exc:`AttributeError`.\n\n\t\tReturns:\n\t\t\t(float, float): The current absolute (x, y) coordinates.\n\t\tRaises:\n\t\t\tAttributeError\n\t\t\"\"\"\n\n\t\tif self.type not in {EventType.TOUCH_DOWN, EventType.TOUCH_MOTION}:\n\t\t\traise AttributeError(_wrong_prop.format(self.type))\n\t\tx = self._libinput.libinput_event_touch_get_x(self._handle)\n\t\ty = self._libinput.libinput_event_touch_get_y(self._handle)\n\t\treturn x, y"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef transform_coords(self, width, height):\n\n\t\tif self.type not in {EventType.TOUCH_DOWN, EventType.TOUCH_MOTION}:\n\t\t\traise AttributeError(_wrong_meth.format(self.type))\n\t\tx = self._libinput.libinput_event_touch_get_x_transformed(\n\t\t\tself._handle, width)\n\t\ty = self._libinput.libinput_event_touch_get_y_transformed(\n\t\t\tself._handle, height)\n\t\treturn x, y", "response": "Return the current absolute coordinates of the touch event transformed to screen coordinates."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nreturning if the gesture ended normally or if it was cancelled.", "response": "def cancelled(self):\n\t\t\"\"\"Return if the gesture ended normally, or if it was cancelled.\n\n\t\tFor gesture events that are not of type\n\t\t:attr:`~libinput.constant.EventType.GESTURE_SWIPE_END` or\n\t\t:attr:`~libinput.constant.EventType.GESTURE_PINCH_END`, this property\n\t\traises :exc:`AttributeError`.\n\n\t\tReturns:\n\t\t\tbool: :obj:`True` indicating that the gesture was cancelled.\n\t\tRaises:\n\t\t\tAttributeError\n\t\t\"\"\"\n\n\t\tif self.type not in {EventType.GESTURE_SWIPE_END,\n\t\t\t\tEventType.GESTURE_PINCH_END}:\n\t\t\traise AttributeError(_wrong_prop.format(self.type))\n\t\treturn self._libinput.libinput_event_gesture_get_cancelled(self._handle)"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef delta(self):\n\n\t\tif self.type not in {EventType.GESTURE_SWIPE_UPDATE,\n\t\t\t\tEventType.GESTURE_PINCH_UPDATE}:\n\t\t\traise AttributeError(_wrong_prop.format(self.type))\n\t\tdelta_x = self._libinput.libinput_event_gesture_get_dx(self._handle)\n\t\tdelta_y = self._libinput.libinput_event_gesture_get_dy(self._handle)\n\t\treturn delta_x, delta_y", "response": "The delta between the last event and the current event."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef delta_unaccelerated(self):\n\n\t\tif self.type not in {EventType.GESTURE_SWIPE_UPDATE,\n\t\t\t\tEventType.GESTURE_PINCH_UPDATE}:\n\t\t\traise AttributeError(_wrong_prop.format(self.type))\n\t\tdelta_x = self._libinput.libinput_event_gesture_get_dx_unaccelerated(\n\t\t\tself._handle)\n\t\tdelta_y = self._libinput.libinput_event_gesture_get_dy_unaccelerated(\n\t\t\tself._handle)\n\t\treturn delta_x, delta_y", "response": "The relative delta of the unaccelerated motion vector of the current event."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nreturns the absolute scale of a pinch gesture.", "response": "def scale(self):\n\t\t\"\"\"The absolute scale of a pinch gesture, the scale is\n\t\tthe division of the current distance between the fingers and\n\t\tthe distance at the start of the gesture.\n\n\t\tThe scale begins at 1.0, and if e.g. the fingers moved together by\n\t\t50% then the scale will become 0.5, if they move twice as far apart\n\t\tas initially the scale becomes 2.0, etc.\n\n\t\tFor gesture events that are of type\n\t\t:attr:`~libinput.constant.EventType.GESTURE_PINCH_BEGIN`, this property\n\t\treturns 1.0.\n\n\t\tFor gesture events that are of type\n\t\t:attr:`~libinput.constant.EventType.GESTURE_PINCH_END`, this property\n\t\treturns the scale value of the most recent\n\t\t:attr:`~libinput.constant.EventType.GESTURE_PINCH_UPDATE` event\n\t\t(if any) or 1.0 otherwise.\n\n\t\tFor all other events this property raises :exc:`AttributeError`.\n\n\t\tReturns:\n\t\t\tfloat: The absolute scale of a pinch gesture.\n\t\tRaises:\n\t\t\tAttributeError\n\t\t\"\"\"\n\n\t\tif self.type not in {EventType.GESTURE_PINCH_BEGIN,\n\t\t\t\tEventType.GESTURE_PINCH_UPDATE, EventType.GESTURE_PINCH_END}:\n\t\t\traise AttributeError(_wrong_prop.format(self.type))\n\t\treturn self._libinput.libinput_event_gesture_get_scale(self._handle)"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef angle_delta(self):\n\n\t\tif self.type != EventType.GESTURE_PINCH_UPDATE:\n\t\t\traise AttributeError(_wrong_prop.format(self.type))\n\t\treturn self._libinput.libinput_event_gesture_get_angle_delta(\n\t\t\tself._handle)", "response": "Returns the angle delta in degrees between the last and the current event."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef coords(self):\n\n\t\tx_changed = self._libinput.libinput_event_tablet_tool_x_has_changed(\n\t\t\tself._handle)\n\t\ty_changed = self._libinput.libinput_event_tablet_tool_y_has_changed(\n\t\t\tself._handle)\n\t\tx = self._libinput.libinput_event_tablet_tool_get_x(self._handle)\n\t\ty = self._libinput.libinput_event_tablet_tool_get_y(self._handle)\n\t\treturn (x, y), x_changed or y_changed", "response": "Returns the x y coordinates of the tablet tool in mm from the top left corner of the tablet in mm from\n\tthe top left corner of the tablet in its current logical orientation and whether they have changed in this event."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef delta(self):\n\n\t\tdelta_x = self._libinput.libinput_event_tablet_tool_get_dx(self._handle)\n\t\tdelta_y = self._libinput.libinput_event_tablet_tool_get_dy(self._handle)\n\t\treturn delta_x, delta_y", "response": "The relative delta between the last event and the current event."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef pressure(self):\n\n\t\tpressure = self._libinput.libinput_event_tablet_tool_get_pressure(\n\t\t\tself._handle)\n\t\tchanged = self._libinput. \\\n\t\t\tlibinput_event_tablet_tool_pressure_has_changed(self._handle)\n\t\treturn pressure, changed", "response": "Returns the current pressure of the current tool in use normalized to the range [ 0 1 ) and whether it has changed in this event."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef distance(self):\n\n\t\tdistance = self._libinput.libinput_event_tablet_tool_get_distance(\n\t\t\tself._handle)\n\t\tchanged = self._libinput. \\\n\t\t\tlibinput_event_tablet_tool_distance_has_changed(self._handle)\n\t\treturn distance, changed", "response": "Returns the current distance from the tablet s sensor to the range [ 0 1 ) and whether it has changed in this event."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef tilt_axes(self):\n\n\t\ttilt_x = self._libinput.libinput_event_tablet_tool_get_tilt_x(\n\t\t\tself._handle)\n\t\ttilt_y = self._libinput.libinput_event_tablet_tool_get_tilt_y(\n\t\t\tself._handle)\n\t\tx_changed = self._libinput. \\\n\t\t\tlibinput_event_tablet_tool_tilt_x_has_changed(self._handle)\n\t\ty_changed = self._libinput. \\\n\t\t\tlibinput_event_tablet_tool_tilt_y_has_changed(self._handle)\n\t\treturn (tilt_x, tilt_y), x_changed or y_changed", "response": "Returns the current tilt along the X Y axes of the tablet s logical orientation in degrees off the tablet s Z axis."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nreturning the current Z rotation of the tool in degrees clockwise from the tool s logical neutral position and whether it has changed in this event.", "response": "def rotation(self):\n\t\t\"\"\"The current Z rotation of the tool in degrees, clockwise\n\t\tfrom the tool's logical neutral position and whether it has changed\n\t\tin this event.\n\n\t\tFor tools of type :attr:`~libinput.constant.TabletToolType.MOUSE`\n\t\tand :attr:`~libinput.constant.TabletToolType.LENS` the logical\n\t\tneutral position is pointing to the current logical north\n\t\tof the tablet. For tools of type\n\t\t:attr:`~libinput.constant.TabletToolType.BRUSH`, the logical\n\t\tneutral position is with the buttons pointing up.\n\n\t\tIf this axis does not exist on the current tool, this property is\n\t\t(0, :obj:`False`).\n\n\t\tReturns:\n\t\t\t(float, bool): The current value of the the axis and whether it has\n\t\t\tchanged.\n\t\t\"\"\"\n\n\t\trotation = self._libinput.libinput_event_tablet_tool_get_rotation(\n\t\t\tself._handle)\n\t\tchanged = self._libinput. \\\n\t\t\tlibinput_event_tablet_tool_rotation_has_changed(self._handle)\n\t\treturn rotation, changed"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef slider_position(self):\n\n\t\tposition = self._libinput. \\\n\t\t\tlibinput_event_tablet_tool_get_slider_position(self._handle)\n\t\tchanged = self._libinput.libinput_event_tablet_tool_slider_has_changed(\n\t\t\tself._handle)\n\t\treturn position, changed", "response": "Returns the current position of the slider on the tool and whether it has changed in this event."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nreturning the delta for the wheel in degrees and whether it has changed in this event.", "response": "def wheel_delta(self):\n\t\t\"\"\"The delta for the wheel in degrees and whether it has changed in\n\t\tthis event.\n\n\t\tReturns:\n\t\t\t(float, bool): The delta of the wheel, in degrees, compared to\n\t\t\tthe last event and whether it has changed.\n\t\t\"\"\"\n\n\t\tdelta = self._libinput.libinput_event_tablet_tool_get_wheel_delta(\n\t\t\tself._handle)\n\t\tchanged = self._libinput.libinput_event_tablet_tool_wheel_has_changed(\n\t\t\tself._handle)\n\t\treturn delta, changed"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nreturning the delta for the wheel in discrete steps compared to the last event and whether it has changed in this event.", "response": "def wheel_delta_discrete(self):\n\t\t\"\"\"The delta for the wheel in discrete steps (e.g. wheel clicks) and\n\t\twhether it has changed in this event.\n\n\t\tReturns:\n\t\t\t(int, bool): The delta of the wheel, in discrete steps, compared to\n\t\t\tthe last event and whether it has changed.\n\t\t\"\"\"\n\n\t\tdelta = self._libinput. \\\n\t\t\tlibinput_event_tablet_tool_get_wheel_delta_discrete(self._handle)\n\t\tchanged = self._libinput.libinput_event_tablet_tool_wheel_has_changed(\n\t\t\tself._handle)\n\t\treturn delta, changed"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nreturning the current absolute x y coordinates of the tablet tool event transformed to screen coordinates and whether they have changed in this event.", "response": "def transform_coords(self, width, height):\n\t\t\"\"\"Return the current absolute (x, y) coordinates of\n\t\tthe tablet tool event, transformed to screen coordinates and\n\t\twhether they have changed in this event.\n\n\t\tNote:\n\t\t\tOn some devices, returned value may be negative or larger than\n\t\t\tthe width of the device. See `Out-of-bounds motion events`_\n\t\t\tfor more details.\n\t\tArgs:\n\t\t\twidth (int): The current output screen width.\n\t\t\theight (int): The current output screen height.\n\t\tReturns:\n\t\t\t((float, float), bool): The current absolute (x, y) coordinates\n\t\t\ttransformed to screen coordinates and whether they have changed.\n\t\t\"\"\"\n\n\t\tx = self._libinput.libinput_event_tablet_tool_get_x_transformed(\n\t\t\tself._handle, width)\n\t\ty = self._libinput.libinput_event_tablet_tool_get_y_transformed(\n\t\t\tself._handle, height)\n\t\tx_changed = self._libinput.libinput_event_tablet_tool_x_has_changed(\n\t\t\tself._handle)\n\t\ty_changed = self._libinput.libinput_event_tablet_tool_y_has_changed(\n\t\t\tself._handle)\n\t\treturn (x, y), x_changed or y_changed"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef tool(self):\n\n\t\thtablettool = self._libinput.libinput_event_tablet_tool_get_tool(\n\t\t\tself._handle)\n\t\treturn TabletTool(htablettool, self._libinput)", "response": "The tool that was in use during this event."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef button_state(self):\n\n\t\tif self.type != EventType.TABLET_TOOL_BUTTON:\n\t\t\traise AttributeError(_wrong_prop.format(self.type))\n\t\treturn self._libinput.libinput_event_tablet_tool_get_button_state(\n\t\t\tself._handle)", "response": "The button state of the event. This property is only available for tables that are not of type EventType. TABLET_TOOL_BUTTON."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef ring_position(self):\n\n\t\tif self.type != EventType.TABLET_PAD_RING:\n\t\t\traise AttributeError(_wrong_prop.format(self.type))\n\t\treturn self._libinput.libinput_event_tablet_pad_get_ring_position(\n\t\t\tself._handle)", "response": "The current position of the ring in degrees\n\tcounterclockwise from the northern - most point of the ring in the tablet s current logical orientation."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef ring_number(self):\n\n\t\tif self.type != EventType.TABLET_PAD_RING:\n\t\t\traise AttributeError(_wrong_prop.format(self.type))\n\t\treturn self._libinput.libinput_event_tablet_pad_get_ring_number(\n\t\t\tself._handle)", "response": "The number of the ring that has changed state."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef ring_source(self):\n\n\t\tif self.type != EventType.TABLET_PAD_RING:\n\t\t\traise AttributeError(_wrong_prop.format(self.type))\n\t\treturn self._libinput.libinput_event_tablet_pad_get_ring_source(\n\t\t\tself._handle)", "response": "The source of the interaction with the ring."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef strip_position(self):\n\n\t\tif self.type != EventType.TABLET_PAD_STRIP:\n\t\t\traise AttributeError(_wrong_prop.format(self.type))\n\t\treturn self._libinput.libinput_event_tablet_pad_get_strip_position(\n\t\t\tself._handle)", "response": "The current position of the strip in the current axis."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef strip_number(self):\n\n\t\tif self.type != EventType.TABLET_PAD_STRIP:\n\t\t\traise AttributeError(_wrong_prop.format(self.type))\n\t\treturn self._libinput.libinput_event_tablet_pad_get_strip_number(\n\t\t\tself._handle)", "response": "The number of the first strip that has changed state."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nreturns the mode group that the button ring or strip that triggered this event is considered in.", "response": "def mode_group(self):\n\t\t\"\"\"The mode group that the button, ring, or strip that\n\t\ttriggered this event is considered in.\n\n\t\tThe mode is a virtual grouping of functionality, usually based on some\n\t\tvisual feedback like LEDs on the pad. See `Tablet pad modes`_\n\t\tfor details.\n\n\t\tReturns:\n\t\t\t~libinput.define.TabletPadModeGroup: The mode group of the button,\n\t\t\tring or strip that caused this event.\n\t\t\"\"\"\n\n\t\thmodegroup = self._libinput.libinput_event_tablet_pad_get_mode_group(\n\t\t\tself._handle)\n\t\treturn TabletPadModeGroup(hmodegroup, self._libinput)"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nparse arguments returns options and args.", "response": "def parse_args(args=None):\n \"\"\"Parses arguments, returns (options, args).\"\"\"\n from argparse import ArgumentParser\n\n if args is None:\n args = sys.argv\n\n parser = ArgumentParser(description='''\nAttempt to automatically find \"adjacent\" shapes in a SVG layer, and on a second\nSVG layer, draw each detected connection between the center points of the\ncorresponding shapes.'''.strip())\n parser.add_argument('svg_input_file', type=path, default=None)\n parser.add_argument('svg_output_file', type=path, default=\"-\",\n help='Output file path (\"-\" for stdout)', nargs='?')\n parser.add_argument('-f', '--overwrite', action='store_true')\n\n args = parser.parse_args()\n\n if not args.overwrite and (args.svg_input_file == args.svg_output_file):\n parser.error('Input and output file are the same. Use `-f` to force '\n 'overwrite.')\n\n return args"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nreturning the Package long description.", "response": "def get_long_description():\n \"\"\"\n Returns the Package long description.\n\n :return: Package long description.\n :rtype: unicode\n \"\"\"\n\n description = []\n with open(\"README.rst\") as file:\n for line in file:\n if \".. code:: python\" in line and len(description) >= 2:\n blockLine = description[-2]\n if re.search(r\":$\", blockLine) and not re.search(r\"::$\", blockLine):\n description[-2] = \"::\".join(blockLine.rsplit(\":\", 1))\n continue\n\n description.append(line)\n return \"\".join(description)"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\ngenerate the full file paths from a root path.", "response": "def ftp_listing_paths(ftpconn: FTP, root: str) -> Iterable[str]:\n \"\"\"Generate the full file paths from a root path.\"\"\"\n\n for current_path, dirs, files in ftp_walk(ftpconn, root):\n yield from (os.path.join(current_path, file) for file in files)"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nreturn the directories and files for single FTP listing.", "response": "def directory_listing(conn: FTP, path: str) -> Tuple[List, List]:\n \"\"\"Return the directories and files for single FTP listing.\"\"\"\n\n entries = deque()\n conn.dir(path, entries.append)\n entries = map(parse_line, entries)\n grouped_entries = defaultdict(list)\n\n for key, value in entries:\n grouped_entries[key].append(value)\n\n directories = grouped_entries[ListingType.directory]\n files = grouped_entries[ListingType.file]\n\n return directories, files"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nparse a single line into a listing type and filename.", "response": "def parse_line(line: str, char_index=0) -> Tuple[ListingType, str]:\n \"\"\"Parse FTP directory listing into (type, filename).\"\"\"\n\n entry_name = str.rpartition(line, ' ')[-1]\n entry_type = LISTING_FLAG_MAP.get(line[char_index], ListingType.other)\n return entry_type, entry_name"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\ndownloads a file from a ftp source url to a target URI.", "response": "def download_ftp_url(source_url, target_uri, buffer_size=8192):\n \"\"\"Uses urllib. thread safe?\"\"\"\n\n ensure_file_directory(target_uri)\n\n with urllib.request.urlopen(source_url) as source_file:\n with open(target_uri, 'wb') as target_file:\n shutil.copyfileobj(source_file, target_file, buffer_size)"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nlists of devices to test", "response": "def devices(self):\n \"\"\"List of devices to test\n \"\"\"\n eax = self.attributes.get('devices')\n if eax is None:\n eax = self._all_devices\n if not isinstance(eax, list):\n eax = [eax]\n return [str(dev) for dev in eax]"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nhandling SQLAlchemy exceptions in a sane way. Args: func: An arbitrary function to wrap. error_is_fatal: Should we exit the program on exception? reraise: Should we reraise the exception, after logging? Only makes sense if error_is_fatal is False. error_messages: A dictionary that assigns an exception class to a customized error message.", "response": "def exceptions(error_is_fatal=True, error_messages=None):\n \"\"\"\n Handle SQLAlchemy exceptions in a sane way.\n\n Args:\n func: An arbitrary function to wrap.\n error_is_fatal: Should we exit the program on exception?\n reraise: Should we reraise the exception, after logging? Only makes sense\n if error_is_fatal is False.\n error_messages: A dictionary that assigns an exception class to a\n customized error message.\n \"\"\"\n\n def exception_decorator(func):\n nonlocal error_messages\n\n @functools.wraps(func)\n def exc_wrapper(*args, **kwargs):\n nonlocal error_messages\n try:\n result = func(*args, **kwargs)\n except sa.exc.SQLAlchemyError as err:\n result = None\n details = None\n err_type = err.__class__\n if error_messages and err_type in error_messages:\n details = error_messages[err_type]\n if details:\n LOG.error(details)\n LOG.error(\"For developers: (%s) %s\", err.__class__, str(err))\n if error_is_fatal:\n sys.exit(\"Abort, SQL operation failed.\")\n if not ui.ask(\n \"I can continue at your own risk, do you want that?\"):\n raise err\n return result\n\n return exc_wrapper\n\n return exception_decorator"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\ninstall versioning on the db.", "response": "def enforce_versioning(force=False):\n \"\"\"Install versioning on the db.\"\"\"\n connect_str, repo_url = get_version_data()\n LOG.warning(\"Your database uses an unversioned benchbuild schema.\")\n if not force and not ui.ask(\n \"Should I enforce version control on your schema?\"):\n LOG.error(\"User declined schema versioning.\")\n return None\n repo_version = migrate.version(repo_url, url=connect_str)\n migrate.version_control(connect_str, repo_url, version=repo_version)\n return repo_version"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef init_functions(connection):\n if settings.CFG[\"db\"][\"create_functions\"]:\n print(\"Refreshing SQL functions...\")\n for file in path.template_files(\"../sql/\", exts=[\".sql\"]):\n func = sa.DDL(path.template_str(file))\n LOG.info(\"Loading: '%s' into database\", file)\n connection.execute(func)\n connection.commit()", "response": "Initialize all SQL functions in the database."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef connect_engine(self):\n try:\n self.connection = self.engine.connect()\n return True\n except sa.exc.OperationalError as opex:\n LOG.fatal(\"Could not connect to the database. The error was: '%s'\",\n str(opex))\n return False", "response": "Establish a connection to the database."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nconfigures the databse connection.", "response": "def configure_engine(self):\n \"\"\"\n Configure the databse connection.\n\n Sets appropriate transaction isolation levels and handle errors.\n\n Returns:\n True, if we did not encounter any unrecoverable errors, else False.\n \"\"\"\n try:\n self.connection.execution_options(isolation_level=\"SERIALIZABLE\")\n except sa.exc.ArgumentError:\n LOG.debug(\"Unable to set isolation level to SERIALIZABLE\")\n return True"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef parse_date(date_str: str, pattern=_RE_DATE) -> dt.date:\n\n groups = re.match(pattern, date_str)\n\n return dt.date(*_date_to_tuple(groups.groupdict()))", "response": "Parse datetime. date from YYYY - MM - DD format."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nconverting a dictionary to a tuple.", "response": "def _datetime_to_tuple(dt_dict):\n \"\"\"datetime.datetime components from dictionary to tuple.\n\n Example\n -------\n dt_dict = {'year': '2014','month': '07','day': '23',\n 'hour': '13','minute': '12','second': '45','microsecond': '321'}\n\n _datetime_to_tuple(dt_dict) -> (2014, 7, 23, 13, 12, 45, 321)\n \"\"\"\n\n year, month, day = _date_to_tuple(dt_dict)\n hour, minute, second, microsecond = _time_to_tuple(dt_dict)\n\n return year, month, day, hour, minute, second, microsecond"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef convert_2_utc(self, datetime_, timezone):\n\n datetime_ = self.tz_mapper[timezone].localize(datetime_)\n return datetime_.astimezone(pytz.UTC)", "response": "convert to datetime to UTC offset."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nsearches for a query using the Entrez ESearch API.", "response": "def esearch(database, query, userhistory=True, webenv=False, query_key=False, retstart=False, retmax=False,\n api_key=False, email=False, **kwargs) -> Optional[EsearchResult]:\n \"\"\"Search for a query using the Entrez ESearch API.\n\n Parameters\n ----------\n database : str\n Entez database to search.\n query : str\n Query string\n userhistory : bool\n Tells API to return a WebEnV and query_key.\n webenv : str\n An Entrez WebEnv to use saved history.\n query_key : str\n An Entrez query_key to use saved history.\n retstart : int\n Return values starting at this index.\n retmax : int\n Return at most this number of values.\n api_key : str\n A users API key which allows more requests per second\n email : str\n A users email which is required if not using API.\n\n Returns\n -------\n EsearchResult\n A named tuple with values [ids, count, webenv, query_key]\n\n \"\"\"\n cleaned_query = urllib.parse.quote_plus(query, safe='/+')\n\n url = BASE_URL + f'esearch.fcgi?db={database}&term={cleaned_query}&retmode=json'\n url = check_userhistory(userhistory, url)\n url = check_webenv(webenv, url)\n url = check_query_key(query_key, url)\n url = check_retstart(retstart, url)\n url = check_retmax(retmax, url)\n url = check_api_key(api_key, url)\n url = check_email(email, url)\n\n time.sleep(PAUSE)\n resp = requests.get(url)\n if resp.status_code != 200:\n print('There was a server error')\n return\n\n text = resp.json()\n time.sleep(.5)\n return EsearchResult(\n text['esearchresult'].get('idlist', []),\n make_number(text['esearchresult'].get('count', ''), int),\n text['esearchresult'].get('webenv', ''),\n text['esearchresult'].get('querykey', '')\n )"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nposting IDs using the Entrez ESearch API.", "response": "def epost(database, ids: List[str], webenv=False, api_key=False, email=False, **kwargs) -> Optional[EpostResult]:\n \"\"\"Post IDs using the Entrez ESearch API.\n\n Parameters\n ----------\n database : str\n Entez database to search.\n ids : list\n List of IDs to submit to the server.\n webenv : str\n An Entrez WebEnv to post ids to.\n api_key : str\n A users API key which allows more requests per second\n email : str\n A users email which is required if not using API.\n\n Returns\n -------\n requests.Response\n\n \"\"\"\n url = BASE_URL + f'epost.fcgi'\n id = ','.join(ids)\n url_params = f'db={database}&id={id}'\n url_params = check_webenv(webenv, url_params)\n url_params = check_api_key(api_key, url_params)\n url_params = check_email(email, url_params)\n resp = entrez_try_put_multiple_times(url, url_params, num_tries=3)\n time.sleep(.5)\n return parse_epost(resp.text)"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef elink(db: str, dbfrom: str, ids=False, webenv=False, query_key=False, api_key=False, email=False,\n **kwargs) -> Optional[ElinkResult]:\n \"\"\"Get document summaries using the Entrez ESearch API.\n\n Parameters\n ----------\n db : str\n Entez database to get ids from.\n dbfrom : str\n Entez database the provided ids are from.\n ids : list or str\n List of IDs to submit to the server.\n webenv : str\n An Entrez WebEnv to use saved history.\n query_key : str\n An Entrez query_key to use saved history.\n api_key : str\n A users API key which allows more requests per second\n email : str\n A users email which is required if not using API.\n\n Returns\n -------\n list\n A list of ElinkResult with values [id, srx, create_date, update_date]\n\n \"\"\"\n url = BASE_URL + f'elink.fcgi?dbfrom={dbfrom}&db={db}&retmode=json&cmd=neighbor_history'\n url = check_webenv(webenv, url)\n url = check_query_key(query_key, url)\n url = check_api_key(api_key, url)\n url = check_email(email, url)\n\n if ids:\n if isinstance(ids, str):\n id = ids\n else:\n id = ','.join(ids)\n url += f'&id={id}'\n\n time.sleep(PAUSE)\n resp = requests.get(url)\n if resp.status_code != 200:\n print('There was a server error')\n return\n\n text = resp.json()\n time.sleep(.5)\n return ElinkResult(\n text['linksets'][0].get('dbfrom', ''),\n text['linksets'][0].get('linksetdbhistories', [{'dbto': ''}])[0].get('dbto', ''),\n text['linksets'][0].get('webenv', ''),\n text['linksets'][0].get('linksetdbhistories', [{'querykey': ''}])[0].get('querykey', ''),\n )", "response": "Get document summaries using the ESearch API."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef esummary(database: str, ids=False, webenv=False, query_key=False, count=False, retstart=False, retmax=False,\n api_key=False, email=False, **kwargs) -> Optional[List[EsummaryResult]]:\n \"\"\"Get document summaries using the Entrez ESearch API.\n\n Parameters\n ----------\n database : str\n Entez database to search.\n ids : list or str\n List of IDs to submit to the server.\n webenv : str\n An Entrez WebEnv to use saved history.\n query_key : str\n An Entrez query_key to use saved history.\n count : int\n Number of records in the webenv\n retstart : int\n Return values starting at this index.\n retmax : int\n Return at most this number of values.\n api_key : str\n A users API key which allows more requests per second\n email : str\n A users email which is required if not using API.\n\n Returns\n -------\n list\n A list of EsummaryResults with values [id, srx, create_date, update_date]\n\n \"\"\"\n url = BASE_URL + f'esummary.fcgi?db={database}'\n url = check_webenv(webenv, url)\n url = check_query_key(query_key, url)\n url = check_api_key(api_key, url)\n url = check_email(email, url)\n\n if ids:\n if isinstance(ids, str):\n id = ids\n else:\n id = ','.join(ids)\n url += f'&id={id}'\n count = len(id.split(','))\n\n for resp in entrez_sets_of_results(url, retstart, retmax, count):\n yield resp.text", "response": "Get document summaries using the Entrez ESearch API."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef efetch(database, ids=False, webenv=False, query_key=False, count=False, retstart=False, retmax=False,\n rettype='full', retmode='xml', api_key=False, email=False, **kwargs) -> str:\n \"\"\"Get documents using the Entrez ESearch API.gg\n\n Parameters\n ----------\n database : str\n Entez database to search.\n ids : list or str\n List of IDs to submit to the server.\n webenv : str\n An Entrez WebEnv to use saved history.\n query_key : str\n An Entrez query_key to use saved history.\n count : int\n Number of records in the webenv\n retstart : int\n Return values starting at this index.\n retmax : int\n Return at most this number of values.\n rettype : str\n The type of document to return. Refer to link for valid return types for each database.\n https://www.ncbi.nlm.nih.gov/books/NBK25499/table/chapter4.T._valid_values_of__retmode_and/?report=objectonly\n retmode : str\n The format of document to return. Refer to link for valid formats for each database.\n https://www.ncbi.nlm.nih.gov/books/NBK25499/table/chapter4.T._valid_values_of__retmode_and/?report=objectonly\n api_key : str\n A users API key which allows more requests per second\n email : str\n A users email which is required if not using API.\n\n Yields\n ------\n str\n Text from effect results. Format depends on parameters passed to retmode\n\n \"\"\"\n url = BASE_URL + f'efetch.fcgi?db={database}&retmode={retmode}&rettype={rettype}'\n url = check_webenv(webenv, url)\n url = check_query_key(query_key, url)\n url = check_api_key(api_key, url)\n url = check_email(email, url)\n\n if ids:\n if isinstance(ids, str):\n id = ids\n else:\n id = ','.join(ids)\n url += f'&id={id}'\n count = len(id.split(','))\n\n for resp in entrez_sets_of_results(url, retstart, retmax, count):\n yield resp.text", "response": "Get documents using the Entrez ESearch API. gg\n "} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef entrez_sets_of_results(url, retstart=False, retmax=False, count=False) -> Optional[List[requests.Response]]:\n if not retstart:\n retstart = 0\n\n if not retmax:\n retmax = 500\n\n if not count:\n count = retmax\n\n retmax = 500 # Entrez can return a max of 500\n while retstart < count:\n diff = count - retstart\n if diff < 500:\n retmax = diff\n\n _url = url + f'&retstart={retstart}&retmax={retmax}'\n resp = entrez_try_get_multiple_times(_url)\n if resp is None:\n return\n\n retstart += retmax\n yield resp", "response": "Returns a generator that returns all the sets of results from Entrez."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef print_runs(query):\n\n if query is None:\n return\n\n for tup in query:\n print((\"{0} @ {1} - {2} id: {3} group: {4}\".format(\n tup.end, tup.experiment_name, tup.project_name,\n tup.experiment_group, tup.run_group)))", "response": "Print all rows in this result query."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nreturns a list of layers from input SVG sources.", "response": "def get_svg_layers(svg_sources):\n '''\n Collect layers from input svg sources.\n\n Args:\n\n svg_sources (list) : A list of file-like objects, each containing\n one or more XML layers.\n\n Returns\n -------\n (width, height), layers : (int, int), list\n The first item in the tuple is the shape of the largest layer, and the\n second item is a list of ``Element`` objects (from :mod:`lxml.etree`\n module), one per SVG layer.\n '''\n layers = []\n width, height = None, None\n\n def extract_length(attr):\n 'Extract length in pixels.'\n match = CRE_MM_LENGTH.match(attr)\n if match:\n # Length is specified in millimeters.\n return INKSCAPE_PPmm.magnitude * float(match.group('length'))\n else:\n return float(attr)\n\n for svg_source_i in svg_sources:\n # Parse input file.\n xml_root = etree.parse(svg_source_i)\n svg_root = xml_root.xpath('/svg:svg', namespaces=INKSCAPE_NSMAP)[0]\n width = max(extract_length(svg_root.attrib['width']), width)\n height = max(extract_length(svg_root.attrib['height']), height)\n layers += svg_root.xpath('//svg:g[@inkscape:groupmode=\"layer\"]',\n namespaces=INKSCAPE_NSMAP)\n\n for i, layer_i in enumerate(layers):\n layer_i.attrib['id'] = 'layer%d' % (i + 1)\n return (width, height), layers"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nmerges layers from input SVG sources into a single XML document.", "response": "def merge_svg_layers(svg_sources, share_transform=True):\n '''\n Merge layers from input svg sources into a single XML document.\n\n Args:\n\n svg_sources (list) : A list of file-like objects, each containing\n one or more XML layers.\n share_transform (bool) : If exactly one layer has a transform, apply it\n to *all* other layers as well.\n\n Returns:\n\n StringIO.StringIO : File-like object containing merge XML document.\n '''\n # Get list of XML layers.\n (width, height), layers = get_svg_layers(svg_sources)\n\n if share_transform:\n transforms = [layer_i.attrib['transform'] for layer_i in layers\n if 'transform' in layer_i.attrib]\n if len(transforms) > 1:\n raise ValueError('Transform can only be shared if *exactly one* '\n 'layer has a transform ({} layers have '\n '`transform` attributes)'.format(len(transforms)))\n elif transforms:\n # Apply single common transform to all layers.\n for layer_i in layers:\n layer_i.attrib['transform'] = transforms[0]\n\n # Create blank XML output document.\n dwg = svgwrite.Drawing(profile='tiny', debug=False, size=(width, height))\n\n # Add append layers to output XML root element.\n output_svg_root = etree.fromstring(dwg.tostring())\n output_svg_root.extend(layers)\n\n # Write merged XML document to output file-like object.\n output = StringIO.StringIO()\n output.write(etree.tostring(output_svg_root))\n output.seek(0)\n return output"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef main(argv=None):\n arguments = cli_common(__doc__, argv=argv)\n driver = CampaignDriver(arguments['CAMPAIGN-DIR'], expandcampvars=False)\n driver(no_exec=True)\n if argv is not None:\n return driver", "response": "ben - ub entry point"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\ncounting the number of times each value occurs in a record sequence for a given field name.", "response": "def count_by(records: Sequence[Dict], field_name: str) -> defaultdict:\n \"\"\"\n Frequency each value occurs in a record sequence for a given field name.\n \"\"\"\n\n counter = defaultdict(int)\n\n for record in records:\n name = record[field_name]\n counter[name] += 1\n\n return counter"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef generate(self):\n exp_name = self.exp_name()\n\n fname = os.path.basename(self.out_path)\n fname = \"{exp}_{prefix}_{name}{ending}\".format(\n exp=exp_name,\n prefix=os.path.splitext(fname)[0],\n ending=os.path.splitext(fname)[-1],\n name=\"full\")\n first = True\n for chunk in self.report():\n print(\"Writing chunk to :'{0}'\".format(fname))\n chunk.to_csv(fname, header=first, mode='a')\n first = False", "response": "Generate a huge. csv file for the current experiment."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nreturning the metatab document for the URL", "response": "def doc(self):\n \"\"\"Return the metatab document for the URL\"\"\"\n from metatab import MetatabDoc\n t = self.get_resource().get_target()\n return MetatabDoc(t.inner)"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef main(argv=None):\n arguments = cli_common(__doc__, argv=argv)\n campaign_path = arguments['CAMPAIGN-DIR']\n driver = CampaignDriver(campaign_path, expandcampvars=False)\n with pushd(campaign_path):\n render(\n template=arguments['--template'],\n ostr=arguments['--output'],\n campaign=driver,\n )\n if argv is not None:\n return driver", "response": "ben - doc entry point"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef class_to_str(obj):\n mod_str = obj.__module__\n name_str = obj.__name__\n if mod_str == '__main__':\n return name_str\n else:\n return '.'.join([mod_str, name_str])", "response": "get class string from object"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef get_module_path(module):\n return pathlib.Path(\n os.path.dirname(os.path.abspath(inspect.getfile(module))))", "response": "return a directory path to a module"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef get_data_path(data, module, check_exists=True):\n basepath = os.path.dirname(os.path.abspath(inspect.getfile(module)))\n\n if isinstance(data, basestring):\n data = [data]\n\n dirpath = os.path.join(basepath, *data)\n\n if check_exists:\n assert os.path.exists(dirpath), '{0} does not exist'.format(dirpath)\n\n return pathlib.Path(dirpath)", "response": "returns a directory path to data within a module"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef memory_usage():\n try:\n import psutil\n import os\n except ImportError:\n return _memory_usage_ps()\n\n process = psutil.Process(os.getpid())\n mem = process.memory_info()[0] / float(2 ** 20)\n return mem", "response": "return memory usage of python process in MB\n "} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nreturn memory usage of python process in MB >>> is_memory_usage_ps(),float", "response": "def _memory_usage_ps():\n \"\"\"return memory usage of python process in MB\n\n >>> isinstance(_memory_usage_ps(),float)\n True\n\n \"\"\"\n out = subprocess.Popen(\n ['ps', 'v', '-p', str(os.getpid())],\n stdout=subprocess.PIPE).communicate()[0].split(b'\\n')\n vsz_index = out[0].split().index(b'RSS')\n mem = float(out[1].split()[vsz_index]) / 1024\n return mem"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nloads memory usage ipython magic and magics class", "response": "def load_memit():\n \"\"\"load memory usage ipython magic,\n require memory_profiler package to be installed\n\n to get usage: %memit?\n\n Author: Vlad Niculae \n Makes use of memory_profiler from Fabian Pedregosa\n available at https://github.com/fabianp/memory_profiler\n \"\"\"\n from IPython.core.magic import Magics, line_magic, magics_class\n from memory_profiler import memory_usage as _mu\n\n try:\n ip = get_ipython()\n except NameError as err:\n raise Exception('not in ipython/jupyter kernel:\\n {}'.format(err))\n\n @magics_class\n class MemMagics(Magics):\n @line_magic\n def memit(self, line='', setup='pass'):\n \"\"\"Measure memory usage of a Python statement\n\n Usage, in line mode:\n %memit [-irt] statement\n\n Options:\n -r: repeat the loop iteration times\n and take the best result. Default: 3\n\n -i: run the code in the current environment, without forking a new\n process.\n This is required on some MacOS versions of Accelerate if your\n line contains a call to `np.dot`.\n\n -t: timeout after seconds. Unused if `-i` is active.\n Default: None\n\n Examples\n --------\n ::\n\n In [1]: import numpy as np\n\n In [2]: %memit np.zeros(1e7)\n maximum of 3: 76.402344 MB per loop\n\n In [3]: %memit np.ones(1e6)\n maximum of 3: 7.820312 MB per loop\n\n In [4]: %memit -r 10 np.empty(1e8)\n maximum of 10: 0.101562 MB per loop\n\n In [5]: memit -t 3 while True: pass;\n Subprocess timed out.\n Subprocess timed out.\n Subprocess timed out.\n ERROR: all subprocesses exited unsuccessfully. Try again with the\n `-i` option.\n maximum of 3: -inf MB per loop\n\n \"\"\"\n opts, stmt = self.parse_options(line, 'r:t:i', posix=False,\n strict=False)\n repeat = int(getattr(opts, 'r', 3))\n if repeat < 1:\n repeat == 1\n timeout = int(getattr(opts, 't', 0))\n if timeout <= 0:\n timeout = None\n run_in_place = hasattr(opts, 'i')\n\n # Don't depend on multiprocessing:\n try:\n import multiprocessing as pr\n from multiprocessing.queues import SimpleQueue\n q = SimpleQueue()\n except ImportError:\n class ListWithPut(list):\n \"\"\"Just a list,\n where the `append` method is aliased to `put`.\"\"\"\n\n def put(self, x):\n self.append(x)\n\n q = ListWithPut()\n print(\n 'WARNING: cannot import module `multiprocessing`. Forcing '\n 'the `-i` option.')\n run_in_place = True\n\n ns = self.shell.user_ns\n\n def _get_usage(q, stmt, setup='pass', ns={}):\n try:\n exec(setup) in ns\n _mu0 = _mu()[0]\n exec(stmt) in ns\n _mu1 = _mu()[0]\n q.put(_mu1 - _mu0)\n except Exception as e:\n q.put(float('-inf'))\n raise e\n\n if run_in_place:\n for _ in range(repeat):\n _get_usage(q, stmt, ns=ns)\n else:\n # run in consecutive subprocesses\n at_least_one_worked = False\n for _ in range(repeat):\n p = pr.Process(\n target=_get_usage, args=(q, stmt, 'pass', ns))\n p.start()\n p.join(timeout=timeout)\n if p.exitcode == 0:\n at_least_one_worked = True\n else:\n p.terminate()\n if p.exitcode is None:\n print('Subprocess timed out.')\n else:\n print(\n 'Subprocess exited with code %d.' % p.exitcode)\n q.put(float('-inf'))\n\n if not at_least_one_worked:\n print('ERROR: all subprocesses exited unsuccessfully. Try '\n 'again with the `-i` option.')\n\n usages = [q.get() for _ in range(repeat)]\n usage = max(usages)\n print(\"maximum of %d: %f MB per loop\" % (repeat, usage))\n\n ip.register_magics(MemMagics)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nprocess key locations in the xml. etree. ElementTree. elementTree object node.", "response": "def parse_tree_from_dict(node, locs):\n \"\"\"Processes key locations.\n\n Parameters\n ----------\n node: xml.etree.ElementTree.ElementTree.element\n Current node.\n locs: dict\n A dictionary mapping key to a tuple. The tuple can either be 2 or 3\n elements long. The first element maps to the location in the\n current node. The second element given a processing hint. Possible\n values are:\n\n * 'text': assumes the text element of the path is wanted.\n * 'child': assumes that the child of the given path is wanted.\n * str: Any other string will be treated as an attribute lookup\n of the path.\n\n If 'child' is given, then a third element needs to be given\n indicating the type of processing. Possible values are:\n\n * 'text': assumes the text element of the path is wanted.\n * 'tag': assumes the class tag of the path is wanted.\n * str: Any other string will be treated as an attribute lookup\n of the path.\n \"\"\"\n d = dict()\n for n, l in locs.items():\n try:\n if l[1] == 'text':\n d[n] = node.find(l[0]).text\n elif l[1] == 'child':\n child = node.find(l[0]).getchildren()\n\n if len(child) > 1:\n raise AmbiguousElementException(\n 'There are too many elements')\n elif l[2] == 'text':\n d[n] = child[0].text\n elif l[2] == 'tag':\n d[n] = child[0].tag\n else:\n d[n] = node.find(l[0]).get(l[1])\n except:\n pass\n\n return d"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef xml_to_root(xml: Union[str, IO]) -> ElementTree.Element:\n if isinstance(xml, str):\n if '<' in xml:\n return ElementTree.fromstring(xml)\n else:\n with open(xml) as fh:\n xml_to_root(fh)\n tree = ElementTree.parse(xml)\n return tree.getroot()", "response": "Parse XML into an ElemeTree object."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef newLayer(self, effect=''):\n\t\tself.layers.append(Layer(effect = effect))\n\t\tself.activeLayer = len(self.layers)-1", "response": "Creates a new layer and set it as the active."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nduplicating the current active layer.", "response": "def duplicateActiveLayer(self):\n\t\t\"\"\"\n\t\tDuplicates the current active :py:class:`Layer`.\n\t\t\n\t\t:rtype: Nothing.\n\t\t\"\"\"\n\t\tactiveLayer = self.layers[self.activeLayer]\n\t\tnewLayer = Layer(data=activeLayer.data, effect=activeLayer.effect)\n\t\tself.layers.append(newLayer)\n\t\tself.activeLayer = len(self.layers)-1"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef mergeAllLayers(self):\n\t\tstart = time.time()\n\t\twhile(len(self.layers)>1):\n\t\t\tself.mergeBottomLayers()\n\t\tprint('merge time:'+str(time.time()-start))\n\t\treturn self.layers[0]", "response": "Merge all the layers together."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef synonyms(self):\n syns = {}\n\n for k, v in self._declared_terms.items():\n k = k.strip()\n if v.get('synonym'):\n syns[k.lower()] = v['synonym']\n\n if not '.' in k:\n syns[ROOT_TERM + '.' + k.lower()] = v['synonym']\n\n return syns", "response": "Return a dict of term synonyms"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef super_terms(self):\n\n # If the doc already has super terms, we've already parsed something, so\n # assume we parsed the declaration, and can use re-use the old decls.\n if self.doc and self.doc.super_terms:\n return self.doc.super_terms\n\n\n return {k.lower(): v['inheritsfrom'].lower()\n for k, v in self._declared_terms.items() if 'inheritsfrom' in v}", "response": "Return a dictionary mapping term names to their super terms"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef declare_dict(self):\n # Run the parser, if it has not been run yet.\n if not self.root:\n for _ in self: pass\n\n return {\n 'sections': self._declared_sections,\n 'terms': self._declared_terms,\n 'synonyms': self.synonyms\n }", "response": "Return the declared sections terms and synonyms as a dict"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nreplacing the record_term and parent_term with a synonym", "response": "def substitute_synonym(self, nt):\n \"\"\"\n Replace the record_term and parent_term with a synonym\n :param nt:\n :return:\n \"\"\"\n\n if nt.join_lc in self.synonyms:\n nt.parent_term, nt.record_term = Term.split_term_lower(self.synonyms[nt.join_lc]);"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef errors_as_dict(self):\n errors = []\n\n for e in self.errors:\n\n errors.append({\n 'file': e.term.file_name,\n 'row': e.term.row if e.term else '',\n 'col': e.term.col if e.term else '',\n 'term': e.term.join if e.term else '',\n 'error': str(e)\n })\n\n return errors", "response": "Return parse errors as a dict"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef find_declare_doc(self, d, name):\n\n path = None\n while True:\n if exists(name):\n path =name\n break\n\n try:\n # Look for a local document\n path = declaration_path(name)\n break\n except IncludeError:\n pass\n\n for fn in (join(d, name), join(d, name + '.csv')):\n if exists(fn):\n path = fn\n break\n if path:\n break\n\n if name.startswith('http'):\n path = name.strip('/') # Look for the file on the web\n break\n elif exists(name):\n path = name\n break\n else:\n path = self.resolver.find_decl_doc(name)\n break\n\n raise IncludeError(\"No local declaration file for '{}'\".format(name))\n\n return parse_app_url(path)", "response": "Given a name try to resolve it to a path or URL to a declaration document."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef find_include_doc(self, d, name):\n from metatab import parse_app_url\n\n include_ref = name.strip('/')\n\n if include_ref.startswith('http'):\n path = include_ref\n else:\n if not d:\n raise IncludeError(\"Can't include '{}' because don't know current path \"\n .format(name))\n\n path = join(d, include_ref)\n\n return parse_app_url(path)", "response": "Resolve a name or path for an include doc to an absolute path or url"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef generate_terms(self, ref, root, file_type=None):\n\n last_section = root\n t = None\n\n if isinstance(ref, Source):\n row_gen = ref\n ref_path = row_gen.__class__.__name__\n else:\n row_gen = get_generator(ref)\n ref_path = ref.path\n\n\n try:\n for line_n, row in enumerate(row_gen, 1):\n\n if not row or not row[0] or not row[0].strip() or row[0].strip().startswith('#'):\n continue\n\n tt = Term(row[0], None) # Just to get the qualified name constructed property\n\n term_class = self.get_term_class(tt.join_lc)\n\n t = term_class(tt.join_lc,\n row[1] if len(row) > 1 else '',\n row[2:] if len(row) > 2 else [],\n row=line_n,\n col=1,\n file_name=ref_path, file_type=file_type, doc=self.doc)\n\n # Why did we remove comments from values? It strips out Markdown\n #if t.value and str(t.value).startswith('#'): # Comments are ignored\n # continue\n\n if t.term_is('include') or t.term_is('declare'):\n\n if t.term_is('include'):\n resolved = self.find_include_doc(dirname(ref_path), t.value.strip())\n else:\n resolved = self.find_declare_doc(dirname(ref_path), t.value.strip())\n\n if row_gen.ref == resolved:\n raise IncludeError(\"Include loop for '{}' \".format(resolved))\n\n yield t\n\n try:\n\n sub_gen = get_generator(resolved.get_resource().get_target())\n\n for t in self.generate_terms(sub_gen, root, file_type=t.record_term_lc):\n yield t\n\n if last_section:\n yield last_section # Re-assert the last section\n\n except IncludeError as e:\n e.term = t\n raise\n\n except (OSError, FileNotFoundError, GenerateError, DownloadError) as e:\n e = IncludeError(\"Failed to Include; {}\".format(e))\n e.term = t\n raise e\n\n continue # Already yielded the include/declare term, and includes can't have children\n\n elif t.term_is('section'):\n\n # If there is already a section in the document, emit the existing section,\n # rather than a new one.\n try:\n last_section = self.doc[t.name]\n t = last_section\n\n except (KeyError, TypeError): # TypeError -> self.doc is None\n last_section = t\n\n yield t\n\n # Yield any child terms, from the term row arguments\n if not t.term_is('section') and not t.term_is('header'):\n for col, value in enumerate(t.args, 0):\n if str(value).strip():\n\n term_name = t.record_term_lc + '.' + str(col)\n\n term_class = self.get_term_class(term_name)\n\n yield term_class(term_name, str(value), [],\n row=line_n,\n col=col + 2, # The 0th argument starts in col 2\n file_name=ref_path,\n file_type=file_type,\n parent=t) #,\n #doc=None,\n #section=last_section)\n except IncludeError as e:\n exc = IncludeError(str(e) + \"; in '{}' \".format(ref_path))\n exc.term = e.term if hasattr(e, 'term') else None\n raise exc", "response": "Generates a generator that yields term objects handling includes and argument\n children."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nyield the inherited children of the input term", "response": "def inherited_children(self, t):\n \"\"\"Generate inherited children based on a terms InhertsFrom property.\n The input term must have both an InheritsFrom property and a defined Section\n\n :param t: A subclassed terms -- has an InheritsFrom value\n \"\"\"\n\n if not t.get('inheritsfrom'):\n return\n\n if not 'section' in t:\n raise DeclarationError(\"DeclareTerm for '{}' must specify a section to use InheritsFrom\"\n .format(t['term']))\n\n t_p, t_r = Term.split_term(t['term'])\n ih_p, ih_r = Term.split_term(t['inheritsfrom'])\n\n # The inherited terms must come from the same section\n section_terms = self._declared_sections[t['section'].lower()]['terms']\n\n # For each of the terms in the section, look for terms that are children\n # of the term that the input term inherits from. Then yield each of those terms\n # after chang the term name to be a child of the input term.\n for st_name in section_terms:\n if st_name.lower().startswith(ih_r.lower() + '.'):\n st_p, st_r = Term.split_term(st_name)\n # Yield the term, but replace the parent part\n subtype_name = t_r + '.' + st_r\n\n subtype_d = dict(self._declared_terms[st_name.lower()].items())\n subtype_d['inheritsfrom'] = '';\n subtype_d['term'] = subtype_name\n\n yield subtype_d"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef get_remote_peer_list(self):\n '''\n listPeer \u8fd4\u56de\u5217\u8868\n {\n \"rtn\":0,\n \"peerList\": [{\n \"category\": \"\",\n \"status\": 0,\n \"name\": \"GUNNER_HOME\",\n \"vodPort\": 43566,\n \"company\": \"XUNLEI_MIPS_BE_MIPS32\",\n \"pid\": \"8498352EB4F5208X0001\",\n \"lastLoginTime\": 1412053233,\n \"accesscode\": \"\",\n \"localIP\": \"\",\n \"location\": \"\\u6d59\\u6c5f\\u7701 \\u8054\\u901a\",\n \"online\": 1,\n \"path_list\": \"C:/\",\n \"type\": 30,\n \"deviceVersion\": 22083310\n }]\n }\n '''\n\n params = {\n 'type': 0,\n 'v': DEFAULT_V,\n 'ct': 2\n }\n res = self._get('listPeer', params=params)\n return res['peerList']", "response": "Get remote peer list."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef get_remote_task_list(\n self, peer_id, list_type=ListType.downloading, pos=0, number=10):\n '''\n list \u8fd4\u56de\u5217\u8868\n {\n \"recycleNum\": 0,\n \"serverFailNum\": 0,\n \"rtn\": 0,\n \"completeNum\": 34,\n \"sync\": 0,\n \"tasks\": [{\n \"failCode\": 15414,\n \"vipChannel\": {\n \"available\": 0,\n \"failCode\": 0,\n \"opened\": 0,\n \"type\": 0,\n \"dlBytes\": 0,\n \"speed\": 0\n },\n \"name\": \"Blablaba\",\n \"url\": \"magnet:?xt=urn:btih:5DF6B321CCBDEBE1D52E8E15CBFC6F002\",\n \"speed\": 0,\n \"lixianChannel\": {\n \"failCode\": 0,\n \"serverProgress\": 0,\n \"dlBytes\": 0,\n \"state\": 0,\n \"serverSpeed\": 0,\n \"speed\": 0\n },\n \"downTime\": 0,\n \"subList\": [],\n \"createTime\": 1412217010,\n \"state\": 8,\n \"remainTime\": 0,\n \"progress\": 0,\n \"path\": \"/tmp/thunder/volumes/C:/TDDOWNLOAD/\",\n \"type\": 2,\n \"id\": \"39\",\n \"completeTime\": 0,\n \"size\": 0\n },\n ...\n ]\n }\n '''\n\n params = {\n 'pid': peer_id,\n 'type': list_type,\n 'pos': pos,\n 'number': number,\n 'needUrl': 1,\n 'v': DEFAULT_V,\n 'ct': DEFAULT_CT\n }\n res = self._get('list', params=params)\n return res['tasks']", "response": "Get remote task list."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef check_url(self, pid, url_list):\n '''\n urlCheck \u8fd4\u56de\u6570\u636e\n {\n \"rtn\": 0,\n \"taskInfo\": {\n \"failCode\": 0,\n \"name\": \".HDTVrip.1024X576.mkv\",\n \"url\": \"ed2k://|file|%E6%B0%\",\n \"type\": 1,\n \"id\": \"0\",\n \"size\": 505005442\n }\n }\n '''\n task_list = []\n for url in url_list:\n params = {\n 'pid': pid,\n 'url': url,\n 'type': 1,\n 'v': DEFAULT_V,\n 'ct': DEFAULT_CT\n }\n res = self._get('urlCheck', params=params)\n\n if res['rtn'] == 0:\n task_info = res['taskInfo']\n task_list.append({\n 'url': task_info['url'],\n 'name': task_info['name'],\n 'filesize': task_info['size'],\n 'gcid': '',\n 'cid': ''\n })\n else:\n print(\n 'url [%s] check failed, code:%s.',\n url,\n task_info['failCode']\n )\n\n return task_list", "response": "Check if a URL is in the list of URLs."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nadding tasks to remote", "response": "def add_tasks_to_remote(self, pid, path='C:/TDDOWNLOAD/', task_list=[]):\n '''\n post data:\n {\n \"path\":\"C:/TDDOWNLOAD/\",\n \"tasks\":[{\n \"url\":\"ed2k://|file|%E6%B0%B8%E6%81%92.Forever...\",\n \"name\":\"\u6c38\u6052.Forever.S01E02.\u4e2d\u82f1\u5b57\u5e55.WEB-HR.mkv\",\n \"gcid\":\"\",\n \"cid\":\"\",\n \"filesize\":512807020\n }]\n }\n\n return data:\n {\n \"tasks\": [{\n \"name\": \"\\u6c38\\u6052.Fore76.x264.mkv\",\n \"url\": \"ed2k://|file|%E6%B0%B8%E6%81%92\",\n \"result\": 202,\n \"taskid\": \"48\",\n \"msg\": \"repeate_taskid:48\",\n \"id\": 1\n }],\n \"rtn\": 0\n }\n '''\n\n if len(task_list) == 0:\n return []\n\n params = {\n 'pid': pid,\n 'v': DEFAULT_V,\n 'ct': DEFAULT_CT,\n }\n\n data = {\n 'path': path,\n 'tasks': task_list\n }\n\n headers = {\n 'Content-Type': 'application/x-www-form-urlencoded'\n }\n\n data = json.dumps(data)\n data = quote(data)\n data = 'json=' + data\n\n res = self._post(\n 'createTask',\n params=params,\n data=data,\n headers=headers\n )\n\n return res"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef read_levels(text):\n x = []\n for i in range(0, len(NAUTILUSRESOLVER.getMetadata(text).citation)):\n x.append(NAUTILUSRESOLVER.getReffs(text, level=i))\n return x", "response": "Read text and get there reffs"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nproviding a manager for flask scripts to perform specific maintenance operations :param resolver: Nautilus Extension Instance :type resolver: NautilusCtsResolver :param flask_nautilus: Flask Application :type flask_nautilus: FlaskNautilus :return: CLI :rtype: click.group Import with .. code-block:: python :lineno: from capitains_nautilus.manager import FlaskNautilusManager manager = FlaskNautilusManager(resolver, flask_nautilus, app) # Where app is the name of your app if __name__ == \"__main__\": manager()", "response": "def FlaskNautilusManager(resolver, flask_nautilus):\n \"\"\" Provides a manager for flask scripts to perform specific maintenance operations\n\n :param resolver: Nautilus Extension Instance\n :type resolver: NautilusCtsResolver\n :param flask_nautilus: Flask Application\n :type flask_nautilus: FlaskNautilus\n :return: CLI\n :rtype: click.group\n\n Import with\n\n .. code-block:: python\n :lineno:\n\n from capitains_nautilus.manager import FlaskNautilusManager\n manager = FlaskNautilusManager(resolver, flask_nautilus, app) # Where app is the name of your app\n\n if __name__ == \"__main__\":\n manager()\n\n \"\"\"\n global NAUTILUSRESOLVER\n NAUTILUSRESOLVER = resolver\n\n @click.group()\n @click.option('--verbose', default=False)\n def CLI(verbose):\n \"\"\" CLI for Flask Nautilus \"\"\"\n click.echo(\"Command Line Interface of Flask\")\n resolver.logger.disabled = not verbose\n\n @CLI.command()\n def flush_resolver():\n \"\"\" Flush the resolver cache system \"\"\"\n if resolver.clear() is True:\n click.echo(\"Caching of Resolver Cleared\")\n\n @CLI.command()\n def flush_http_cache():\n \"\"\" Flush the http cache\n\n Warning : Might flush other Flask Caching data !\n \"\"\"\n flask_nautilus.flaskcache.clear()\n\n @CLI.command()\n def flush_both():\n \"\"\" Flush all caches\n\n \"\"\"\n if resolver.cache.clear() is True:\n click.echo(\"Caching of Resolver Cleared\")\n if flask_nautilus.flaskcache.clear() is True:\n click.echo(\"Caching of HTTP Cleared\")\n\n @CLI.command()\n def parse():\n \"\"\" Preprocess the inventory and cache it \"\"\"\n ret = resolver.parse()\n click.echo(\"Preprocessed %s texts\" % len(ret.readableDescendants))\n\n @CLI.command()\n @click.option('--threads', default=0, type=int)\n def process_reffs(threads):\n \"\"\" Preprocess the inventory and cache it \"\"\"\n if threads < 1:\n threads = THREADS\n texts = list(resolver.getMetadata().readableDescendants)\n click.echo(\"Using {} processes to parse references of {} texts\".format(threads, len(texts)))\n with Pool(processes=threads) as executor:\n for future in executor.imap_unordered(read_levels, [t.id for t in texts]):\n del future\n click.echo(\"References parsed\")\n\n return CLI"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef is_iter_non_string(obj):\n if isinstance(obj, list) or isinstance(obj, tuple):\n return True\n return False", "response": "test if object is a list or tuple"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\ntesting if object is dict like", "response": "def is_dict_like(obj, attr=('keys', 'items')):\n \"\"\"test if object is dict like\"\"\"\n for a in attr:\n if not hasattr(obj, a):\n return False\n return True"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\ntest if object is a list containing dict like items", "response": "def is_list_of_dict_like(obj, attr=('keys', 'items')):\n \"\"\"test if object is a list only containing dict like items \"\"\"\n try:\n if len(obj) == 0:\n return False\n return all([is_dict_like(i, attr) for i in obj])\n except Exception:\n return False"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef is_path_like(obj, attr=('name', 'is_file', 'is_dir', 'iterdir')):\n for a in attr:\n if not hasattr(obj, a):\n return False\n return True", "response": "test if object is pathlib. Path like"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nconverts all values of one type to another type", "response": "def convert_type(d, intype, outtype, convert_list=True, in_place=True):\n \"\"\" convert all values of one type to another\n\n Parameters\n ----------\n d : dict\n intype : type_class\n outtype : type_class\n convert_list : bool\n whether to convert instances inside lists and tuples\n in_place : bool\n if True, applies conversions to original dict, else returns copy\n\n Examples\n --------\n\n >>> from pprint import pprint\n\n >>> d = {'a':'1','b':'2'}\n >>> pprint(convert_type(d,str,float))\n {'a': 1.0, 'b': 2.0}\n\n >>> d = {'a':['1','2']}\n >>> pprint(convert_type(d,str,float))\n {'a': [1.0, 2.0]}\n\n >>> d = {'a':[('1','2'),[3,4]]}\n >>> pprint(convert_type(d,str,float))\n {'a': [(1.0, 2.0), [3, 4]]}\n\n \"\"\"\n if not in_place:\n out_dict = copy.deepcopy(d)\n else:\n out_dict = d\n\n def _convert(obj):\n if isinstance(obj, intype):\n try:\n obj = outtype(obj)\n except Exception:\n pass\n elif isinstance(obj, list) and convert_list:\n obj = _traverse_iter(obj)\n elif isinstance(obj, tuple) and convert_list:\n obj = tuple(_traverse_iter(obj))\n\n return obj\n\n def _traverse_dict(dic):\n for key in dic.keys():\n if is_dict_like(dic[key]):\n _traverse_dict(dic[key])\n else:\n dic[key] = _convert(dic[key])\n\n def _traverse_iter(iter):\n new_iter = []\n for key in iter:\n if is_dict_like(key):\n _traverse_dict(key)\n new_iter.append(key)\n else:\n new_iter.append(_convert(key))\n\n return new_iter\n\n if is_dict_like(out_dict):\n _traverse_dict(out_dict)\n else:\n _convert(out_dict)\n\n return out_dict"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef pprint(d, lvlindent=2, initindent=0, delim=':',\n max_width=80, depth=3, no_values=False,\n align_vals=True, print_func=None,\n keycolor=None, compress_lists=None,\n round_floats=None, _dlist=False):\n \"\"\" print a nested dict in readable format\n (- denotes an element in a list of dictionaries)\n\n Parameters\n ----------\n d : object\n lvlindent : int\n additional indentation spaces for each level\n initindent : int\n initial indentation spaces\n delim : str\n delimiter between key and value nodes\n max_width : int\n max character width of each line\n depth : int or None\n maximum levels to display\n no_values : bool\n whether to print values\n align_vals : bool\n whether to align values for each level\n print_func : callable or None\n function to print strings (print if None)\n keycolor : None or str\n if str, color keys by this color,\n allowed: red, green, yellow, blue, magenta, cyan, white\n compress_lists : int\n compress lists/tuples longer than this,\n e.g. [1,1,1,1,1,1] -> [1, 1,..., 1]\n round_floats : int\n significant figures for floats\n\n Examples\n --------\n\n >>> d = {'a':{'b':{'c':'\u00c5','de':[4,5,[7,'x'],9]}}}\n >>> pprint(d,depth=None)\n a:\n b:\n c: \u00c5\n de: [4, 5, [7, x], 9]\n >>> pprint(d,max_width=17,depth=None)\n a:\n b:\n c: \u00c5\n de: [4, 5,\n [7, x],\n 9]\n >>> pprint(d,no_values=True,depth=None)\n a:\n b:\n c:\n de:\n >>> pprint(d,depth=2)\n a:\n b: {...}\n >>> pprint({'a':[1,1,1,1,1,1,1,1]},\n ... compress_lists=3)\n a: [1, 1, 1, ...(x5)]\n\n \"\"\"\n if print_func is None:\n print_func = _default_print_func\n\n if not is_dict_like(d):\n d = {'': d}\n # print_func('{}'.format(d))\n # return\n\n extra = lvlindent if _dlist else 0\n\n def decode_to_str(obj):\n val_string = obj\n if isinstance(obj, list):\n if compress_lists is not None:\n if len(obj) > compress_lists:\n diff = str(len(obj) - compress_lists)\n obj = obj[:compress_lists] + ['...(x{})'.format(diff)]\n val_string = '[' + ', '.join([decode_to_str(o) for o in obj]) + ']'\n elif isinstance(obj, tuple):\n if compress_lists is not None:\n if len(obj) > compress_lists:\n diff = str(len(obj) - compress_lists)\n obj = list(\n obj[:compress_lists]) + ['...(x{})'.format(diff)]\n val_string = '(' + ', '.join([decode_to_str(o) for o in obj]) + ')'\n elif isinstance(obj, float) and round_floats is not None:\n round_str = '{0:.' + str(round_floats - 1) + 'E}'\n val_string = str(float(round_str.format(obj)))\n else:\n try:\n val_string = encode(obj, outtype='str')\n except (TypeError, UnicodeError):\n pass\n # convert unicode to str (so no u'' prefix in python 2)\n try:\n return str(val_string)\n except Exception:\n return unicode(val_string)\n\n if align_vals:\n key_width = 0\n for key, val in d.items():\n if not is_dict_like(val):\n key_str = decode_to_str(key)\n key_width = max(key_width, len(key_str))\n\n max_depth = depth\n for i, key in enumerate(natural_sort(d.keys())):\n value = d[key]\n if _dlist and i == 0:\n key_str = '- ' + decode_to_str(key)\n elif _dlist:\n key_str = ' ' + decode_to_str(key)\n else:\n key_str = decode_to_str(key)\n\n if keycolor is not None:\n key_str = colortxt(key_str, keycolor)\n\n if align_vals:\n key_str = '{0: <{1}} '.format(\n key_str + delim, key_width + len(delim))\n else:\n key_str = '{0}{1} '.format(key_str, delim)\n\n depth = max_depth if max_depth is not None else 2\n if keycolor is not None:\n key_length = len(_strip_ansi(key_str))\n else:\n key_length = len(key_str)\n key_line = ' ' * initindent + key_str\n new_line = ' ' * initindent + ' ' * key_length\n\n if depth <= 0:\n continue\n if is_dict_like(value):\n if depth <= 1:\n print_func(' ' * initindent + key_str + '{...}')\n else:\n print_func(' ' * initindent + key_str)\n pprint(value, lvlindent, initindent + lvlindent + extra, delim,\n max_width,\n depth=max_depth - 1 if max_depth is not None else None,\n no_values=no_values, align_vals=align_vals,\n print_func=print_func, keycolor=keycolor,\n compress_lists=compress_lists,\n round_floats=round_floats)\n continue\n\n if isinstance(value, list):\n if all([is_dict_like(o) for o in value]) and value:\n if depth <= 1:\n print_func(key_line + '[...]')\n continue\n print_func(key_line)\n for obj in value:\n pprint(\n obj, lvlindent, initindent + lvlindent + extra, delim,\n max_width,\n depth=max_depth - 1 if max_depth is not None else None,\n no_values=no_values, align_vals=align_vals,\n print_func=print_func, keycolor=keycolor,\n compress_lists=compress_lists,\n round_floats=round_floats, _dlist=True)\n continue\n\n val_string_all = decode_to_str(value) if not no_values else ''\n for i, val_string in enumerate(val_string_all.split('\\n')):\n if max_width is not None:\n if len(key_line) + 1 > max_width:\n raise Exception(\n 'cannot fit keys and data within set max_width')\n # divide into chuncks and join by same indentation\n val_indent = ' ' * (initindent + key_length)\n n = max_width - len(val_indent)\n val_string = val_indent.join(\n [s + '\\n' for s in textwrap.wrap(val_string, n)])[:-1]\n\n if i == 0:\n print_func(key_line + val_string)\n else:\n print_func(new_line + val_string)", "response": "Pretty print a nested dict in readable format"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef extract(d, path=None):\n path = [] if path is None else path\n\n d_new = copy.deepcopy(d)\n d_sub = d_new\n for key in path[:-1]:\n d_sub = d_sub[key]\n\n key = path[-1]\n d_extract = {key: d_sub[key]}\n d_sub.pop(key)\n\n return d_new, d_extract", "response": "extract section of dictionary d"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nindexing dictionary by multiple keys", "response": "def indexes(dic, keys=None):\n \"\"\" index dictionary by multiple keys\n\n Parameters\n ----------\n dic : dict\n keys : list\n\n Examples\n --------\n\n >>> d = {1:{\"a\":\"A\"},2:{\"b\":\"B\"}}\n >>> indexes(d,[1,'a'])\n 'A'\n\n \"\"\"\n keys = [] if keys is None else keys\n\n assert hasattr(dic, 'keys')\n new = dic.copy()\n old_key = None\n for key in keys:\n if not hasattr(new, 'keys'):\n raise KeyError('No indexes after: {}'.format(old_key))\n old_key = key\n new = new[key]\n return new"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\ngetting nested dict as flat dict", "response": "def flatten(d, key_as_tuple=True, sep='.', list_of_dicts=None, all_iters=None):\n \"\"\" get nested dict as flat {key:val,...},\n where key is tuple/string of all nested keys\n\n Parameters\n ----------\n d : object\n key_as_tuple : bool\n whether keys are list of nested keys or delimited string of nested keys\n sep : str\n if key_as_tuple=False, delimiter for keys\n list_of_dicts: str or None\n if not None, flatten lists of dicts using this prefix\n all_iters: str or None\n if not None, flatten all lists and tuples using this prefix\n\n Examples\n --------\n\n >>> from pprint import pprint\n\n >>> d = {1:{\"a\":\"A\"}, 2:{\"b\":\"B\"}}\n >>> pprint(flatten(d))\n {(1, 'a'): 'A', (2, 'b'): 'B'}\n\n >>> d = {1:{\"a\":\"A\"},2:{\"b\":\"B\"}}\n >>> pprint(flatten(d,key_as_tuple=False))\n {'1.a': 'A', '2.b': 'B'}\n\n >>> d = [{'a':1},{'b':[1, 2]}]\n >>> pprint(flatten(d,list_of_dicts='__list__'))\n {('__list__0', 'a'): 1, ('__list__1', 'b'): [1, 2]}\n\n >>> d = [{'a':1},{'b':[1, 2]}]\n >>> pprint(flatten(d,all_iters='__iter__'))\n {('__iter__0', 'a'): 1,\n ('__iter__1', 'b', '__iter__0'): 1,\n ('__iter__1', 'b', '__iter__1'): 2}\n\n \"\"\"\n\n def expand(key, value):\n if is_dict_like(value):\n flatten_dict = flatten(value, key_as_tuple, sep,\n list_of_dicts, all_iters)\n if key_as_tuple:\n return [(key + k, v) for k, v in flatten_dict.items()]\n else:\n return [(str(key) + sep + k, v)\n for k, v in flatten_dict.items()]\n elif is_iter_non_string(value) and all_iters is not None:\n value = {'{0}{1}'.format(all_iters, i): v\n for i, v in enumerate(value)}\n flatten_dict = flatten(value, key_as_tuple, sep,\n list_of_dicts, all_iters)\n if key_as_tuple:\n return [(key + k, v) for k, v in flatten_dict.items()]\n else:\n return [(str(key) + sep + k, v)\n for k, v in flatten_dict.items()]\n elif is_list_of_dict_like(value) and list_of_dicts is not None:\n value = {'{0}{1}'.format(list_of_dicts, i): v\n for i, v in enumerate(value)}\n flatten_dict = flatten(value, key_as_tuple, sep,\n list_of_dicts, all_iters)\n if key_as_tuple:\n return [(key + k, v) for k, v in flatten_dict.items()]\n else:\n return [(str(key) + sep + k, v)\n for k, v in flatten_dict.items()]\n else:\n return [(key, value)]\n\n if is_iter_non_string(d) and all_iters is not None:\n d = {'{0}{1}'.format(all_iters, i): v for i, v in enumerate(d)}\n elif is_list_of_dict_like(d) and list_of_dicts is not None:\n d = {'{0}{1}'.format(list_of_dicts, i): v for i, v in enumerate(d)}\n elif not is_dict_like(d):\n raise TypeError('d is not dict like: {}'.format(d))\n\n if key_as_tuple:\n items = [item for k, v in d.items() for item in expand((k,), v)]\n else:\n items = [item for k, v in d.items() for item in expand(k, v)]\n\n return dict(items)"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _single_merge(a, b, error_path=None, overwrite=False,\n append=False, list_of_dicts=False):\n \"\"\"merges b into a\n \"\"\"\n if error_path is None:\n error_path = []\n\n if list_of_dicts and is_list_of_dict_like(a) and is_list_of_dict_like(b):\n if len(a) != len(b):\n raise ValueError(\n 'list of dicts are of different lengths at '\n '\"{0}\": old: {1}, new: {2}'.format('.'.join(error_path), a, b))\n return [_single_merge(a_item, b_item,\n error_path + [\"iter_{}\".format(i)],\n overwrite, append, list_of_dicts)\n for i, (a_item, b_item) in enumerate(zip(a, b))]\n\n for key in b:\n if key in a:\n if is_dict_like(a[key]) and is_dict_like(b[key]):\n _single_merge(a[key], b[key], error_path +\n [str(key)], overwrite, append, list_of_dicts)\n elif (isinstance(a[key], list)\n and isinstance(b[key], list) and append):\n a[key] += b[key]\n elif (list_of_dicts\n and is_list_of_dict_like(a[key])\n and is_list_of_dict_like(b[key])):\n if len(a[key]) != len(b[key]):\n raise ValueError(\n 'list of dicts are of different lengths at '\n '\"{0}\": old: {1}, new: {2}'.format(\n '.'.join(error_path + [str(key)]), a[key], b[key]))\n for i, (a_item, b_item) in enumerate(zip(a[key], b[key])):\n _single_merge(a_item, b_item,\n error_path + [str(key), \"iter_{}\".format(i)],\n overwrite, append, list_of_dicts)\n elif a[key] == b[key]:\n pass # same leaf value\n elif overwrite:\n a[key] = b[key]\n else:\n raise ValueError(\n 'different data already exists at '\n '\"{0}\": old: {1}, new: {2}'.format(\n '.'.join(error_path + [str(key)]), a[key], b[key]))\n else:\n a[key] = b[key]\n return a", "response": "merges a into b"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef merge(dicts, overwrite=False, append=False, list_of_dicts=False):\n outdict = copy.deepcopy(dicts[0])\n\n def single_merge(a, b):\n return _single_merge(a, b, overwrite=overwrite, append=append,\n list_of_dicts=list_of_dicts)\n\n reduce(single_merge, [outdict] + dicts[1:])\n\n return outdict", "response": "Merge dicts into a single base base."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef flattennd(d, levels=0, key_as_tuple=True, delim='.',\n list_of_dicts=None):\n \"\"\" get nested dict as {key:dict,...},\n where key is tuple/string of all-n levels of nested keys\n\n Parameters\n ----------\n d : dict\n levels : int\n the number of levels to leave unflattened\n key_as_tuple : bool\n whether keys are list of nested keys or delimited string of nested keys\n delim : str\n if key_as_tuple=False, delimiter for keys\n list_of_dicts: str or None\n if not None, flatten lists of dicts using this prefix\n\n Examples\n --------\n\n >>> from pprint import pprint\n\n >>> d = {1:{2:{3:{'b':'B','c':'C'},4:'D'}}}\n >>> pprint(flattennd(d,0))\n {(1, 2, 3, 'b'): 'B', (1, 2, 3, 'c'): 'C', (1, 2, 4): 'D'}\n\n >>> pprint(flattennd(d,1))\n {(1, 2): {4: 'D'}, (1, 2, 3): {'b': 'B', 'c': 'C'}}\n\n >>> pprint(flattennd(d,2))\n {(1,): {2: {4: 'D'}}, (1, 2): {3: {'b': 'B', 'c': 'C'}}}\n\n >>> pprint(flattennd(d,3))\n {(): {1: {2: {4: 'D'}}}, (1,): {2: {3: {'b': 'B', 'c': 'C'}}}}\n\n >>> pprint(flattennd(d,4))\n {(): {1: {2: {3: {'b': 'B', 'c': 'C'}, 4: 'D'}}}}\n\n >>> pprint(flattennd(d,5))\n {(): {1: {2: {3: {'b': 'B', 'c': 'C'}, 4: 'D'}}}}\n\n >>> pprint(flattennd(d,1,key_as_tuple=False,delim='.'))\n {'1.2': {4: 'D'}, '1.2.3': {'b': 'B', 'c': 'C'}}\n\n >>> test_dict = {\"a\":[{\"b\":[{\"c\":1, \"d\": 2}, {\"e\":3, \"f\": 4}]}, {\"b\":[{\"c\":5, \"d\": 6}, {\"e\":7, \"f\": 8}]}]}\n >>> pprint(flattennd(test_dict, list_of_dicts=\"__list__\", levels=2))\n {('a', '__list__0', 'b'): [{'c': 1, 'd': 2}, {'e': 3, 'f': 4}],\n ('a', '__list__1', 'b'): [{'c': 5, 'd': 6}, {'e': 7, 'f': 8}]}\n\n >>> pprint(flattennd(test_dict, list_of_dicts=\"__list__\", levels=3))\n {('a', '__list__0'): {'b': [{'c': 1, 'd': 2}, {'e': 3, 'f': 4}]},\n ('a', '__list__1'): {'b': [{'c': 5, 'd': 6}, {'e': 7, 'f': 8}]}}\n\n \"\"\" # noqa: E501\n if levels < 0:\n raise ValueError('unflattened levels must be greater than 0')\n\n new_d = {}\n flattened = flatten(d, True, delim, list_of_dicts=list_of_dicts)\n if levels == 0:\n return flattened\n\n for key, value in flattened.items():\n if key_as_tuple:\n new_key = key[: - (levels)]\n else:\n new_key = delim.join([str(k) for k in key[:-(levels)]])\n new_levels = key[-(levels):]\n\n # val_dict = {new_levels: value}\n # val_dict = unflatten(val_dict, True, delim)\n\n if new_key not in new_d:\n new_d[new_key] = {new_levels: value}\n else:\n if new_levels in new_d[new_key]:\n raise ValueError(\n \"key clash for: {0}; {1}\".format(new_key, new_levels))\n new_d[new_key][new_levels] = value\n\n for nkey, nvalue in new_d.items():\n new_d[nkey] = unflatten(\n nvalue, list_of_dicts=list_of_dicts, deepcopy=False)\n\n return new_d", "response": "get nested dict as dict where key is tuple or delimited string of all - n levels of nested keys or list of dicts with delimiter"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef flatten2d(d, key_as_tuple=True, delim='.',\n list_of_dicts=None):\n \"\"\" get nested dict as {key:dict,...},\n where key is tuple/string of all-1 nested keys\n\n NB: is same as flattennd(d,1,key_as_tuple,delim)\n\n Parameters\n ----------\n d : dict\n key_as_tuple : bool\n whether keys are list of nested keys or delimited string of nested keys\n delim : str\n if key_as_tuple=False, delimiter for keys\n list_of_dicts: str or None\n if not None, flatten lists of dicts using this prefix\n\n Examples\n --------\n\n >>> from pprint import pprint\n\n >>> d = {1:{2:{3:{'b':'B','c':'C'},4:'D'}}}\n >>> pprint(flatten2d(d))\n {(1, 2): {4: 'D'}, (1, 2, 3): {'b': 'B', 'c': 'C'}}\n\n >>> pprint(flatten2d(d,key_as_tuple=False,delim=','))\n {'1,2': {4: 'D'}, '1,2,3': {'b': 'B', 'c': 'C'}}\n\n \"\"\"\n return flattennd(d, 1, key_as_tuple, delim, list_of_dicts=list_of_dicts)", "response": "get nested dict as dict where key is tuple or delimited string of all - 1 nested keys"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef remove_keys(d, keys=None, use_wildcards=True,\n list_of_dicts=False, deepcopy=True):\n \"\"\"remove certain keys from nested dict, retaining preceeding paths\n\n Parameters\n ----------\n keys: list\n use_wildcards : bool\n if true, can use * (matches everything)\n and ? (matches any single character)\n list_of_dicts: bool\n treat list of dicts as additional branches\n deepcopy: bool\n\n Examples\n --------\n\n >>> from pprint import pprint\n >>> d = {1:{\"a\":\"A\"},\"a\":{\"b\":\"B\"}}\n >>> pprint(remove_keys(d,['a']))\n {1: 'A', 'b': 'B'}\n\n >>> pprint(remove_keys({'abc':1},['a*'],use_wildcards=False))\n {'abc': 1}\n >>> pprint(remove_keys({'abc':1},['a*'],use_wildcards=True))\n {}\n\n \"\"\"\n keys = [] if keys is None else keys\n list_of_dicts = '__list__' if list_of_dicts else None\n\n def is_in(a, bs):\n if use_wildcards:\n for b in bs:\n try:\n if a == b:\n return True\n if fnmatch(a, b):\n return True\n except Exception:\n pass\n return False\n else:\n try:\n return a in bs\n except Exception:\n return False\n\n if not hasattr(d, 'items'):\n return d\n else:\n dic = flatten(d, list_of_dicts=list_of_dicts)\n new_dic = {}\n for key, value in dic.items():\n new_key = tuple([i for i in key if not is_in(i, keys)])\n if not new_key:\n continue\n try:\n if new_key[-1].startswith(list_of_dicts):\n continue\n except Exception:\n pass\n new_dic[new_key] = value\n return unflatten(\n new_dic, list_of_dicts=list_of_dicts, deepcopy=deepcopy)", "response": "remove certain keys from nested dict retaining preceeding paths"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef remove_keyvals(d, keyvals=None, list_of_dicts=False, deepcopy=True):\n keyvals = [] if keyvals is None else keyvals\n list_of_dicts = '__list__' if list_of_dicts else None\n\n if hasattr(keyvals, 'items'):\n keyvals = [(k, v) for k, v in keyvals.items()]\n\n if not hasattr(d, 'items'):\n return d\n\n flatd = flatten(d, list_of_dicts=list_of_dicts)\n\n def is_in(a, b):\n try:\n return a in b\n except Exception:\n return False\n\n prune = [k[0] for k, v in flatd.items() if is_in((k[-1], v), keyvals)]\n flatd = {k: v for k, v in flatd.items() if not is_in(k[0], prune)}\n\n return unflatten(flatd, list_of_dicts=list_of_dicts, deepcopy=deepcopy)", "response": "remove paths with at least one branch leading\n to certain keyvals from dict d"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nremoves paths containing certain keys from dict d", "response": "def remove_paths(d, keys, list_of_dicts=False, deepcopy=True):\n \"\"\" remove paths containing certain keys from dict\n\n Parameters\n ----------\n d: dict\n keys : list\n list of keys to find and remove path\n list_of_dicts: bool\n treat list of dicts as additional branches\n deepcopy: bool\n deepcopy values\n\n Examples\n --------\n\n >>> from pprint import pprint\n >>> d = {1:{\"a\":\"A\"},2:{\"b\":\"B\"},4:{5:{6:'a',7:'b'}}}\n >>> pprint(remove_paths(d,[6,'a']))\n {2: {'b': 'B'}, 4: {5: {7: 'b'}}}\n\n >>> d = {1:{2: 3}, 1:{4: 5}}\n >>> pprint(remove_paths(d,[(1, 2)]))\n {1: {4: 5}}\n\n\n >>> d2 = {'a':[{'b':1,'c':{'b':3}},{'b':1,'c':2}]}\n >>> pprint(remove_paths(d2,[\"b\"],list_of_dicts=False))\n {'a': [{'b': 1, 'c': {'b': 3}}, {'b': 1, 'c': 2}]}\n\n >>> pprint(remove_paths(d2,[\"b\"],list_of_dicts=True))\n {'a': [{'c': 2}]}\n\n \"\"\"\n keys = [(key,) if not isinstance(key, tuple) else key for key in keys]\n list_of_dicts = '__list__' if list_of_dicts else None\n\n def contains(path):\n for k in keys:\n if set(k).issubset(path):\n return True\n return False\n\n flatd = flatten(d, list_of_dicts=list_of_dicts)\n\n flatd = {path: v for path, v in flatd.items() if not contains(path)}\n\n return unflatten(flatd, list_of_dicts=list_of_dicts, deepcopy=deepcopy)"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nfiltering leaf nodes of nested dictionary d by values vals", "response": "def filter_values(d, vals=None, list_of_dicts=False, deepcopy=True):\n \"\"\" filters leaf nodes of nested dictionary\n\n Parameters\n ----------\n d : dict\n vals : list\n values to filter by\n list_of_dicts: bool\n treat list of dicts as additional branches\n deepcopy: bool\n deepcopy values\n\n Examples\n --------\n\n >>> d = {1:{\"a\":\"A\"},2:{\"b\":\"B\"},4:{5:{6:'a'}}}\n >>> filter_values(d,['a'])\n {4: {5: {6: 'a'}}}\n\n \"\"\"\n vals = [] if vals is None else vals\n list_of_dicts = '__list__' if list_of_dicts else None\n\n flatd = flatten(d, list_of_dicts=list_of_dicts)\n\n def is_in(a, b):\n try:\n return a in b\n except Exception:\n return False\n\n flatd = {k: v for k, v in flatd.items() if is_in(v, vals)}\n return unflatten(flatd, list_of_dicts=list_of_dicts, deepcopy=deepcopy)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef filter_keyvals(d, keyvals, logic=\"OR\", keep_siblings=False,\n list_of_dicts=False, deepcopy=True):\n \"\"\" filters leaf nodes key:value pairs of nested dictionary\n\n Parameters\n ----------\n d : dict\n keyvals : dict or list[tuple]\n (key,value) pairs to filter by\n logic : str\n \"OR\" or \"AND\" for matching pairs\n keep_siblings : bool\n keep all sibling paths\n list_of_dicts : bool\n treat list of dicts as additional branches\n deepcopy: bool\n deepcopy values\n\n Examples\n --------\n\n >>> from pprint import pprint\n\n >>> d = {1:{6:'a'},3:{7:'a'},2:{6:\"b\"},4:{5:{6:'a'}}}\n >>> pprint(filter_keyvals(d,[(6,'a')]))\n {1: {6: 'a'}, 4: {5: {6: 'a'}}}\n\n >>> d2 = {'a':{'b':1,'c':2,'d':3}, 'e':4}\n\n >>> pprint(filter_keyvals(d2, {'b': 1, 'e': 4}, logic=\"OR\", keep_siblings=False))\n {'a': {'b': 1}, 'e': 4}\n\n >>> pprint(filter_keyvals(d2,[('b',1)], logic=\"OR\", keep_siblings=True))\n {'a': {'b': 1, 'c': 2, 'd': 3}}\n\n >>> pprint(filter_keyvals(d2, {'b': 1, 'e': 4}, logic=\"AND\", keep_siblings=False))\n {}\n\n >>> pprint(filter_keyvals(d2, {'b': 1, 'c': 2}, logic=\"AND\", keep_siblings=False))\n {'a': {'b': 1, 'c': 2}}\n\n >>> pprint(filter_keyvals(d2,[('b',1), ('c',2)], logic=\"AND\", keep_siblings=True))\n {'a': {'b': 1, 'c': 2, 'd': 3}}\n\n >>> d3 = {\"a\": {\"b\": 1, \"f\": {\"d\": 3}}, \"e\": {\"b\": 1, \"c\": 2, \"f\": {\"d\": 3}}, \"g\": 5}\n >>> pprint(filter_keyvals(d3,[('b',1), ('c', 2)], logic=\"OR\", keep_siblings=True))\n {'a': {'b': 1, 'f': {'d': 3}}, 'e': {'b': 1, 'c': 2, 'f': {'d': 3}}}\n\n >>> pprint(filter_keyvals(d3,[('b',1), ('c', 2)], logic=\"AND\", keep_siblings=True))\n {'e': {'b': 1, 'c': 2, 'f': {'d': 3}}}\n\n\n \"\"\" # noqa: E501\n if len(keyvals) != len(dict(keyvals)):\n raise ValueError(\"repeating keys in keyvals: {}\".format(keyvals))\n\n keyvals = dict(keyvals)\n list_of_dicts = '__list__' if list_of_dicts else None\n\n flattened = flatten(d, list_of_dicts=list_of_dicts)\n\n if logic == \"OR\":\n if keep_siblings:\n pruned = {\n tuple(k[:-1]) for k, v in flattened.items()\n if any(key == k[-1] and v == keyvals[key] for key in keyvals)}\n filtered = {k: v for k, v in flattened.items()\n if _in_pruned(k, pruned)}\n else:\n filtered = {\n k: v for k, v in flattened.items()\n if any(key == k[-1] and v == keyvals[key] for key in keyvals)}\n elif logic == \"AND\":\n pruned = {}\n for k, v in flattened.items():\n if any(key == k[-1] and v == keyvals[key] for key in keyvals):\n pruned[tuple(k[:-1])] = pruned.get(tuple(k[:-1]), []) + [k[-1]]\n all_keys = set(keyvals.keys())\n pruned = [k for k, v in pruned.items() if set(v) == all_keys]\n\n if keep_siblings:\n filtered = {k: v for k, v in flattened.items()\n if _in_pruned(k, pruned)}\n else:\n filtered = {k: v for k, v in flattened.items(\n ) if k[-1] in all_keys and _in_pruned(k, pruned)}\n else:\n raise ValueError(\"logic must be AND or OR: {}\".format(logic))\n\n return unflatten(filtered, list_of_dicts=list_of_dicts, deepcopy=deepcopy)", "response": "Filters the nested dictionary d by the given keyvals."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef filter_keys(d, keys, use_wildcards=False,\n list_of_dicts=False, deepcopy=True):\n \"\"\" filter dict by certain keys\n\n Parameters\n ----------\n d : dict\n keys: list\n use_wildcards : bool\n if true, can use * (matches everything)\n and ? (matches any single character)\n list_of_dicts: bool\n treat list of dicts as additional branches\n deepcopy: bool\n deepcopy values\n\n Examples\n --------\n\n >>> from pprint import pprint\n\n >>> d = {1:{\"a\":\"A\"},2:{\"b\":\"B\"},4:{5:{6:'a',7:'b'}}}\n >>> pprint(filter_keys(d,['a',6]))\n {1: {'a': 'A'}, 4: {5: {6: 'a'}}}\n\n >>> d = {1:{\"axxxx\":\"A\"},2:{\"b\":\"B\"}}\n >>> pprint(filter_keys(d,['a*'],use_wildcards=True))\n {1: {'axxxx': 'A'}}\n\n \"\"\"\n list_of_dicts = '__list__' if list_of_dicts else None\n flatd = flatten(d, list_of_dicts=list_of_dicts)\n\n def is_in(a, bs):\n if use_wildcards:\n for b in bs:\n try:\n if a == b:\n return True\n if fnmatch(b, a):\n return True\n except Exception:\n pass\n return False\n else:\n try:\n return a in bs\n except Exception:\n return False\n\n flatd = {paths: v for paths, v in flatd.items() if any(\n [is_in(k, paths) for k in keys])}\n return unflatten(flatd, list_of_dicts=list_of_dicts, deepcopy=deepcopy)", "response": "filter dict by certain keys"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef filter_paths(d, paths, list_of_dicts=False, deepcopy=True):\n list_of_dicts = '__list__' if list_of_dicts else None\n\n all_keys = [x for y in paths if isinstance(y, tuple) for x in y]\n all_keys += [x for x in paths if not isinstance(x, tuple)]\n # faster to filter first I think\n new_d = filter_keys(d, all_keys, list_of_dicts=list_of_dicts)\n new_d = flatten(d, list_of_dicts=list_of_dicts)\n for key in list(new_d.keys()):\n if not any([\n set(key).issuperset(path if isinstance(path, tuple) else[path])\n for path in paths]):\n new_d.pop(key)\n return unflatten(new_d, list_of_dicts=list_of_dicts, deepcopy=deepcopy)", "response": "filter dict by certain paths containing key sets"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nrenaming keys in dict", "response": "def rename_keys(d, keymap=None, list_of_dicts=False, deepcopy=True):\n \"\"\" rename keys in dict\n\n Parameters\n ----------\n d : dict\n keymap : dict\n dictionary of key name mappings\n list_of_dicts: bool\n treat list of dicts as additional branches\n deepcopy: bool\n deepcopy values\n\n Examples\n --------\n\n >>> from pprint import pprint\n >>> d = {'a':{'old_name':1}}\n >>> pprint(rename_keys(d,{'old_name':'new_name'}))\n {'a': {'new_name': 1}}\n\n \"\"\"\n list_of_dicts = '__list__' if list_of_dicts else None\n keymap = {} if keymap is None else keymap\n\n flatd = flatten(d, list_of_dicts=list_of_dicts)\n\n flatd = {\n tuple([keymap.get(k, k) for k in path]): v for path, v in flatd.items()\n }\n\n return unflatten(flatd, list_of_dicts=list_of_dicts, deepcopy=deepcopy)"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef split_key(d, key, new_keys, before=True,\n list_of_dicts=False, deepcopy=True):\n \"\"\" split an existing key(s) into multiple levels\n\n Parameters\n ----------\n d : dict\n or dict like\n key: str\n existing key value\n new_keys: list[str]\n new levels to add\n before: bool\n add level before existing key (else after)\n list_of_dicts: bool\n treat list of dicts as additional branches\n\n Examples\n --------\n >>> from pprint import pprint\n >>> d = {'a':1,'b':2}\n >>> pprint(split_key(d,'a',['c','d']))\n {'b': 2, 'c': {'d': {'a': 1}}}\n\n >>> pprint(split_key(d,'a',['c','d'],before=False))\n {'a': {'c': {'d': 1}}, 'b': 2}\n\n >>> d2 = [{'a':1},{'a':2},{'a':3}]\n >>> pprint(split_key(d2,'a',['b'],list_of_dicts=True))\n [{'b': {'a': 1}}, {'b': {'a': 2}}, {'b': {'a': 3}}]\n\n \"\"\"\n list_of_dicts = '__list__' if list_of_dicts else None\n flatd = flatten(d, list_of_dicts=list_of_dicts)\n\n newd = {}\n for path, v in flatd.items():\n if key in path:\n newk = []\n for k in path:\n if k == key:\n if before:\n newk = newk + new_keys + [k]\n else:\n newk = newk + [k] + new_keys\n else:\n newk.append(k)\n newd[tuple(newk)] = v\n else:\n newd[path] = v\n\n return unflatten(newd, list_of_dicts=list_of_dicts, deepcopy=deepcopy)", "response": "split an existing key into multiple levels"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\napplies a function to all values with a certain leaf key.", "response": "def apply(d, leaf_key, func, new_name=None, remove_lkey=True,\n list_of_dicts=False, unflatten_level=0, deepcopy=True, **kwargs):\n \"\"\" apply a function to all values with a certain leaf (terminal) key\n\n Parameters\n ----------\n d : dict\n leaf_key : str\n name of leaf key\n func : callable\n function to apply\n new_name : str\n if not None, rename leaf_key\n remove_lkey: bool\n whether to remove original leaf_key (if new_name is not None)\n list_of_dicts: bool\n treat list of dicts as additional branches\n unflatten_level : int or None\n the number of levels to leave unflattened before combining,\n for instance if you need dicts as inputs\n deepcopy: bool\n deepcopy values\n kwargs : dict\n additional keywords to parse to function\n\n Examples\n --------\n\n >>> from pprint import pprint\n >>> d = {'a':1,'b':1}\n >>> func = lambda x: x+1\n >>> pprint(apply(d,'a',func))\n {'a': 2, 'b': 1}\n >>> pprint(apply(d,'a',func,new_name='c'))\n {'b': 1, 'c': 2}\n >>> pprint(apply(d,'a',func,new_name='c', remove_lkey=False))\n {'a': 1, 'b': 1, 'c': 2}\n\n >>> test_dict = {\"a\":[{\"b\":[{\"c\":1, \"d\": 2}, {\"e\":3, \"f\": 4}]}, {\"b\":[{\"c\":5, \"d\": 6}, {\"e\":7, \"f\": 8}]}]}\n >>> pprint(apply(test_dict, \"b\", lambda x: x[-1], list_of_dicts=True, unflatten_level=2))\n {'a': [{'b': {'e': 3, 'f': 4}}, {'b': {'e': 7, 'f': 8}}]}\n\n \"\"\" # noqa: E501\n list_of_dicts = '__list__' if list_of_dicts else None\n if unflatten_level == 0:\n flatd = flatten(d, list_of_dicts=list_of_dicts)\n else:\n flatd = flattennd(d, unflatten_level, list_of_dicts=list_of_dicts)\n newd = {k: (func(v, **kwargs) if k[-1] == leaf_key else v)\n for k, v in flatd.items()}\n if new_name is not None:\n newd = {(tuple(list(k[:-1]) + [new_name]) if k[-1]\n == leaf_key else k): v for k, v in newd.items()}\n if not remove_lkey:\n newd.update(flatd)\n\n return unflatten(newd, list_of_dicts=list_of_dicts, deepcopy=deepcopy)"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef combine_apply(d, leaf_keys, func, new_name,\n unflatten_level=1, remove_lkeys=True, overwrite=False,\n list_of_dicts=False, deepcopy=True, **kwargs):\n \"\"\" combine values with certain leaf (terminal) keys by a function\n\n Parameters\n ----------\n d : dict\n leaf_keys : list\n names of leaf keys\n func : callable\n function to apply,\n must take at least len(leaf_keys) arguments\n new_name : str\n new key name\n unflatten_level : int or None\n the number of levels to leave unflattened before combining,\n for instance if you need dicts as inputs (None means all)\n remove_lkeys: bool\n whether to remove original leaf_keys\n overwrite: bool\n whether to overwrite any existing new_name key\n list_of_dicts: bool\n treat list of dicts as additional branches\n deepcopy: bool\n deepcopy values\n kwargs : dict\n additional keywords to parse to function\n\n Examples\n --------\n\n >>> from pprint import pprint\n >>> d = {'a':1,'b':2}\n >>> func = lambda x,y: x+y\n >>> pprint(combine_apply(d,['a','b'],func,'c'))\n {'c': 3}\n >>> pprint(combine_apply(d,['a','b'],func,'c',remove_lkeys=False))\n {'a': 1, 'b': 2, 'c': 3}\n\n >>> d = {1:{'a':1,'b':2},2:{'a':4,'b':5},3:{'a':1}}\n >>> pprint(combine_apply(d,['a','b'],func,'c'))\n {1: {'c': 3}, 2: {'c': 9}, 3: {'a': 1}}\n\n >>> func2 = lambda x: sorted(list(x.keys()))\n >>> d2 = {'d':{'a':{'b':1,'c':2}}}\n >>> pprint(combine_apply(d2,['a'],func2,'a',unflatten_level=2))\n {'d': {'a': ['b', 'c']}}\n\n \"\"\"\n list_of_dicts = '__list__' if list_of_dicts else None\n if unflatten_level is not None:\n flatd = flattennd(d, levels=unflatten_level,\n list_of_dicts=list_of_dicts)\n else:\n # TODO could do this better?\n flatd = unflatten(d, key_as_tuple=False,\n delim='*@#$', deepcopy=deepcopy)\n\n for dic in flatd.values():\n if not is_dict_like(dic):\n continue\n if all([k in list(dic.keys()) for k in leaf_keys]):\n if remove_lkeys:\n vals = [dic.pop(k) for k in leaf_keys]\n else:\n vals = [dic[k] for k in leaf_keys]\n if new_name in dic and not overwrite:\n raise ValueError('{} already in sub-dict'.format(new_name))\n dic[new_name] = func(*vals, **kwargs)\n\n if unflatten_level is not None:\n return unflatten(flatd, list_of_dicts=list_of_dicts, deepcopy=deepcopy)\n else:\n return flatd", "response": "combine values with certain leaf keys by a function"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef split_lists(d, split_keys, new_name='split',\n check_length=True, deepcopy=True):\n \"\"\"split_lists key:list pairs into dicts for each item in the lists\n NB: will only split if all split_keys are present\n\n Parameters\n ----------\n d : dict\n split_keys : list\n keys to split\n new_name : str\n top level key for split items\n check_length : bool\n if true, raise error if any lists are of a different length\n deepcopy: bool\n deepcopy values\n\n Examples\n --------\n\n >>> from pprint import pprint\n\n >>> d = {'path_key':{'x':[1,2],'y':[3,4],'a':1}}\n >>> new_d = split_lists(d,['x','y'])\n >>> pprint(new_d)\n {'path_key': {'a': 1, 'split': [{'x': 1, 'y': 3}, {'x': 2, 'y': 4}]}}\n\n >>> split_lists(d,['x','a'])\n Traceback (most recent call last):\n ...\n ValueError: \"a\" data at the following path is not a list ('path_key',)\n\n >>> d2 = {'path_key':{'x':[1,7],'y':[3,4,5]}}\n >>> split_lists(d2,['x','y'])\n Traceback (most recent call last):\n ...\n ValueError: lists at the following path do not have the same size ('path_key',)\n\n\n \"\"\" # noqa: E501\n flattened = flatten2d(d)\n\n new_d = {}\n for key, value in flattened.items():\n if set(split_keys).issubset(value.keys()):\n # combine_d = {}\n combine_d = []\n sub_d = {}\n length = None\n for subkey, subvalue in value.items():\n if subkey in split_keys:\n if not isinstance(subvalue, list):\n raise ValueError(\n '\"{0}\" data at the following path is not a list '\n '{1}'.format(subkey, key))\n\n if check_length and length is not None:\n if len(subvalue) != length:\n raise ValueError(\n 'lists at the following path '\n 'do not have the same size {0}'.format(key))\n if length is None:\n combine_d = [{subkey: v} for v in subvalue]\n else:\n for item, val in zip(combine_d, subvalue):\n item[subkey] = val\n\n length = len(subvalue)\n # new_combine = {k:{subkey:v}\n # for k,v in enumerate(subvalue)}\n # combine_d = merge([combine_d,new_combine])\n else:\n sub_d[subkey] = subvalue\n try:\n new_d[key] = merge([sub_d, {new_name: combine_d}])\n except ValueError:\n raise ValueError(\n 'split data key: {0}, already exists at '\n 'this level for {1}'.format(new_name, key))\n else:\n new_d[key] = value\n\n return unflatten(new_d, deepcopy=deepcopy)", "response": "split_lists key = list pairs into dicts for each item in the lists\n "} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\ncombine lists of dicts", "response": "def combine_lists(d, keys=None, deepcopy=True):\n \"\"\"combine lists of dicts\n\n Parameters\n ----------\n d : dict or list[dict]\n keys : list\n keys to combine (all if None)\n deepcopy: bool\n deepcopy values\n\n Example\n -------\n >>> from pprint import pprint\n >>> d = {'path_key': {'a': 1, 'split': [{'x': 1, 'y': 3}, {'x': 2, 'y': 4}]}}\n >>> pprint(combine_lists(d,['split']))\n {'path_key': {'a': 1, 'split': {'x': [1, 2], 'y': [3, 4]}}}\n\n >>> combine_lists([{\"a\":2}, {\"a\":1}])\n {'a': [2, 1]}\n\n\n \"\"\" # noqa: E501\n if isinstance(d, list):\n init_list = True\n d = {'dummy_key843': d}\n else:\n init_list = False\n\n flattened = flatten(d, list_of_dicts=None)\n for key, value in list(flattened.items()):\n if keys is not None:\n try:\n if not key[-1] in keys:\n continue\n except Exception:\n continue\n if not isinstance(value, list):\n continue\n if not all([is_dict_like(d) for d in value]):\n continue\n newd = {}\n for subdic in value:\n for subk, subv in subdic.items():\n if subk not in newd:\n newd[subk] = []\n newd[subk].append(subv)\n flattened[key] = newd\n\n final = unflatten(flattened, list_of_dicts=None, deepcopy=deepcopy)\n\n if init_list:\n return list(final.values())[0]\n else:\n return final"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef list_to_dict(lst, key=None, remove_key=True):\n\n assert all([is_dict_like(d) for d in lst])\n if key is not None:\n assert all([key in d for d in lst])\n new_dict = {}\n for i, d in enumerate(lst):\n d = unflatten(flatten(d))\n if key is None:\n new_dict[str(i)] = d\n else:\n if remove_key:\n k = d.pop(key)\n else:\n k = d[key]\n new_dict[k] = d\n\n return new_dict", "response": "convert a list of dicts to a dict with root keys"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef diff(new_dict, old_dict, iter_prefix='__iter__',\n np_allclose=False, **kwargs):\n \"\"\" return the difference between two dict_like objects\n\n Parameters\n ----------\n new_dict: dict\n old_dict: dict\n iter_prefix: str\n prefix to use for list and tuple indexes\n np_allclose: bool\n if True, try using numpy.allclose to assess differences\n **kwargs:\n keyword arguments to parse to numpy.allclose\n\n Returns\n -------\n outcome: dict\n Containing none or more of:\n\n - \"insertions\" : list of (path, val)\n - \"deletions\" : list of (path, val)\n - \"changes\" : list of (path, (val1, val2))\n - \"uncomparable\" : list of (path, (val1, val2))\n\n Examples\n --------\n >>> from pprint import pprint\n\n >>> diff({'a':1},{'a':1})\n {}\n\n >>> pprint(diff({'a': 1, 'b': 2, 'c': 5},{'b': 3, 'c': 4, 'd': 6}))\n {'changes': [(('b',), (2, 3)), (('c',), (5, 4))],\n 'deletions': [(('d',), 6)],\n 'insertions': [(('a',), 1)]}\n\n >>> pprint(diff({'a': [{\"b\":1}, {\"c\":2}, 1]},{'a': [{\"b\":1}, {\"d\":2}, 2]}))\n {'changes': [(('a', '__iter__2'), (1, 2))],\n 'deletions': [(('a', '__iter__1', 'd'), 2)],\n 'insertions': [(('a', '__iter__1', 'c'), 2)]}\n\n >>> diff({'a':1}, {'a':1+1e-10})\n {'changes': [(('a',), (1, 1.0000000001))]}\n\n >>> diff({'a':1}, {'a':1+1e-10}, np_allclose=True)\n {}\n\n \"\"\"\n if np_allclose:\n try:\n import numpy\n except ImportError:\n raise ValueError(\"to use np_allclose, numpy must be installed\")\n\n dct1_flat = flatten(new_dict, all_iters=iter_prefix)\n dct2_flat = flatten(old_dict, all_iters=iter_prefix)\n\n outcome = {'insertions': [], 'deletions': [],\n 'changes': [], 'uncomparable': []}\n\n for path, val in dct1_flat.items():\n if path not in dct2_flat:\n outcome['insertions'].append((path, val))\n continue\n other_val = dct2_flat.pop(path)\n if np_allclose:\n try:\n if numpy.allclose(val, other_val, **kwargs):\n continue\n except Exception:\n pass\n try:\n if val != other_val:\n outcome['changes'].append((path, (val, other_val)))\n except Exception:\n outcome['uncomparable'].append((path, (val, other_val)))\n\n for path2, val2 in dct2_flat.items():\n outcome['deletions'].append((path2, val2))\n\n # remove any empty lists and sort\n for key in list(outcome.keys()):\n if not outcome[key]:\n outcome.pop(key)\n try:\n outcome[key] = sorted(outcome[key])\n except Exception:\n pass\n\n return outcome", "response": "Returns the difference between two dict_like objects"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\noutput dict to json", "response": "def to_json(dct, jfile, overwrite=False, dirlevel=0, sort_keys=True, indent=2,\n default_name='root.json', **kwargs):\n \"\"\" output dict to json\n\n Parameters\n ----------\n dct : dict\n jfile : str or file_like\n if file_like, must have write method\n overwrite : bool\n whether to overwrite existing files\n dirlevel : int\n if jfile is path to folder,\n defines how many key levels to set as sub-folders\n sort_keys : bool\n if true then the output of dictionaries will be sorted by key\n indent : int\n if non-negative integer, then JSON array elements and object members\n will be pretty-printed on new lines with that indent level spacing.\n kwargs : dict\n keywords for json.dump\n\n Examples\n --------\n\n >>> from jsonextended.utils import MockPath\n >>> file_obj = MockPath('test.json',is_file=True,exists=False)\n >>> dct = {'a':{'b':1}}\n >>> to_json(dct, file_obj)\n >>> print(file_obj.to_string())\n File(\"test.json\") Contents:\n {\n \"a\": {\n \"b\": 1\n }\n }\n\n >>> from jsonextended.utils import MockPath\n >>> folder_obj = MockPath()\n >>> dct = {'x':{'a':{'b':1},'c':{'d':3}}}\n >>> to_json(dct, folder_obj, dirlevel=0,indent=None)\n >>> print(folder_obj.to_string(file_content=True))\n Folder(\"root\")\n File(\"x.json\") Contents:\n {\"a\": {\"b\": 1}, \"c\": {\"d\": 3}}\n\n >>> folder_obj = MockPath()\n >>> to_json(dct, folder_obj, dirlevel=1,indent=None)\n >>> print(folder_obj.to_string(file_content=True))\n Folder(\"root\")\n Folder(\"x\")\n File(\"a.json\") Contents:\n {\"b\": 1}\n File(\"c.json\") Contents:\n {\"d\": 3}\n\n\n \"\"\"\n if hasattr(jfile, 'write'):\n json.dump(dct, jfile, sort_keys=sort_keys,\n indent=indent, default=encode)\n return\n\n if isinstance(jfile, basestring):\n path = pathlib.Path(jfile)\n else:\n path = jfile\n\n file_attrs = ['exists', 'is_dir', 'is_file', 'touch', 'open']\n if not all([hasattr(path, attr) for attr in file_attrs]):\n raise ValueError(\n 'jfile should be a str or file_like object: {}'.format(jfile))\n\n if path.is_file() and path.exists() and not overwrite:\n raise IOError('jfile already exists and '\n 'overwrite is set to false: {}'.format(jfile))\n\n if not path.is_dir() and dirlevel <= 0:\n path.touch() # try to create file if doesn't already exist\n with path.open('w') as outfile:\n outfile.write(unicode(json.dumps(\n dct, sort_keys=sort_keys,\n indent=indent, default=encode, **kwargs)))\n return\n\n if not path.is_dir():\n path.mkdir()\n dirlevel -= 1\n\n # if one or more values if not a nested dict\n if not all([hasattr(v, 'items') for v in dct.values()]):\n newpath = path.joinpath(default_name)\n newpath.touch()\n with newpath.open('w') as outfile:\n outfile.write(unicode(json.dumps(\n dct, sort_keys=sort_keys,\n indent=indent, default=encode, **kwargs)))\n return\n\n for key, val in dct.items():\n if dirlevel <= 0:\n newpath = path.joinpath('{}.json'.format(key))\n newpath.touch()\n with newpath.open('w') as outfile:\n outfile.write(unicode(json.dumps(\n val, ensure_ascii=False, sort_keys=sort_keys,\n indent=indent, default=encode, **kwargs)))\n else:\n newpath = path.joinpath('{}'.format(key))\n if not newpath.exists():\n newpath.mkdir()\n to_json(val, newpath, overwrite=overwrite, dirlevel=dirlevel - 1,\n sort_keys=sort_keys, indent=indent,\n default_name='{}.json'.format(key), **kwargs)"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef dump(dct, jfile, overwrite=False, dirlevel=0, sort_keys=True,\n indent=2, default_name='root.json', **kwargs):\n \"\"\" output dict to json\n\n Parameters\n ----------\n dct : dict\n jfile : str or file_like\n if file_like, must have write method\n overwrite : bool\n whether to overwrite existing files\n dirlevel : int\n if jfile is path to folder,\n defines how many key levels to set as sub-folders\n sort_keys : bool\n if true then the output of dictionaries will be sorted by key\n indent : int\n if non-negative integer, then JSON array elements and object members\n will be pretty-printed on new lines with that indent level spacing.\n kwargs : dict\n keywords for json.dump\n \"\"\"\n to_json(dct, jfile, overwrite=overwrite, dirlevel=dirlevel,\n sort_keys=sort_keys, indent=indent,\n default_name=default_name, **kwargs)", "response": "dump dict to json"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nexpands a collection of items in a dict in which a key will be inserted or expanded.", "response": "def expand_collection_in_dict(d, key, new_items, no_duplicates=True):\n \"\"\"\n Parameters\n d: dict\n dict in which a key will be inserted/expanded\n key: hashable\n key in d\n new_items: iterable\n d[key] will be extended with items in new_items\n no_duplicates: bool\n avoid inserting duplicates in d[key] (default: True)\n \"\"\"\n if key in d:\n if no_duplicates:\n new_items = filter(lambda x: x not in d[key], new_items)\n if isinstance(d[key], set):\n map(d[key].add, new_items)\n elif isinstance(d[key], list):\n map(d[key].append, new_items)\n else:\n d[key] = d[key] + new_items\n else:\n d[key] = new_items"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\naugment shutil. copy with extra options and slightly slightly CTYPE modified behaviour", "response": "def copy(src, dst, only_update=False, copystat=True, cwd=None,\n dest_is_dir=False, create_dest_dirs=False, logger=None):\n \"\"\"\n Augmented shutil.copy with extra options and slightly\n modified behaviour\n\n Parameters\n ==========\n src: string\n path to source file\n dst: string\n path to destingation\n only_update: bool\n only copy if source is newer than destination\n (returns None if it was newer), default: False\n copystat: bool\n See shutil.copystat. default: True\n cwd: string\n Path to working directory (root of relative paths)\n dest_is_dir: bool\n ensures that dst is treated as a directory. default: False\n create_dest_dirs: bool\n creates directories if needed.\n logger: logging.Looger\n debug level info emitted. Passed onto make_dirs.\n\n Returns\n =======\n Path to the copied file.\n\n \"\"\"\n # Handle virtual working directory\n if cwd:\n if not os.path.isabs(src):\n src = os.path.join(cwd, src)\n if not os.path.isabs(dst):\n dst = os.path.join(cwd, dst)\n\n # Make sure source file extists\n if not os.path.exists(src):\n # Source needs to exist\n msg = \"Source: `{}` does not exist\".format(src)\n raise FileNotFoundError(msg)\n\n # We accept both (re)naming destination file _or_\n # passing a (possible non-existant) destination directory\n if dest_is_dir:\n if not dst[-1] == '/':\n dst = dst+'/'\n else:\n if os.path.exists(dst) and os.path.isdir(dst):\n dest_is_dir = True\n\n if dest_is_dir:\n dest_dir = dst\n dest_fname = os.path.basename(src)\n dst = os.path.join(dest_dir, dest_fname)\n else:\n dest_dir = os.path.dirname(dst)\n dest_fname = os.path.basename(dst)\n\n if not os.path.exists(dest_dir):\n if create_dest_dirs:\n make_dirs(dest_dir, logger=logger)\n else:\n msg = \"You must create directory first.\"\n raise FileNotFoundError(msg)\n\n if only_update:\n if not missing_or_other_newer(dst, src):\n if logger:\n logger.debug(\n \"Did not copy {} to {} (source not newer)\".format(\n src, dst))\n return\n\n if os.path.islink(dst):\n if os.path.abspath(os.path.realpath(dst)) == \\\n os.path.abspath(dst):\n pass # destination is a symlic pointing to src\n else:\n if logger:\n logger.debug(\"Copying {} to {}\".format(src, dst))\n shutil.copy(src, dst)\n if copystat:\n shutil.copystat(src, dst)\n return dst"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\ncomputing the md5 hash of a file.", "response": "def md5_of_file(path, nblocks=128):\n \"\"\"\n Computes the md5 hash of a file.\n\n Parameters\n ==========\n path: string\n path to file to compute hash of\n\n Returns\n =======\n hashlib md5 hash object. Use .digest() or .hexdigest()\n on returned object to get binary or hex encoded string.\n \"\"\"\n md = md5()\n with open(path, 'rb') as f:\n for chunk in iter(lambda: f.read(nblocks*md.block_size), b''):\n md.update(chunk)\n return md"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef missing_or_other_newer(path, other_path, cwd=None):\n cwd = cwd or '.'\n path = get_abspath(path, cwd=cwd)\n other_path = get_abspath(other_path, cwd=cwd)\n if not os.path.exists(path):\n return True\n if os.path.getmtime(other_path) - 1e-6 >= os.path.getmtime(path):\n # 1e-6 is needed beacuse http://stackoverflow.com/questions/17086426/\n return True\n return False", "response": "Returns True if path is non - existant or older than other_path."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nimport a module from a file.", "response": "def import_module_from_file(filename, only_if_newer_than=None):\n \"\"\"\n Imports (cython generated) shared object file (.so)\n\n Provide a list of paths in `only_if_newer_than` to check\n timestamps of dependencies. import_ raises an ImportError\n if any is newer.\n\n Word of warning: Python's caching or the OS caching (unclear to author)\n is horrible for reimporting same path of an .so file. It will\n not detect the new time stamp nor new checksum but will use old\n module.\n\n Use unique names for this reason.\n\n Parameters\n ==========\n filename: string\n path to shared object\n only_if_newer_than: iterable of strings\n paths to dependencies of the shared object\n\n Raises\n ======\n ImportError if any of the files specified in only_if_newer_than are newer\n than the file given by filename.\n \"\"\"\n import imp\n path, name = os.path.split(filename)\n name, ext = os.path.splitext(name)\n name = name.split('.')[0]\n fobj, filename, data = imp.find_module(name, [path])\n if only_if_newer_than:\n for dep in only_if_newer_than:\n if os.path.getmtime(filename) < os.path.getmtime(dep):\n raise ImportError(\"{} is newer than {}\".format(dep, filename))\n mod = imp.load_module(name, fobj, filename, data)\n return mod"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nfind the first binary that can be used for the given list of candidates.", "response": "def find_binary_of_command(candidates):\n \"\"\"\n Calls `find_executable` from distuils for\n provided candidates and returns first hit.\n If no candidate mathces, a RuntimeError is raised\n \"\"\"\n from distutils.spawn import find_executable\n for c in candidates:\n binary_path = find_executable(c)\n if c and binary_path:\n return c, binary_path\n raise RuntimeError('No binary located for candidates: {}'.format(\n candidates))"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\ncheck if a Cython source file is a C ++ file.", "response": "def pyx_is_cplus(path):\n \"\"\"\n Inspect a Cython source file (.pyx) and look for comment line like:\n\n # distutils: language = c++\n\n Returns True if such a file is present in the file, else False.\n \"\"\"\n for line in open(path, 'rt'):\n if line.startswith('#') and '=' in line:\n splitted = line.split('=')\n if len(splitted) != 2:\n continue\n lhs, rhs = splitted\n if lhs.strip().split()[-1].lower() == 'language' and \\\n rhs.strip().split()[0].lower() == 'c++':\n return True\n return False"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nget value of key in metadata file dict.", "response": "def get_from_metadata_file(cls, dirpath, key):\n \"\"\"\n Get value of key in metadata file dict.\n \"\"\"\n fullpath = os.path.join(dirpath, cls.metadata_filename)\n if os.path.exists(fullpath):\n d = pickle.load(open(fullpath, 'rb'))\n return d[key]\n else:\n raise FileNotFoundError(\n \"No such file: {0}\".format(fullpath))"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nsaving key value in metadata file dict.", "response": "def save_to_metadata_file(cls, dirpath, key, value):\n \"\"\"\n Store `key: value` in metadata file dict.\n \"\"\"\n fullpath = os.path.join(dirpath, cls.metadata_filename)\n if os.path.exists(fullpath):\n d = pickle.load(open(fullpath, 'rb'))\n d.update({key: value})\n pickle.dump(d, open(fullpath, 'wb'))\n else:\n pickle.dump({key: value}, open(fullpath, 'wb'))"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef view_interfaces(category=None):\n if category is not None:\n return sorted(_plugins_interface[category][:])\n else:\n return {k: v[:] for k, v in _plugins_interface.items()}", "response": "returns a view of the plugin minimal class attribute interface"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nreturning a view of the loaded plugin names and descriptions for a single plugin category", "response": "def view_plugins(category=None):\n \"\"\" return a view of the loaded plugin names and descriptions\n\n Parameters\n ----------\n category : None or str\n if str, apply for single plugin category\n\n Examples\n --------\n\n >>> from pprint import pprint\n >>> pprint(view_plugins())\n {'decoders': {}, 'encoders': {}, 'parsers': {}}\n\n >>> class DecoderPlugin(object):\n ... plugin_name = 'example'\n ... plugin_descript = 'a decoder for dicts containing _example_ key'\n ... dict_signature = ('_example_',)\n ...\n >>> errors = load_plugin_classes([DecoderPlugin])\n\n >>> pprint(view_plugins())\n {'decoders': {'example': 'a decoder for dicts containing _example_ key'},\n 'encoders': {},\n 'parsers': {}}\n\n >>> view_plugins('decoders')\n {'example': 'a decoder for dicts containing _example_ key'}\n\n >>> unload_all_plugins()\n\n \"\"\"\n if category is not None:\n if category == 'parsers':\n return {\n name: {\"descript\": klass.plugin_descript,\n \"regex\": klass.file_regex}\n for name, klass in _all_plugins[category].items()\n }\n return {\n name: klass.plugin_descript\n for name, klass in _all_plugins[category].items()\n }\n else:\n return {cat: {name: klass.plugin_descript\n for name, klass in plugins.items()}\n for cat, plugins in _all_plugins.items()}"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nunloads a single plugin from the _all_plugins list.", "response": "def unload_plugin(name, category=None):\n \"\"\" remove single plugin\n\n Parameters\n ----------\n name : str\n plugin name\n category : str\n plugin category\n\n Examples\n --------\n\n >>> from pprint import pprint\n >>> pprint(view_plugins())\n {'decoders': {}, 'encoders': {}, 'parsers': {}}\n\n >>> class DecoderPlugin(object):\n ... plugin_name = 'example'\n ... plugin_descript = 'a decoder for dicts containing _example_ key'\n ... dict_signature = ('_example_',)\n ...\n >>> errors = load_plugin_classes([DecoderPlugin],category='decoders')\n\n >>> pprint(view_plugins())\n {'decoders': {'example': 'a decoder for dicts containing _example_ key'},\n 'encoders': {},\n 'parsers': {}}\n\n >>> unload_plugin('example','decoders')\n >>> pprint(view_plugins())\n {'decoders': {}, 'encoders': {}, 'parsers': {}}\n\n \"\"\"\n if category is not None:\n _all_plugins[category].pop(name)\n else:\n for cat in _all_plugins:\n if name in _all_plugins[cat]:\n _all_plugins[cat].pop(name)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nloading all plugins from a list of classes.", "response": "def load_plugin_classes(classes, category=None, overwrite=False):\n \"\"\" load plugins from class objects\n\n Parameters\n ----------\n classes: list\n list of classes\n category : None or str\n if str, apply for single plugin category\n overwrite : bool\n if True, allow existing plugins to be overwritten\n\n Examples\n --------\n\n >>> from pprint import pprint\n >>> pprint(view_plugins())\n {'decoders': {}, 'encoders': {}, 'parsers': {}}\n\n >>> class DecoderPlugin(object):\n ... plugin_name = 'example'\n ... plugin_descript = 'a decoder for dicts containing _example_ key'\n ... dict_signature = ('_example_',)\n ...\n >>> errors = load_plugin_classes([DecoderPlugin])\n\n >>> pprint(view_plugins())\n {'decoders': {'example': 'a decoder for dicts containing _example_ key'},\n 'encoders': {},\n 'parsers': {}}\n\n >>> unload_all_plugins()\n\n \"\"\"\n load_errors = []\n for klass in classes:\n for pcat, pinterface in _plugins_interface.items():\n if category is not None and not pcat == category:\n continue\n if all([hasattr(klass, attr) for attr in pinterface]):\n if klass.plugin_name in _all_plugins[pcat] and not overwrite:\n err = '{0} is already set for {1}'.format(\n klass.plugin_name, pcat)\n load_errors.append((klass.__name__, '{}'.format(err)))\n continue\n _all_plugins[pcat][klass.plugin_name] = klass()\n else:\n load_errors.append((\n klass.__name__,\n 'does not match {} interface: {}'.format(pcat, pinterface)\n ))\n return load_errors"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef plugins_context(classes, category=None):\n original = {cat: list(_all_plugins[cat].keys()) for cat in _all_plugins}\n load_plugin_classes(classes, category, overwrite=True)\n # if errors:\n # for cat in _all_plugins:\n # for name, kls in list(_all_plugins[cat].items()):\n # if name not in original[cat]:\n # _all_plugins[cat].pop(name)\n # raise RuntimeError(\n # \"errors occurred while loading plugins: {}\".format(errors))\n yield\n for cat in _all_plugins:\n for name, kls in list(_all_plugins[cat].items()):\n if name not in original[cat]:\n _all_plugins[cat].pop(name)", "response": "context manager to load plugin classes then unload on exit"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nloading plugins from a directory.", "response": "def load_plugins_dir(path, category=None, overwrite=False):\n \"\"\" load plugins from a directory\n\n Parameters\n ----------\n path : str or path_like\n category : None or str\n if str, apply for single plugin category\n overwrite : bool\n if True, allow existing plugins to be overwritten\n\n \"\"\"\n # get potential plugin python files\n if hasattr(path, 'glob'):\n pypaths = path.glob('*.py')\n else:\n pypaths = glob.glob(os.path.join(path, '*.py'))\n\n load_errors = []\n for pypath in pypaths:\n # use uuid to ensure no conflicts in name space\n mod_name = str(uuid.uuid4())\n try:\n if hasattr(pypath, 'resolve'):\n # Make the path absolute, resolving any symlinks\n pypath = pypath.resolve()\n\n with warnings.catch_warnings(record=True):\n warnings.filterwarnings(\"ignore\", category=ImportWarning)\n\n # for MockPaths\n if hasattr(pypath, 'maketemp'):\n with pypath.maketemp() as f:\n module = load_source(mod_name, f.name)\n else:\n module = load_source(mod_name, str(pypath))\n\n except Exception as err:\n load_errors.append((str(pypath), 'Load Error: {}'.format(err)))\n continue\n\n # only get classes that are local to the module\n class_members = inspect.getmembers(module, inspect.isclass)\n classes = [klass for klass_name, klass in class_members if\n klass.__module__ == mod_name]\n load_errors += load_plugin_classes(classes, category, overwrite)\n\n return load_errors"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef load_builtin_plugins(category=None, overwrite=False):\n load_errors = []\n for cat, path in _plugins_builtin.items():\n if cat != category and category is not None:\n continue\n load_errors += load_plugins_dir(path, cat, overwrite=overwrite)\n return load_errors", "response": "load plugins from builtin directories"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef encode(obj, outtype='json', raise_error=False):\n for encoder in get_plugins('encoders').values():\n if (isinstance(obj, encoder.objclass)\n and hasattr(encoder, 'to_{}'.format(outtype))):\n return getattr(encoder, 'to_{}'.format(outtype))(obj)\n break\n\n if raise_error:\n raise ValueError(\n \"No JSON serializer is available for\"\n \"{0} (of type {1})\".format(obj, type(obj)))\n else:\n return obj", "response": "Encodes the object obj to a new type in the specified outtype."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef decode(dct, intype='json', raise_error=False):\n for decoder in get_plugins('decoders').values():\n if (set(list(decoder.dict_signature)).issubset(dct.keys())\n and hasattr(decoder, 'from_{}'.format(intype))\n and getattr(decoder, 'allow_other_keys', False)):\n return getattr(decoder, 'from_{}'.format(intype))(dct)\n break\n elif (sorted(list(decoder.dict_signature)) == sorted(dct.keys())\n and hasattr(decoder, 'from_{}'.format(intype))):\n return getattr(decoder, 'from_{}'.format(intype))(dct)\n break\n\n if raise_error:\n raise ValueError('no suitable plugin found for: {}'.format(dct))\n else:\n return dct", "response": "decode dict objects via decoder plugins to new type\n "} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\ntests if parser plugin available for fpath", "response": "def parser_available(fpath):\n \"\"\" test if parser plugin available for fpath\n\n Examples\n --------\n\n >>> load_builtin_plugins('parsers')\n []\n >>> test_file = StringIO('{\"a\":[1,2,3.4]}')\n >>> test_file.name = 'test.json'\n >>> parser_available(test_file)\n True\n >>> test_file.name = 'test.other'\n >>> parser_available(test_file)\n False\n\n >>> unload_all_plugins()\n\n \"\"\"\n if isinstance(fpath, basestring):\n fname = fpath\n elif hasattr(fpath, 'open') and hasattr(fpath, 'name'):\n fname = fpath.name\n elif hasattr(fpath, 'readline') and hasattr(fpath, 'name'):\n fname = fpath.name\n else:\n raise ValueError(\n 'fpath should be a str or file_like object: {}'.format(fpath))\n\n for parser in get_plugins('parsers').values():\n if fnmatch(fname, parser.file_regex):\n return True\n\n return False"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nparses file contents via parser plugins to dict like object", "response": "def parse(fpath, **kwargs):\n \"\"\" parse file contents, via parser plugins, to dict like object\n NB: the longest file regex will be used from plugins\n\n Parameters\n ----------\n fpath : file_like\n string, object with 'open' and 'name' attributes, or\n object with 'readline' and 'name' attributes\n kwargs :\n to pass to parser plugin\n\n Examples\n --------\n\n >>> load_builtin_plugins('parsers')\n []\n\n >>> from pprint import pformat\n\n >>> json_file = StringIO('{\"a\":[1,2,3.4]}')\n >>> json_file.name = 'test.json'\n\n >>> dct = parse(json_file)\n >>> print(pformat(dct).replace(\"u'\",\"'\"))\n {'a': [1, 2, 3.4]}\n\n >>> reset = json_file.seek(0)\n >>> from decimal import Decimal\n >>> dct = parse(json_file, parse_float=Decimal,other=1)\n >>> print(pformat(dct).replace(\"u'\",\"'\"))\n {'a': [1, 2, Decimal('3.4')]}\n\n >>> class NewParser(object):\n ... plugin_name = 'example'\n ... plugin_descript = 'loads test.json files'\n ... file_regex = 'test.json'\n ... def read_file(self, file_obj, **kwargs):\n ... return {'example':1}\n >>> load_plugin_classes([NewParser],'parsers')\n []\n >>> reset = json_file.seek(0)\n >>> parse(json_file)\n {'example': 1}\n\n >>> unload_all_plugins()\n\n \"\"\"\n if isinstance(fpath, basestring):\n fname = fpath\n elif hasattr(fpath, 'open') and hasattr(fpath, 'name'):\n fname = fpath.name\n elif hasattr(fpath, 'readline') and hasattr(fpath, 'name'):\n fname = fpath.name\n else:\n raise ValueError(\n 'fpath should be a str or file_like object: {}'.format(fpath))\n\n parser_dict = {\n plugin.file_regex: plugin\n for plugin in get_plugins('parsers').values()}\n\n # find longest match first\n for regex in sorted(parser_dict.keys(), key=len, reverse=True):\n parser = parser_dict[regex]\n if fnmatch(fname, regex):\n if isinstance(fpath, basestring):\n with open(fpath, 'r') as file_obj:\n data = parser.read_file(file_obj, **kwargs)\n elif hasattr(fpath, 'open'):\n with fpath.open('r') as file_obj:\n data = parser.read_file(file_obj, **kwargs)\n elif hasattr(fpath, 'readline'):\n data = parser.read_file(fpath, **kwargs)\n return data\n\n raise ValueError('{} does not match any regex'.format(fname))"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef colortxt(text, color=None, on_color=None, attrs=None):\n _RESET = '\\033[0m'\n __ISON = True\n if __ISON and os.getenv('ANSI_COLORS_DISABLED') is None:\n fmt_str = '\\033[%dm%s'\n if color is not None:\n text = fmt_str % (_COLORS[color], text)\n\n if on_color is not None:\n text = fmt_str % (_HIGHLIGHTS[on_color], text)\n\n if attrs is not None:\n for attr in attrs:\n text = fmt_str % (_ATTRIBUTES[attr], text)\n\n text += _RESET\n return text", "response": "Colorize text.\n\n Available text colors:\n red, green, yellow, blue, magenta, cyan, white.\n\n Available text highlights:\n on_red, on_green, on_yellow, on_blue, on_magenta, on_cyan, on_white.\n\n Available attributes:\n bold, dark, underline, blink, reverse, concealed.\n\n Examples\n --------\n >>> txt = colortxt('Hello, World!', 'red', 'on_grey', ['bold'])\n >>> print(txt)\n \\x1b[1m\\x1b[40m\\x1b[31mHello, World!\\x1b[0m"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nreading delimited text from zipfile.", "response": "def from_zipfile(cls, path, filename, encoding, dialect, fields, converters):\n \"\"\"Read delimited text from zipfile.\"\"\"\n\n stream = ZipReader(path, filename).readlines(encoding)\n return cls(stream, dialect, fields, converters)"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef from_file(cls, path, encoding, dialect, fields, converters, field_index):\n\n return cls(open(path, 'r', encoding=encoding), dialect, fields, converters, field_index)", "response": "Read delimited text from a text file."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nread content into byte string.", "response": "def read_bytes(self):\n \"\"\"Read content into byte string.\"\"\"\n\n with ZipFile(self.path, mode='r') as archive:\n return archive.read(self.filename)"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nreads content into byte str line iterator.", "response": "def readlines_bytes(self):\n \"\"\"Read content into byte str line iterator.\"\"\"\n\n with open_zipfile_archive(self.path, self.filename) as file:\n for line in file:\n yield line.rstrip(b'\\r\\n')"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nparsing a XML file into a Tree object.", "response": "def xmlparse(self, file):\n \"\"\" Parse a XML file\n\n :param file: Opened File\n :return: Tree\n \"\"\"\n if self.CACHE_FULL_TEI is True:\n return self.get_or(\n _cache_key(\"Nautilus\", self.name, \"File\", \"Tree\", file.name),\n super(ProtoNautilusCtsResolver, self).xmlparse, file\n )\n return super(ProtoNautilusCtsResolver, self).xmlparse(file)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef get_or(self, cache_key, callback, *args, **kwargs):\n cached = self.cache.get(cache_key)\n if cached is not None:\n return cached\n else:\n try:\n output = callback(*args, **kwargs)\n except MyCapytain.errors.UnknownCollection as E:\n raise UnknownCollection(str(E))\n except Exception as E:\n raise E\n self.cache.set(cache_key, output, self.TIMEOUT)\n return output", "response": "Get or set the cache using callback and arguments\n\n "} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nreads a text object given an identifier and a path.", "response": "def read(self, identifier, path=None):\n \"\"\" Read a text object given an identifier and a path\n\n :param identifier: Identifier of the text\n :param path: Path of the text files\n :return: Text\n \"\"\"\n if self.CACHE_FULL_TEI is True:\n o = self.cache.get(_cache_key(self.texts_parsed_cache_key, identifier))\n if o is not None:\n return o\n else:\n with open(path) as f:\n o = Text(urn=identifier, resource=self.xmlparse(f))\n self.cache.set(_cache_key(self.texts_parsed_cache_key, identifier), o)\n else:\n with open(path) as f:\n o = Text(urn=identifier, resource=self.xmlparse(f))\n return o"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef parse(self, resource=None):\n if resource is None:\n resource = self.__resources__\n\n self.inventory = self.dispatcher.collection\n\n try:\n self._parse(resource)\n except MyCapytain.errors.UndispatchedTextError as E:\n if self.RAISE_ON_UNDISPATCHED is True:\n raise UndispatchedTextError(E)\n\n self.inventory = self.dispatcher.collection\n return self.inventory", "response": "Parse a list of directories ans\n\n "} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nretrieve the siblings of a textual node.", "response": "def getReffs(self, textId, level=1, subreference=None):\n \"\"\" Retrieve the siblings of a textual node\n\n :param textId: PrototypeText Identifier\n :type textId: str\n :param level: Depth for retrieval\n :type level: int\n :param subreference: Passage Reference\n :type subreference: str\n :return: List of references\n :rtype: [str]\n \"\"\"\n return self.get_or(\n self.__cache_key_reffs__(textId, level, subreference),\n super(ProtoNautilusCtsResolver, self).getReffs, textId, level, subreference\n )"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef getTextualNode(self, textId, subreference=None, prevnext=False, metadata=False):\n key = _cache_key(\"Nautilus\", self.name, \"Passage\", textId, subreference)\n o = self.cache.get(key)\n if o is not None:\n return o\n text, text_metadata = self.__getText__(textId)\n if subreference is not None:\n subreference = Reference(subreference)\n\n passage = text.getTextualNode(subreference)\n passage.set_metadata_from_collection(text_metadata)\n self.cache.set(key, passage)\n return passage", "response": "Retrieve a text node from the API and return it"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef getSiblings(self, textId, subreference):\n key = _cache_key(\"Nautilus\", self.name, \"Siblings\", textId, subreference)\n o = self.cache.get(key)\n if o is not None:\n return o\n passage = self.getTextualNode(textId, subreference, prevnext=True)\n siblings = passage.siblingsId\n self.cache.set(key, siblings)\n return siblings", "response": "Retrieve the siblings of a textual node."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef getMetadata(self, objectId=None, **filters):\n return self.get_or(\n _cache_key(\"Nautilus\", self.name, \"GetMetadata\", objectId),\n super(ProtoNautilusCtsResolver, self).getMetadata, objectId\n )", "response": "Request metadata about a text or a collection of objects."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _dispatch(self, textgroup, directory):\n self.dispatcher.dispatch(textgroup, path=directory)", "response": "Dispatches the object in the given directory to the appropriate handler"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nask for user input.", "response": "def ask(question, default_answer=False, default_answer_str=\"no\"):\n \"\"\"\n Ask for user input.\n\n This asks a yes/no question with a preset default.\n You can bypass the user-input and fetch the default answer, if\n you set\n\n Args:\n question: The question to ask on stdout.\n default_answer: The default value to return.\n default_answer_str:\n The default answer string that we present to the user.\n\n Tests:\n >>> os.putenv(\"TEST\", \"yes\"); ask(\"Test?\", default_answer=True)\n True\n >>> os.putenv(\"TEST\", \"yes\"); ask(\"Test?\", default_answer=False)\n False\n \"\"\"\n response = default_answer\n\n def should_ignore_tty():\n \"\"\"\n Check, if we want to ignore an opened tty result.\n \"\"\"\n ret_to_bool = {\"yes\": True, \"no\": False, \"true\": True, \"false\": False}\n envs = [os.getenv(\"CI\", default=\"no\"), os.getenv(\"TEST\", default=\"no\")]\n vals = [ret_to_bool[val] for val in envs if val in ret_to_bool]\n return any(vals)\n\n ignore_stdin_istty = should_ignore_tty()\n has_tty = sys.stdin.isatty() and not ignore_stdin_istty\n if has_tty:\n response = query_yes_no(question, default_answer_str)\n else:\n LOG.debug(\"NoTTY: %s -> %s\", question, response)\n\n return response"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef load_csv(path, delimiter=','):\n try:\n with open(path, 'rb') as csvfile:\n reader = DictReader(csvfile, delimiter=delimiter)\n for row in reader:\n yield row\n except (OSError, IOError):\n raise ClientException(\"File not found: {}\".format(path))", "response": "Load CSV file from path and yield CSV rows"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nloads a JSON file from path", "response": "def load_json(path):\n \"\"\"\n Load JSON file from path\n :param path: file path\n :return: A Python object (e.g. a dict)\n \"\"\"\n try:\n with open(path, 'r') as json_file:\n return json.load(json_file)\n except (OSError, IOError):\n raise ClientException(\"File not found: {}\".format(path))"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef partition_payload(data, key, thresh):\n data = data[key]\n for i in range(0, len(data), thresh):\n yield {key: data[i:i + thresh]}", "response": "Yield partitions of a payload by a threshold"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef search_auth_file(filename='dish.json'):\n if 'DHIS_HOME' in os.environ:\n return os.path.join(os.environ['DHIS_HOME'], filename)\n else:\n home_path = os.path.expanduser(os.path.join('~'))\n for root, dirs, files in os.walk(home_path):\n if filename in files:\n return os.path.join(root, filename)\n raise ClientException(\"'{}' not found - searched in $DHIS_HOME and in home folder\".format(filename))", "response": "Search filename in DHIS_HOME or user s home folder and return full path of the file"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef version_to_int(value):\n # remove '-SNAPSHOT'\n value = value.replace('-SNAPSHOT', '')\n # remove '-RCx'\n if '-RC' in value:\n value = value.split('-RC', 1)[0]\n try:\n return int(value.split('.')[1])\n except (ValueError, IndexError):\n return", "response": "Convert version info to integer"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef generate_uid():\n # first must be a letter\n first = random.choice(string.ascii_letters)\n # rest must be letters or numbers\n rest = ''.join(random.choice(string.ascii_letters + string.digits) for _ in range(10))\n return first + rest", "response": "Create DHIS2 UID matching to Regex\n ^[A-Za-z][A-Za-z0-9]{10}$\n :return: UID string"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef is_valid_uid(uid):\n pattern = r'^[A-Za-z][A-Za-z0-9]{10}$'\n if not isinstance(uid, string_types):\n return False\n return bool(re.compile(pattern).match(uid))", "response": "Check if the given UID is a valid DHIS2 UID."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef pretty_json(obj):\n if isinstance(obj, string_types):\n try:\n obj = json.loads(obj)\n except ValueError:\n raise ClientException(\"`obj` is not a json string\")\n json_str = json.dumps(obj, sort_keys=True, indent=2)\n print(highlight(json_str, JsonLexer(), TerminalFormatter()))", "response": "Print JSON with indentation and colours"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef commands(self):\n for child in self._children:\n with open(osp.join(child, YAML_REPORT_FILE)) as istr:\n command = yaml.safe_load(istr)['command']\n yield ' '.join(map(six.moves.shlex_quote, command))", "response": "Get all commands of the benchmark category"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nset current process environment according to execution environment and modules", "response": "def _module_env(self, execution):\n \"\"\"Set current process environment according\n to execution `environment` and `modules`\n \"\"\"\n env = copy.copy(os.environ)\n try:\n for mod in execution.get('modules') or []:\n Module.load(mod)\n os.environ.update(execution.get('environment') or {})\n yield\n finally:\n os.environ = env"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef gather_metrics(self, runs):\n for run_dirs in runs.values():\n with open(JSON_METRICS_FILE, 'w') as ostr:\n ostr.write('[\\n')\n for i in range(len(run_dirs)):\n with open(osp.join(run_dirs[i], YAML_REPORT_FILE)) as istr:\n data = yaml.safe_load(istr)\n data.pop('category', None)\n data.pop('command', None)\n data['id'] = run_dirs[i]\n json.dump(data, ostr, indent=2)\n if i != len(run_dirs) - 1:\n ostr.write(',')\n ostr.write('\\n')\n ostr.write(']\\n')", "response": "Write a JSON file with the result of every runs\nxid"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef _check_metrics(cls, schema, metrics):\n for name, value in metrics.items():\n metric = schema.get(name)\n if not metric:\n message = \"Unexpected metric '{}' returned\".format(name)\n raise Exception(message)\n cls._check_metric(schema, metric, name, value)", "response": "Ensure that returned metrics are properly exposed."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef add_inputs_to_object(obj, values):\n for item in obj.inputs:\n if hasattr(obj, item):\n # print(item)\n setattr(obj, item, values[item])", "response": "A generic function to load object parameters based on a dictionary list."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nloads sample data for the SoilStructureSystem object.", "response": "def load_sample_data(sss):\n \"\"\"\n Sample data for the SoilStructureSystem object\n :param sss:\n :return:\n \"\"\"\n\n load_soil_sample_data(sss.sp) # soil\n load_foundation_sample_data(sss.fd) # foundation\n load_structure_sample_data(sss.bd) # structure\n load_hazard_sample_data(sss.hz)"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef load_soil_sample_data(sp):\n # soil\n sp.g_mod = 60.0e6 # [Pa]\n sp.phi = 30 # [degrees]\n sp.relative_density = .40 # [decimal]\n sp.gwl = 2. # [m], ground water level\n sp.unit_dry_weight = 17000 # [N/m3]\n sp.unit_sat_weight = 18000 # [N/m3]\n sp.unit_weight_water = 9800 # [N/m3]\n sp.cohesion = 10.0 # [Pa]\n sp.poissons_ratio = 0.22\n sp.e_min = 0.55\n sp.e_max = 0.95\n sp.e_critical0 = 0.79 # Jin et al. 2015\n sp.p_critical0 = 0.7 # Jin et al. 2015\n sp.lamb_crl = 0.015", "response": "Load sample data for the Soil object."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nloading the sample data for the Foundation object", "response": "def load_foundation_sample_data(fd):\n \"\"\"\n Sample data for the Foundation object\n :param fd: Foundation Object\n :return:\n \"\"\"\n # foundation\n fd.width = 16.0 # m\n fd.length = 18.0 # m\n fd.depth = 0.0 # m\n fd.mass = 0.0"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nloading the sample data for the structure object", "response": "def load_structure_sample_data(st):\n \"\"\"\n Sample data for the Structure object\n :param st: Structure Object\n :return:\n \"\"\"\n # structure\n st.h_eff = 9.0 # m\n st.mass_eff = 120e3 # kg\n st.t_eff = 1.0 # s\n st.mass_ratio = 1.0"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nload sample data for the Hazard object.", "response": "def load_hazard_sample_data(hz):\n \"\"\"\n Sample data for the Hazard object\n :param hz: Hazard Object\n :return:\n \"\"\"\n # hazard\n hz.z_factor = 0.3 # Hazard factor\n hz.r_factor = 1.0 # Return period factor\n hz.n_factor = 1.0 # Near-fault factor\n hz.magnitude = 7.5 # Magnitude of earthquake\n hz.corner_period = 4.0 # s\n hz.corner_acc_factor = 0.55\n return hz"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef load_building_sample_data(bd):\n number_of_storeys = 6\n interstorey_height = 3.4 # m\n masses = 40.0e3 # kg\n\n bd.interstorey_heights = interstorey_height * np.ones(number_of_storeys)\n bd.floor_length = 18.0 # m\n bd.floor_width = 16.0 # m\n bd.storey_masses = masses * np.ones(number_of_storeys)", "response": "Load the sample data for the Building object."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef load_frame_building_sample_data():\n number_of_storeys = 6\n interstorey_height = 3.4 # m\n masses = 40.0e3 # kg\n n_bays = 3\n\n fb = models.BuildingFrame(number_of_storeys, n_bays)\n fb.interstorey_heights = interstorey_height * np.ones(number_of_storeys)\n fb.floor_length = 18.0 # m\n fb.floor_width = 16.0 # m\n fb.storey_masses = masses * np.ones(number_of_storeys) # kg\n\n fb.bay_lengths = [6., 6.0, 6.0]\n fb.set_beam_prop(\"depth\", [0.5, 0.5, 0.5], repeat=\"up\")\n fb.set_beam_prop(\"width\", [0.4, 0.4, 0.4], repeat=\"up\")\n fb.set_column_prop(\"width\", [0.5, 0.5, 0.5, 0.5], repeat=\"up\")\n fb.set_column_prop(\"depth\", [0.5, 0.5, 0.5, 0.5], repeat=\"up\")\n fb.n_seismic_frames = 3\n fb.n_gravity_frames = 0\n return fb", "response": "Load the sample data for the BuildingFrame object."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef _get_random_fp_raw():\n '''\n \u751f\u6210\u968f\u673a\u7684\u539f\u59cb\u6307\u7eb9\u5217\u8868\n '''\n fp_list = []\n fp_list.append(get_random_ua()) # ua\n fp_list.append('zh-CN') # language\n fp_list.append('24') # color depth\n fp_list.append(__get_random_screen_resolution())\n fp_list.append('-480') # time zone offsite\n fp_list.append('true') # session storage\n fp_list.append('true') # local storage\n fp_list.append('true') # indexed db\n fp_list.append('') # add behavior\n fp_list.append('function') # open database\n fp_list.append('') # cpu class\n fp_list.append('MacIntel') # platform\n fp_list.append('') # do not track\n fp_list.append(\n 'Widevine Content Decryption Module::Enables Widevine \\\n licenses for playback of HTML audio/video content. \\\n (version: 1.4.8.962)::application/x-ppapi-widevine-cdm~;'\n ) # plugin string\n\n return fp_list", "response": "Generate a random FPGA - style list of random FPGA - style values."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nget the raw version of the FPGA file.", "response": "def get_fp_raw():\n '''\n \u751f\u6210fp_raw_str\n '''\n fp_file_path = os.path.expanduser('~/.xunleipy_fp')\n fp_list = []\n try:\n with open(fp_file_path, 'r') as fp_file:\n fp_str = fp_file.readline()\n if len(fp_str) > 0:\n fp_list = fp_str.split('###')\n except IOError:\n pass\n\n if len(fp_list) < 14:\n fp_list = _get_random_fp_raw()\n fp_str = '###'.join(fp_list)\n with open(fp_file_path, 'w') as fp_file:\n fp_file.write(fp_str)\n\n source = fp_str.strip()\n if six.PY3:\n source = source.encode('utf-8')\n\n fp_raw = base64.b64encode(source)\n return fp_raw"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\ndefines functions composition like f g", "response": "def compose(*functions):\n \"\"\"Define functions composition like f \u2218 g \u2218 h\n :return: callable object that will perform\n function composition of callables given in argument.\n \"\"\"\n\n def _compose2(f, g): # pylint: disable=invalid-name\n return lambda x: f(g(x))\n\n return functools.reduce(_compose2, functions, lambda x: x)"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef chunks(iterator, size):\n for item in iterator:\n yield [item] + list(islice(iterator, size - 1))", "response": "Split an iterator into chunks with size elements each."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nyields the sub - group element dictionary that match a regex pattern.", "response": "def match_subgroup(sequence, pattern):\n \"\"\"Yield the sub-group element dictionary that match a regex pattern.\"\"\"\n\n for element in sequence:\n match = re.match(pattern, element)\n\n if match:\n yield match.groupdict()"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef add_regex_start_end(pattern_function):\n\n @wraps(pattern_function)\n def func_wrapper(*args, **kwargs):\n return r'^{}$'.format(pattern_function(*args, **kwargs))\n return func_wrapper", "response": "Decorator for adding regex start and end characters."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nconverts a foundation stress to an equivalent mass.", "response": "def convert_stress_to_mass(q, width, length, gravity):\n \"\"\"\n Converts a foundation stress to an equivalent mass.\n\n :param q: applied stress [Pa]\n :param width: foundation width [m]\n :param length: foundation length [m]\n :param gravity: applied gravitational acceleration [m/s2]\n :return:\n \"\"\"\n mass = q * width * length / gravity\n return mass"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef add_to_obj(obj, dictionary, objs=None, exceptions=None, verbose=0):\n if exceptions is None:\n exceptions = []\n for item in dictionary:\n if item in exceptions:\n continue\n if dictionary[item] is not None:\n if verbose:\n print(\"process: \", item, dictionary[item])\n key, value = get_key_value(dictionary[item], objs, key=item)\n if verbose:\n print(\"assign: \", key, value)\n try:\n setattr(obj, key, value)\n except AttributeError:\n raise AttributeError(\"Can't set {0}={1} on object: {2}\".format(key, value, obj))", "response": "Adds the key - value pairs to an object."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef create_quantiles(items: Sequence, lower_bound, upper_bound):\n\n interval = (upper_bound - lower_bound) / len(items)\n\n quantiles = ((g, (x - interval, x)) for g, x in\n zip(items, accumulate(repeat(interval, len(items)))))\n\n return quantiles", "response": "Create quantile start and end boundaries."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef tupleize(element, ignore_types=(str, bytes)):\n if hasattr(element, '__iter__') and not isinstance(element, ignore_types):\n return element\n else:\n return tuple((element,))", "response": "Cast a single element to a tuple."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\ncreating dictionaries mapping fields to record data.", "response": "def dictionize(fields: Sequence, records: Sequence) -> Generator:\n \"\"\"Create dictionaries mapping fields to record data.\"\"\"\n\n return (dict(zip(fields, rec)) for rec in records)"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\ntransform dictionary to unpack values to map to respective key.", "response": "def flip_iterable_dict(d: dict) -> dict:\n \"\"\"Transform dictionary to unpack values to map to respective key.\"\"\"\n\n value_keys = disjoint_union((cartesian_product((v, k))\n for k, v in d.items()))\n return dict(value_keys)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef get_signed_area(self):\n accum = 0.0\n for i in range(len(self.verts)):\n j = (i + 1) % len(self.verts)\n accum += (\n self.verts[j][0] * self.verts[i][1] -\n self.verts[i][0] * self.verts[j][1])\n return accum / 2", "response": "Return the area of a simple polygon."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef call_next(self, *args, **kwargs) -> t.List[run.RunInfo]:\n all_results = []\n for ext in self.next_extensions:\n LOG.debug(\" %s \", ext)\n results = ext(*args, **kwargs)\n LOG.debug(\" %s => %s\", ext, results)\n if results is None:\n LOG.warning(\"No result from: %s\", ext)\n continue\n result_list = []\n if isinstance(results, c.Iterable):\n result_list.extend(results)\n else:\n result_list.append(results)\n all_results.extend(result_list)\n\n return all_results", "response": "Calls all child extensions with the given arguments and returns the results of all child extensions."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nprinting a structural view of the registered extensions.", "response": "def print(self, indent=0):\n \"\"\"Print a structural view of the registered extensions.\"\"\"\n LOG.info(\"%s:: %s\", indent * \" \", self.__class__)\n for ext in self.next_extensions:\n ext.print(indent=indent + 2)"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef print_results(matches):\n output = \"\"\n for n, match in enumerate(matches):\n matched_text = match[0][:40]+\"...\" if len(match[0]) > 40 else match[0]\n search_result = match[1]\n if search_result.uri==surf.ns.EFRBROO['F10_Person']:\n label = unicode(search_result)[:40]+\"...\" if len(unicode(search_result)) > 40 else unicode(search_result)\n print(\"\\n{:5}) {:50} {:40} (Matched: \\\"{}\\\")\\n\".format(n+1, label, search_result.get_urn(), matched_text))\n elif search_result.uri==surf.ns.EFRBROO['F1_Work']:\n label = \"{}, {}\".format(search_result.author, search_result)\n label = label[:40]+\"...\" if len(label) > 40 else label\n print(\"\\n{:5}) {:50} {:40} (Matched: \\\"{}\\\")\\n\".format(n+1, label, search_result.get_urn(), matched_text))", "response": "Prints the results of the search."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nprinting the result of a resource.", "response": "def show_result(resource, verbose=False):\n \"\"\"\n TODO\n \"\"\"\n if resource.uri == surf.ns.EFRBROO['F10_Person']:\n print(\"\\n{} ({})\\n\".format(unicode(resource), resource.get_urn()))\n works = resource.get_works()\n print(\"Works by {} ({}):\\n\".format(resource, len(works)))\n [show_result(work) for work in works]\n print(\"\\n\")\n elif resource.uri == surf.ns.EFRBROO['F1_Work']:\n if verbose:\n print(\"\\n{} ({})\".format(unicode(resource), resource.get_urn()))\n print(\"\\nTitles:\")\n print(\"\\n\".join([\"{:20} ({})\".format(title, lang) for lang, title in resource.get_titles()]))\n if len(resource.get_abbreviations()) > 0:\n print(\"\\nAbbreviations: {}\\n\".format(\", \".join([\"{}\".format(abbr) for abbr in resource.get_abbreviations()])))\n else:\n print(\"{:50} {:40}\".format(unicode(resource), resource.get_urn()))"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\ndefining the CLI inteface / commands.", "response": "def main():\n \"\"\"Define the CLI inteface/commands.\"\"\"\n arguments = docopt(__doc__)\n cfg_filename = pkg_resources.resource_filename(\n 'knowledge_base',\n 'config/virtuoso.ini'\n )\n kb = KnowledgeBase(cfg_filename)\n\n # the user has issued a `find` command\n if arguments[\"find\"]:\n search_string = arguments[\"\"]\n try:\n urn = CTS_URN(search_string)\n match = kb.get_resource_by_urn(str(urn))\n show_result(match, verbose=True)\n return\n except BadCtsUrnSyntax as e:\n pass\n except IndexError as e:\n raise e\n print(\"\\nNo records with this CTS URN!\\n\")\n return\n try:\n matches = kb.search(search_string)\n print(\"\\nSearching for \\\"%s\\\" yielded %s results\" % (\n search_string,\n len(matches)\n ))\n print_results(matches)\n return\n except SparqlReaderException as e:\n print(\"\\nWildcard word needs at least 4 leading characters\")\n # the user has issued an `add` command\n elif arguments[\"add\"]:\n input_urn = arguments[\"--to\"]\n\n # first let's check if it's a valid URN\n try:\n urn = CTS_URN(input_urn)\n except Exception as e:\n print(\"The provided URN ({}) is invalid!\".format(input_urn))\n return\n\n try:\n resource = kb.get_resource_by_urn(urn)\n assert resource is not None\n except ResourceNotFound:\n print(\"The KB does not contain a resource identified by {}\".format(\n urn\n ))\n return\n\n\n print(arguments)\n #if arguments[\"\"]\n pass"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\ncheck if a container exists and is unpacked.", "response": "def is_valid(container, path):\n \"\"\"\n Checks if a container exists and is unpacked.\n\n Args:\n path: The location where the container is expected.\n\n Returns:\n True if the container is valid, False if the container needs to\n unpacked or if the path does not exist yet.\n \"\"\"\n try:\n tmp_hash_path = container.filename + \".hash\"\n with open(tmp_hash_path, 'r') as tmp_file:\n tmp_hash = tmp_file.readline()\n except IOError:\n LOG.info(\"No .hash-file in the tmp-directory.\")\n\n container_hash_path = local.path(path) / \"gentoo.tar.bz2.hash\"\n if container_hash_path.exists():\n with open(container_hash_path, 'r') as hash_file:\n container_hash = hash_file.readline()\n return container_hash == tmp_hash\n return False"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef unpack(container, path):\n from benchbuild.utils.run import run\n from benchbuild.utils.uchroot import no_args\n\n path = local.path(path)\n c_filename = local.path(container.filename)\n name = c_filename.basename\n\n if not path.exists():\n path.mkdir()\n\n with local.cwd(path):\n Wget(container.remote, name)\n\n uchroot = no_args()\n uchroot = uchroot[\"-E\", \"-A\", \"-C\", \"-r\", \"/\", \"-w\",\n os.path.abspath(\".\"), \"--\"]\n\n # Check, if we need erlent support for this archive.\n has_erlent = bash[\n \"-c\", \"tar --list -f './{0}' | grep --silent '.erlent'\".format(\n name)]\n has_erlent = (has_erlent & TF)\n\n untar = local[\"/bin/tar\"][\"xf\", \"./\" + name]\n if not has_erlent:\n untar = uchroot[untar]\n\n run(untar[\"--exclude=dev/*\"])\n if not os.path.samefile(name, container.filename):\n rm(name)\n else:\n LOG.warning(\"File contents do not match: %s != %s\", name,\n container.filename)\n cp(container.filename + \".hash\", path)", "response": "Unpack a container usable by uchroot."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef local(self):\n assert self.name in CFG[\"container\"][\"images\"].value\n tmp_dir = local.path(str(CFG[\"tmp_dir\"]))\n target_dir = tmp_dir / self.name\n\n if not target_dir.exists() or not is_valid(self, target_dir):\n unpack(self, target_dir)\n\n return target_dir", "response": "Find the current location of a container."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\ngetting the latest src_uri for a stage 3 tarball.", "response": "def src_file(self):\n \"\"\"\n Get the latest src_uri for a stage 3 tarball.\n\n Returns (str):\n Latest src_uri from gentoo's distfiles mirror.\n \"\"\"\n try:\n src_uri = (curl[Gentoo._LATEST_TXT] | tail[\"-n\", \"+3\"]\n | cut[\"-f1\", \"-d \"])().strip()\n except ProcessExecutionError as proc_ex:\n src_uri = \"NOT-FOUND\"\n LOG.error(\"Could not determine latest stage3 src uri: %s\",\n str(proc_ex))\n return src_uri"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef version(self):\n try:\n _version = (curl[Gentoo._LATEST_TXT] | \\\n awk['NR==2{print}'] | \\\n cut[\"-f2\", \"-d=\"])().strip()\n _version = datetime.utcfromtimestamp(int(_version))\\\n .strftime(\"%Y-%m-%d\")\n except ProcessExecutionError as proc_ex:\n _version = \"unknown\"\n LOG.error(\"Could not determine timestamp: %s\",\n str(proc_ex))\n return _version", "response": "Return the build date of the gentoo container."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\ndetermine the new start and end indices of the time series.", "response": "def time_indices(npts, dt, start, end, index):\n \"\"\"\n Determine the new start and end indices of the time series.\n\n :param npts: Number of points in original time series\n :param dt: Time step of original time series\n :param start: int or float, optional, New start point\n :param end: int or float, optional, New end point\n :param index: bool, optional, if False then start and end are considered values in time.\n :return: tuple, start index, end index\n \"\"\"\n if index is False: # Convert time values into indices\n if end != -1:\n e_index = int(end / dt) + 1\n else:\n e_index = end\n s_index = int(start / dt)\n else:\n s_index = start\n e_index = end\n if e_index > npts:\n raise exceptions.ModelWarning(\"Cut point is greater than time series length\")\n return s_index, e_index"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\ncut the time series to reduce its length.", "response": "def cut(self, start=0, end=-1, index=False):\n \"\"\"\n The method cuts the time series to reduce its length.\n :param start: int or float, optional, New start point\n :param end: int or float, optional, New end point\n :param index: bool, optional, if False then start and end are considered values in time.\n \"\"\"\n s_index, e_index = time_indices(self.npts, self.dt, start, end, index)\n self._values = np.array(self.values[s_index:e_index])"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef finished(cls, jobid):\n output = subprocess.check_output(\n [SACCT, '-n', '-X', '-o', \"end\", '-j', str(jobid)]\n )\n end = output.strip().decode()\n return end not in {'Unknown', ''}", "response": "Check whether a SLURM job is finished or not"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nbeginning a run_group in the database.", "response": "def begin_run_group(project):\n \"\"\"\n Begin a run_group in the database.\n\n A run_group groups a set of runs for a given project. This models a series\n of runs that form a complete binary runtime test.\n\n Args:\n project: The project we begin a new run_group for.\n\n Returns:\n ``(group, session)`` where group is the created group in the\n database and session is the database session this group lives in.\n \"\"\"\n from benchbuild.utils.db import create_run_group\n from datetime import datetime\n\n group, session = create_run_group(project)\n group.begin = datetime.now()\n group.status = 'running'\n\n session.commit()\n return group, session"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nends the run_group successfully.", "response": "def end_run_group(group, session):\n \"\"\"\n End the run_group successfully.\n\n Args:\n group: The run_group we want to complete.\n session: The database transaction we will finish.\n \"\"\"\n from datetime import datetime\n\n group.end = datetime.now()\n group.status = 'completed'\n session.commit()"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nend the run_group unsuccessfully.", "response": "def fail_run_group(group, session):\n \"\"\"\n End the run_group unsuccessfully.\n\n Args:\n group: The run_group we want to complete.\n session: The database transaction we will finish.\n \"\"\"\n from datetime import datetime\n\n group.end = datetime.now()\n group.status = 'failed'\n session.commit()"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef exit_code_from_run_infos(run_infos: t.List[RunInfo]) -> int:\n assert run_infos is not None\n\n if not hasattr(run_infos, \"__iter__\"):\n return run_infos.retcode\n\n rcs = [ri.retcode for ri in run_infos]\n max_rc = max(rcs)\n min_rc = min(rcs)\n if max_rc == 0:\n return min_rc\n return max_rc", "response": "Generate a single exit code from a list of RunInfo objects."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nguard the execution of the given command. The given command (`cmd`) will be executed inside a database context. As soon as you leave the context we will commit the transaction. Any necessary modifications to the database can be identified inside the context with the RunInfo object. Args: cmd: The command we guard. project: The project we track for. experiment: The experiment we track for. Yields: RunInfo: A context object that carries the necessary database transaction.", "response": "def track_execution(cmd, project, experiment, **kwargs):\n \"\"\"Guard the execution of the given command.\n\n The given command (`cmd`) will be executed inside a database context.\n As soon as you leave the context we will commit the transaction.\n Any necessary modifications to the database can be identified inside\n the context with the RunInfo object.\n\n Args:\n cmd: The command we guard.\n project: The project we track for.\n experiment: The experiment we track for.\n\n Yields:\n RunInfo: A context object that carries the necessary\n database transaction.\n \"\"\"\n\n runner = RunInfo(cmd=cmd, project=project, experiment=experiment, **kwargs)\n yield runner\n runner.commit()"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef with_env_recursive(cmd, **envvars):\n from plumbum.commands.base import BoundCommand, BoundEnvCommand\n if isinstance(cmd, BoundCommand):\n cmd.cmd = with_env_recursive(cmd.cmd, **envvars)\n elif isinstance(cmd, BoundEnvCommand):\n cmd.envvars.update(envvars)\n cmd.cmd = with_env_recursive(cmd.cmd, **envvars)\n return cmd", "response": "Recursively updates the environment of the specified command and all its subcommands."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef in_builddir(sub='.'):\n from functools import wraps\n\n def wrap_in_builddir(func):\n \"\"\"Wrap the function for the new build directory.\"\"\"\n\n @wraps(func)\n def wrap_in_builddir_func(self, *args, **kwargs):\n \"\"\"The actual function inside the wrapper for the new builddir.\"\"\"\n p = local.path(self.builddir) / sub\n if not p.exists():\n LOG.error(\"%s does not exist.\", p)\n\n if p == local.cwd:\n LOG.debug(\"CWD already is %s\", p)\n return func(self, *args, *kwargs)\n with local.cwd(p):\n return func(self, *args, **kwargs)\n\n return wrap_in_builddir_func\n\n return wrap_in_builddir", "response": "Decorator for creating a new build directory in a project phase with a local working directory change."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef store_config(func):\n from functools import wraps\n\n @wraps(func)\n def wrap_store_config(self, *args, **kwargs):\n \"\"\"Wrapper that contains the actual storage call for the config.\"\"\"\n CFG.store(local.path(self.builddir) / \".benchbuild.yml\")\n return func(self, *args, **kwargs)\n\n return wrap_store_config", "response": "Decorator for storing the configuration in the project s builddir."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef clean_directories(builddir, in_dir=True, out_dir=True):\n container_in = local.path(builddir) / \"container-in\"\n container_out = local.path(builddir) / \"container-out\"\n\n if in_dir and container_in.exists():\n if ui.ask(\"Should I delete '{0}'?\".format(container_in)):\n container_in.delete()\n if out_dir and container_out.exists():\n if ui.ask(\"Should I delete '{0}'?\".format(container_out)):\n container_out.delete()", "response": "Remove the in and out of the container if confirmed by the user."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\ncreates the in and out directories of the container.", "response": "def setup_directories(builddir):\n \"\"\"Create the in and out directories of the container.\"\"\"\n build_dir = local.path(builddir)\n in_dir = build_dir / \"container-in\"\n out_dir = build_dir / \"container-out\"\n\n if not in_dir.exists():\n in_dir.mkdir()\n if not out_dir.exists():\n out_dir.mkdir()"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef setup_container(builddir, _container):\n build_dir = local.path(builddir)\n in_dir = build_dir / \"container-in\"\n container_path = local.path(_container)\n with local.cwd(builddir):\n container_bin = container_path.basename\n container_in = in_dir / container_bin\n download.Copy(_container, container_in)\n uchrt = uchroot.no_args()\n\n with local.cwd(\"container-in\"):\n uchrt = uchrt[\"-E\", \"-A\", \"-u\", \"0\", \"-g\", \"0\", \"-C\", \"-r\", \"/\",\n \"-w\",\n os.path.abspath(\".\"), \"--\"]\n\n # Check, if we need erlent support for this archive.\n has_erlent = bash[\n \"-c\", \"tar --list -f './{0}' | grep --silent '.erlent'\".format(\n container_in)]\n has_erlent = (has_erlent & TF)\n\n # Unpack input container to: container-in\n if not has_erlent:\n cmd = local[\"/bin/tar\"][\"xf\"]\n cmd = uchrt[cmd[container_bin]]\n else:\n cmd = tar[\"xf\"]\n cmd = cmd[container_in]\n\n with local.cwd(\"container-in\"):\n cmd(\"--exclude=dev/*\")\n rm(container_in)\n return in_dir", "response": "Prepare the container and returns the path where it can be found."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef run_in_container(command, container_dir):\n container_p = local.path(container_dir)\n with local.cwd(container_p):\n uchrt = uchroot.with_mounts()\n uchrt = uchrt[\"-E\", \"-A\", \"-u\", \"0\", \"-g\", \"0\", \"-C\", \"-w\", \"/\", \"-r\",\n container_p]\n uchrt = uchrt[\"--\"]\n\n cmd_path = container_p / command[0].lstrip('/')\n if not cmd_path.exists():\n LOG.error(\"The command does not exist inside the container! %s\",\n cmd_path)\n return\n\n cmd = uchrt[command]\n return cmd & FG", "response": "Runs a given command inside a container."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\npacking a container image into a. tar. bz2 archive.", "response": "def pack_container(in_container, out_file):\n \"\"\"\n Pack a container image into a .tar.bz2 archive.\n\n Args:\n in_container (str): Path string to the container image.\n out_file (str): Output file name.\n \"\"\"\n container_filename = local.path(out_file).basename\n out_container = local.cwd / \"container-out\" / container_filename\n out_dir = out_container.dirname\n\n # Pack the results to: container-out\n with local.cwd(in_container):\n tar(\"cjf\", out_container, \".\")\n c_hash = download.update_hash(out_container)\n if out_dir.exists():\n mkdir(\"-p\", out_dir)\n mv(out_container, out_file)\n mv(out_container + \".hash\", out_file + \".hash\")\n\n new_container = {\"path\": out_file, \"hash\": str(c_hash)}\n CFG[\"container\"][\"known\"] += new_container"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef setup_bash_in_container(builddir, _container, outfile, shell):\n with local.cwd(builddir):\n # Switch to bash inside uchroot\n print(\"Entering bash inside User-Chroot. Prepare your image and \"\n \"type 'exit' when you are done. If bash exits with a non-zero\"\n \"exit code, no new container will be stored.\")\n store_new_container = True\n try:\n run_in_container(shell, _container)\n except ProcessExecutionError:\n store_new_container = False\n\n if store_new_container:\n print(\"Packing new container image.\")\n pack_container(_container, outfile)\n config_path = str(CFG[\"config_file\"])\n CFG.store(config_path)\n print(\"Storing config in {0}\".format(os.path.abspath(config_path)))", "response": "Setup a bash environment inside a container."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nsave the input for the container in the configurations.", "response": "def set_input_container(_container, cfg):\n \"\"\"Save the input for the container in the configurations.\"\"\"\n if not _container:\n return False\n if _container.exists():\n cfg[\"container\"][\"input\"] = str(_container)\n return True\n return False"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef run(self, context):\n # Don't do something when running non-interactive.\n if not sys.stdout.isatty():\n return\n\n with local.cwd(context.in_container):\n from benchbuild.projects.gentoo import gentoo\n gentoo.setup_networking()\n gentoo.configure_portage()\n\n sed_in_chroot = uchroot.uchroot()[\"/bin/sed\"]\n emerge_in_chroot = uchroot.uchroot()[\"/usr/bin/emerge\"]\n has_pkg = uchroot.uchroot()[\"/usr/bin/qlist\", \"-I\"]\n\n run.run(sed_in_chroot[\"-i\", '/CC=/d', \"/etc/portage/make.conf\"])\n run.run(sed_in_chroot[\"-i\", '/CXX=/d', \"/etc/portage/make.conf\"])\n\n want_sync = bool(CFG[\"container\"][\"strategy\"][\"polyjit\"][\"sync\"])\n want_upgrade = bool(\n CFG[\"container\"][\"strategy\"][\"polyjit\"][\"upgrade\"])\n\n packages = \\\n CFG[\"container\"][\"strategy\"][\"polyjit\"][\"packages\"].value\n with local.env(MAKEOPTS=\"-j{0}\".format(int(CFG[\"jobs\"]))):\n if want_sync:\n LOG.debug(\"Synchronizing portage.\")\n run.run(emerge_in_chroot[\"--sync\"])\n if want_upgrade:\n LOG.debug(\"Upgrading world.\")\n run.run(emerge_in_chroot[\"--autounmask-only=y\", \"-uUDN\",\n \"--with-bdeps=y\", \"@world\"])\n for pkg in packages:\n if has_pkg[pkg[\"name\"]] & TF:\n continue\n env = pkg[\"env\"]\n with local.env(**env):\n run.run(emerge_in_chroot[pkg[\"name\"]])\n\n gentoo.setup_benchbuild()\n\n print(\"Packing new container image.\")\n with local.cwd(context.builddir):\n pack_container(context.in_container, context.out_container)", "response": "Setup a gentoo container suitable for PolyJIT."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef input_file(self, _container):\n p = local.path(_container)\n if set_input_container(p, CFG):\n return\n\n p = find_hash(CFG[\"container\"][\"known\"].value, container)\n if set_input_container(p, CFG):\n return\n\n raise ValueError(\"The path '{0}' does not exist.\".format(p))", "response": "Find the input path of a uchroot container."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef output_file(self, _container):\n p = local.path(_container)\n if p.exists():\n if not ui.ask(\"Path '{0}' already exists.\"\n \" Overwrite?\".format(p)):\n sys.exit(0)\n CFG[\"container\"][\"output\"] = str(p)", "response": "Find and writes the output path of a chroot container."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef discretize_soil_profile(sp, incs=None, target=1.0):\n\n if incs is None:\n incs = np.ones(sp.n_layers) * target\n dd = {}\n dd[\"thickness\"] = []\n dd[\"unit_mass\"] = []\n dd[\"shear_vel\"] = []\n cum_thickness = 0\n for i in range(sp.n_layers):\n sl = sp.layer(i + 1)\n thickness = sp.layer_height(i + 1)\n n_slices = max(int(thickness / incs[i]), 1)\n slice_thickness = float(thickness) / n_slices\n for j in range(n_slices):\n cum_thickness += slice_thickness\n if cum_thickness >= sp.gwl:\n rho = sl.unit_sat_mass\n saturation = True\n else:\n rho = sl.unit_dry_mass\n saturation = False\n if hasattr(sl, \"get_shear_vel_at_v_eff_stress\"):\n v_eff = sp.vertical_effective_stress(cum_thickness)\n vs = sl.get_shear_vel_at_v_eff_stress(v_eff, saturation)\n else:\n vs = sl.calc_shear_vel(saturation)\n dd[\"shear_vel\"].append(vs)\n dd[\"unit_mass\"].append(rho)\n dd[\"thickness\"].append(slice_thickness)\n for item in dd:\n dd[item] = np.array(dd[item])\n return dd", "response": "Discretizes the soil profile into slices and stores as dictionary\n "} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef reset_all(self):\n for item in self.inputs:\n setattr(self, \"_%s\" % item, None)\n self.stack = []", "response": "Resets all parameters to None"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nadd a parameter - value pair to the stack of parameters that have been set.", "response": "def _add_to_stack(self, item, value):\n \"\"\"\n Add a parameter-value pair to the stack of parameters that have been set.\n :param item:\n :param value:\n :return:\n \"\"\"\n p_value = (item, value)\n if p_value not in self.stack:\n self.stack.append(p_value)"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\ncalculates the shear wave velocity of the user.", "response": "def get_shear_vel(self, saturated):\n \"\"\"\n Calculate the shear wave velocity\n\n :param saturated: bool, if true then use saturated mass\n :return:\n \"\"\"\n try:\n if saturated:\n return np.sqrt(self.g_mod / self.unit_sat_mass)\n else:\n return np.sqrt(self.g_mod / self.unit_dry_mass)\n except TypeError:\n return None"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nvoluming of water to volume of voids", "response": "def saturation(self, value):\n \"\"\"Volume of water to volume of voids\"\"\"\n value = clean_float(value)\n if value is None:\n return\n try:\n unit_moisture_weight = self.unit_moist_weight - self.unit_dry_weight\n unit_moisture_volume = unit_moisture_weight / self._pw\n saturation = unit_moisture_volume / self._calc_unit_void_volume()\n if saturation is not None and not ct.isclose(saturation, value, rel_tol=self._tolerance):\n raise ModelError(\"New saturation (%.3f) is inconsistent \"\n \"with calculated value (%.3f)\" % (value, saturation))\n except TypeError:\n pass\n old_value = self.saturation\n self._saturation = value\n try:\n self.recompute_all_weights_and_void()\n self._add_to_stack(\"saturation\", value)\n except ModelError as e:\n self._saturation = old_value\n raise ModelError(e)"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef specific_gravity(self, value):\n value = clean_float(value)\n if value is None:\n return\n specific_gravity = self._calc_specific_gravity()\n if specific_gravity is not None and not ct.isclose(specific_gravity, value, rel_tol=self._tolerance):\n raise ModelError(\"specific gravity is inconsistent with set unit_dry_weight and void_ratio\")\n\n self._specific_gravity = float(value)\n self.stack.append((\"specific_gravity\", float(value)))\n self.recompute_all_weights_and_void()", "response": "Set the relative weight of the solid"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef add_layer(self, depth, soil):\n\n self._layers[depth] = soil\n self._sort_layers()\n if self.hydrostatic:\n if depth >= self.gwl:\n soil.saturation = 1.0\n else:\n li = self.get_layer_index_by_depth(depth)\n layer_height = self.layer_height(li)\n if layer_height is None:\n soil.saturation = 0.0\n elif depth + layer_height <= self.gwl:\n soil.saturation = 0.0\n else:\n sat_height = depth + self.layer_height(li) - self.gwl\n soil.saturation = sat_height / self.layer_height(li)", "response": "Adds a soil to the SoilProfile at a set depth."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nsorting the layers by depth.", "response": "def _sort_layers(self):\n \"\"\"Sort the layers by depth.\"\"\"\n self._layers = OrderedDict(sorted(self._layers.items(), key=lambda t: t[0]))"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef layer_height(self, layer_int):\n if layer_int == self.n_layers:\n if self.height is None:\n return None\n return self.height - self.layer_depth(layer_int)\n else:\n return self.layer_depth(layer_int + 1) - self.layer_depth(layer_int)", "response": "Get the layer height by layer id number."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\ncalculate the equivalent crust cohesion strength according to Karamitros et al. 2013 sett pg 8 eq. 14", "response": "def equivalent_crust_cohesion(self):\n \"\"\"\n Calculate the equivalent crust cohesion strength according to Karamitros et al. 2013 sett, pg 8 eq. 14\n\n :return: equivalent cohesion [Pa]\n \"\"\"\n deprecation(\"Will be moved to a function\")\n if len(self.layers) > 1:\n crust = self.layer(0)\n crust_phi_r = np.radians(crust.phi)\n equivalent_cohesion = crust.cohesion + crust.k_0 * self.crust_effective_unit_weight * \\\n self.layer_depth(1) / 2 * np.tan(crust_phi_r)\n return equivalent_cohesion"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nreturning the vertical total stress at depth z.", "response": "def get_v_total_stress_at_depth(self, z):\n \"\"\"\n Determine the vertical total stress at depth z, where z can be a number or an array of numbers.\n \"\"\"\n\n if not hasattr(z, \"__len__\"):\n return self.one_vertical_total_stress(z)\n else:\n sigma_v_effs = []\n for value in z:\n sigma_v_effs.append(self.one_vertical_total_stress(value))\n return np.array(sigma_v_effs)"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef one_vertical_total_stress(self, z_c):\n total_stress = 0.0\n depths = self.depths\n end = 0\n for layer_int in range(1, len(depths) + 1):\n l_index = layer_int - 1\n if z_c > depths[layer_int - 1]:\n if l_index < len(depths) - 1 and z_c > depths[l_index + 1]:\n height = depths[l_index + 1] - depths[l_index]\n bottom_depth = depths[l_index + 1]\n else:\n end = 1\n height = z_c - depths[l_index]\n bottom_depth = z_c\n\n if bottom_depth <= self.gwl:\n total_stress += height * self.layer(layer_int).unit_dry_weight\n else:\n if self.layer(layer_int).unit_sat_weight is None:\n raise AnalysisError(\"Saturated unit weight not defined for layer %i.\" % layer_int)\n sat_height = bottom_depth - max(self.gwl, depths[l_index])\n dry_height = height - sat_height\n total_stress += dry_height * self.layer(layer_int).unit_dry_weight + \\\n sat_height * self.layer(layer_int).unit_sat_weight\n else:\n end = 1\n if end:\n break\n return total_stress", "response": "Determines the vertical total stress at a single depth z_c."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef get_v_eff_stress_at_depth(self, y_c):\n sigma_v_c = self.get_v_total_stress_at_depth(y_c)\n pp = self.get_hydrostatic_pressure_at_depth(y_c)\n sigma_veff_c = sigma_v_c - pp\n return sigma_veff_c", "response": "Determine the vertical effective stress at a single depth z_c."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nget the shear wave velocity at a depth.", "response": "def shear_vel_at_depth(self, y_c):\n \"\"\"\n Get the shear wave velocity at a depth.\n\n :param y_c: float, depth from surface\n :return:\n \"\"\"\n sl = self.get_soil_at_depth(y_c)\n if y_c <= self.gwl:\n saturation = False\n else:\n saturation = True\n if hasattr(sl, \"get_shear_vel_at_v_eff_stress\"):\n v_eff = self.get_v_eff_stress_at_depth(y_c)\n vs = sl.get_shear_vel_at_v_eff_stress(v_eff, saturation)\n else:\n vs = sl.get_shear_vel(saturation)\n return vs"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef create_package_file(root, master_package, subroot, py_files, opts, subs):\n package = os.path.split(root)[-1]\n text = format_heading(1, '%s Package' % package)\n # add each package's module\n for py_file in py_files:\n if shall_skip(os.path.join(root, py_file)):\n continue\n is_package = py_file == INIT\n py_file = os.path.splitext(py_file)[0]\n py_path = makename(subroot, py_file)\n if is_package:\n heading = ':mod:`%s` Package' % package\n else:\n heading = ':mod:`%s` Module' % py_file\n text += format_heading(2, heading)\n text += format_directive(is_package and subroot or py_path, master_package)\n text += '\\n'\n\n # build a list of directories that are packages (they contain an INIT file)\n subs = [sub for sub in subs if os.path.isfile(os.path.join(root, sub, INIT))]\n # if there are some package directories, add a TOC for theses subpackages\n if subs:\n text += format_heading(2, 'Subpackages')\n text += '.. toctree::\\n\\n'\n for sub in subs:\n text += ' %s.%s\\n' % (makename(master_package, subroot), sub)\n text += '\\n'\n\n write_file(makename(master_package, subroot), text, opts)", "response": "Create the text of the file and write it."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef create_modules_toc_file(master_package, modules, opts, name='modules'):\n text = format_heading(1, '%s Modules' % opts.header)\n text += '.. toctree::\\n'\n text += ' :maxdepth: %s\\n\\n' % opts.maxdepth\n\n modules.sort()\n prev_module = ''\n for module in modules:\n # look if the module is a subpackage and, if yes, ignore it\n if module.startswith(prev_module + '.'):\n continue\n prev_module = module\n text += ' %s\\n' % module\n\n write_file(name, text, opts)", "response": "Create the modules toc file."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nlooking for every file in the directory tree and create the corresponding ReST files.", "response": "def recurse_tree(path, excludes, opts):\n \"\"\"\n Look for every file in the directory tree and create the corresponding\n ReST files.\n \"\"\"\n # use absolute path for root, as relative paths like '../../foo' cause\n # 'if \"/.\" in root ...' to filter out *all* modules otherwise\n path = os.path.abspath(path)\n # check if the base directory is a package and get is name\n if INIT in os.listdir(path):\n package_name = path.split(os.path.sep)[-1]\n else:\n package_name = None\n\n toc = []\n tree = os.walk(path, False)\n for root, subs, files in tree:\n # keep only the Python script files\n py_files = sorted([f for f in files if os.path.splitext(f)[1] == '.py'])\n if INIT in py_files:\n py_files.remove(INIT)\n py_files.insert(0, INIT)\n # remove hidden ('.') and private ('_') directories\n subs = sorted([sub for sub in subs if sub[0] not in ['.', '_']])\n # check if there are valid files to process\n # TODO: could add check for windows hidden files\n if \"/.\" in root or \"/_\" in root \\\n or not py_files \\\n or is_excluded(root, excludes):\n continue\n if INIT in py_files:\n # we are in package ...\n if (# ... with subpackage(s)\n subs\n or\n # ... with some module(s)\n len(py_files) > 1\n or\n # ... with a not-to-be-skipped INIT file\n not shall_skip(os.path.join(root, INIT))\n ):\n subroot = root[len(path):].lstrip(os.path.sep).replace(os.path.sep, '.')\n create_package_file(root, package_name, subroot, py_files, opts, subs)\n toc.append(makename(package_name, subroot))\n elif root == path:\n # if we are at the root level, we don't require it to be a package\n for py_file in py_files:\n if not shall_skip(os.path.join(path, py_file)):\n module = os.path.splitext(py_file)[0]\n create_module_file(package_name, module, opts)\n toc.append(makename(package_name, module))\n\n # create the module's index\n if not opts.notoc:\n create_modules_toc_file(package_name, toc, opts)"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef normalize_excludes(rootpath, excludes):\n sep = os.path.sep\n f_excludes = []\n for exclude in excludes:\n if not os.path.isabs(exclude) and not exclude.startswith(rootpath):\n exclude = os.path.join(rootpath, exclude)\n if not exclude.endswith(sep):\n exclude += sep\n f_excludes.append(exclude)\n return f_excludes", "response": "Normalize the excluded directory list"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef is_excluded(root, excludes):\n sep = os.path.sep\n if not root.endswith(sep):\n root += sep\n for exclude in excludes:\n if root.startswith(exclude):\n return True\n return False", "response": "Check if the directory is in the exclude list."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef main():\n parser = optparse.OptionParser(usage=\"\"\"usage: %prog [options] [exclude paths, ...]\n\nNote: By default this script will not overwrite already created files.\"\"\")\n parser.add_option(\"-n\", \"--doc-header\", action=\"store\", dest=\"header\", help=\"Documentation Header (default=Project)\", default=\"Project\")\n parser.add_option(\"-d\", \"--dest-dir\", action=\"store\", dest=\"destdir\", help=\"Output destination directory\", default=\"\")\n parser.add_option(\"-s\", \"--suffix\", action=\"store\", dest=\"suffix\", help=\"module suffix (default=txt)\", default=\"txt\")\n parser.add_option(\"-m\", \"--maxdepth\", action=\"store\", dest=\"maxdepth\", help=\"Maximum depth of submodules to show in the TOC (default=4)\", type=\"int\", default=4)\n parser.add_option(\"-r\", \"--dry-run\", action=\"store_true\", dest=\"dryrun\", help=\"Run the script without creating the files\")\n parser.add_option(\"-f\", \"--force\", action=\"store_true\", dest=\"force\", help=\"Overwrite all the files\")\n parser.add_option(\"-t\", \"--no-toc\", action=\"store_true\", dest=\"notoc\", help=\"Don't create the table of content file\")\n (opts, args) = parser.parse_args()\n if not args:\n parser.error(\"package path is required.\")\n else:\n rootpath, excludes = args[0], args[1:]\n if os.path.isdir(rootpath):\n # check if the output destination is a valid directory\n if opts.destdir and os.path.isdir(opts.destdir):\n excludes = normalize_excludes(rootpath, excludes)\n recurse_tree(rootpath, excludes, opts)\n else:\n print '%s is not a valid output destination directory.' % opts.destdir\n else:\n print '%s is not a valid directory.' % rootpath", "response": "Parse and check the command line arguments and run the main function."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef path(self):\n\n if not isinstance(self.ref, str):\n return None\n\n u = parse_app_url(self.ref)\n\n if u.inner.proto != 'file':\n return None\n\n return u.path", "response": "Return the path to the file"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef doc_dir(self):\n from os.path import abspath\n\n if not self.ref:\n return None\n\n u = parse_app_url(self.ref)\n return abspath(dirname(u.path))", "response": "The absolute directory of the document"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef new_section(self, name, params=None):\n self.sections[name.lower()] = SectionTerm(None, name, term_args=params, doc=self)\n\n # Set the default arguments\n s = self.sections[name.lower()]\n\n if name.lower() in self.decl_sections:\n s.args = self.decl_sections[name.lower()]['args']\n\n return s", "response": "Return a new section"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef get_or_new_section(self, name, params=None):\n if name not in self.sections:\n self.sections[name.lower()] = SectionTerm(None, name, term_args=params, doc=self)\n\n return self.sections[name.lower()]", "response": "Create a new section or return an existing one of the same name"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nsort the sections according to the section names in the order list.", "response": "def sort_sections(self, order):\n \"\"\"\n Sort sections according to the section names in the order list. All remaining sections\n are added to the end in their original order\n\n :param order: Iterable of section names\n :return:\n \"\"\"\n\n order_lc = [e.lower() for e in order]\n\n sections = OrderedDict( (k,self.sections[k]) for k in order_lc if k in self.sections)\n\n sections.update( (k,self.sections[k]) for k in self.sections.keys() if k not in order_lc)\n\n assert len(self.sections) == len(sections)\n\n self.sections = sections"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nreturns a list of terms that are in a particular section.", "response": "def find(self, term, value=False, section=None, _expand_derived=True, **kwargs):\n \"\"\"Return a list of terms, possibly in a particular section. Use joined term notation, such as 'Root.Name' The kwargs arg is used to set term properties, all of which match returned terms, so ``name='foobar'`` will match terms that have a ``name`` property of ``foobar``\n\n :param term: The type of term to find, in fully-qulified notation, or use '*' for wild cards in either the parent or the record parts, such as 'Root.*', '*.Table' or '*.*'\n :param value: Select terms with a given value\n :param section: The name of the section in which to restrict the search\n :param kwargs: See additional properties on which to match terms.\n\n \"\"\"\n\n import itertools\n\n if kwargs: # Look for terms with particular property values\n\n terms = self.find(term, value, section)\n\n found_terms = []\n\n for t in terms:\n if all(t.get_value(k) == v for k, v in kwargs.items()):\n found_terms.append(t)\n\n return found_terms\n\n def in_section(term, section):\n\n if section is None:\n return True\n\n if term.section is None:\n return False\n\n if isinstance(section, (list, tuple)):\n return any(in_section(t, e) for e in section)\n else:\n return section.lower() == term.section.name.lower()\n\n # Try to replace the term with the list of its derived terms; that is, replace the super-class with all\n # of the derived classes, but only do this expansion once.\n if _expand_derived:\n try:\n try:\n # Term is a string\n term = list(self.derived_terms[term.lower()]) + [term]\n except AttributeError: # Term is hopefully a list\n terms = []\n for t in term:\n terms.append(term)\n for dt in self.derived_terms[t.lower()]:\n terms.append(dt)\n except KeyError as e:\n pass\n\n # Find any of a list of terms\n if isinstance(term, (list, tuple)):\n return list(itertools.chain(*[self.find(e, value=value, section=section, _expand_derived=False) for e in term]))\n\n else:\n\n term = term.lower()\n\n found = []\n\n if not '.' in term:\n term = 'root.' + term\n\n if term.startswith('root.'):\n term_gen = self.terms # Just the root level terms\n else:\n term_gen = self.all_terms # All terms, root level and children.\n\n for t in term_gen:\n\n if t.join_lc == 'root.root':\n continue\n\n assert t.section or t.join_lc == 'root.root' or t.join_lc == 'root.section', t\n\n if (t.term_is(term)\n and in_section(t, section)\n and (value is False or value == t.value)):\n found.append(t)\n\n return found"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nreturning the first term in the list returning the default if no term is found", "response": "def get(self, term, default=None):\n \"\"\"Return the first term, returning the default if no term is found\"\"\"\n v = self.find_first(term)\n\n if not v:\n return default\n else:\n return v"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef get_value(self, term, default=None, section=None):\n term = self.find_first(term, value=False, section=section)\n\n if term is None:\n return default\n else:\n return term.value", "response": "Return the first value of a term in the specified section."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef load_terms(self, terms):\n\n #if self.root and len(self.root.children) > 0:\n # raise MetatabError(\"Can't run after adding terms to document.\")\n\n for t in terms:\n\n t.doc = self\n\n if t.term_is('root.root'):\n if not self.root:\n self.root = t\n self.add_section(t)\n\n continue\n\n if t.term_is('root.section'):\n self.add_section(t)\n\n elif t.parent_term_lc == 'root':\n self.add_term(t)\n\n else:\n # These terms aren't added to the doc because they are attached to a\n # parent term that is added to the doc.\n assert t.parent is not None\n\n\n try:\n dd = terms.declare_dict\n\n self.decl_terms.update(dd['terms'])\n self.decl_sections.update(dd['sections'])\n\n self.super_terms.update(terms.super_terms())\n\n kf = lambda e: e[1] # Sort on the value\n self.derived_terms ={ k:set( e[0] for e in g)\n for k, g in groupby(sorted(self.super_terms.items(), key=kf), kf)}\n\n except AttributeError as e:\n pass\n\n try:\n self.errors = terms.errors_as_dict()\n except AttributeError:\n self.errors = {}\n\n return self", "response": "Create a builder from a sequence of terms."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\ncleaning up some terms like ensuring that the name is a slug", "response": "def cleanse(self):\n \"\"\"Clean up some terms, like ensuring that the name is a slug\"\"\"\n from .util import slugify\n\n self.ensure_identifier()\n\n try:\n self.update_name()\n except MetatabError:\n\n identifier = self['Root'].find_first('Root.Identifier')\n\n name = self['Root'].find_first('Root.Name')\n\n if name and name.value:\n name.value = slugify(name.value)\n elif name:\n name.value = slugify(identifier.value)\n else:\n self['Root'].get_or_new_term('Root.Name').value = slugify(identifier.value)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\ngenerating the Root. Name term from DatasetName Version Origin TIme and Space and Create the Term Root. Name and Update the Name term.", "response": "def update_name(self, force=False, create_term=False, report_unchanged=True):\n \"\"\"Generate the Root.Name term from DatasetName, Version, Origin, TIme and Space\"\"\"\n\n updates = []\n\n self.ensure_identifier()\n\n name_term = self.find_first('Root.Name')\n\n if not name_term:\n if create_term:\n name_term = self['Root'].new_term('Root.Name','')\n else:\n updates.append(\"No Root.Name, can't update name\")\n return updates\n\n orig_name = name_term.value\n\n identifier = self.get_value('Root.Identifier')\n\n datasetname = self.get_value('Root.Dataset')\n\n if datasetname:\n\n name = self._generate_identity_name()\n\n if name != orig_name or force:\n name_term.value = name\n updates.append(\"Changed Name\")\n else:\n if report_unchanged:\n updates.append(\"Name did not change\")\n\n elif not orig_name:\n\n if not identifier:\n updates.append(\"Failed to find DatasetName term or Identity term. Giving up\")\n\n else:\n updates.append(\"Setting the name to the identifier\")\n name_term.value = identifier\n\n elif orig_name == identifier:\n if report_unchanged:\n updates.append(\"Name did not change\")\n\n else:\n # There is no DatasetName, so we can't gneerate name, and the Root.Name is not empty, so we should\n # not set it to the identity.\n updates.append(\"No Root.Dataset, so can't update the name\")\n\n return updates"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef as_dict(self, replace_value_names=True):\n\n # This function is a hack, due to confusion between the root of the document, which\n # should contain all terms, and the root section, which has only terms that are not\n # in another section. So, here we are taking the Root section, and adding all of the other\n # terms to it, as if it were also the root of the document tree.\n\n r = RootSectionTerm(doc=self)\n\n for s in self: # Iterate over sections\n for t in s: # Iterate over the terms in each section.\n r.terms.append(t)\n\n return r.as_dict(replace_value_names)", "response": "Iterate link terms and convert to a dict"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef rows(self):\n\n\n for s_name, s in self.sections.items():\n\n # Yield the section header\n if s.name != 'Root':\n yield [''] # Unecessary, but makes for nice formatting. Should actually be done just before write\n yield ['Section', s.value] + s.property_names\n\n # Yield all of the rows for terms in the section\n for row in s.rows:\n term, value = row\n\n term = term.replace('root.', '').title()\n\n try:\n yield [term] + value\n except:\n yield [term] + [value]", "response": "Iterate over all of the rows in the archive"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\niterate over all of the terms in the hierarchy.", "response": "def all_terms(self):\n \"\"\"Iterate over all of the terms. The self.terms property has only root level terms. This iterator\n iterates over all terms\"\"\"\n\n for s_name, s in self.sections.items():\n\n # Yield the section header\n if s.name != 'Root':\n yield s\n\n # Yield all of the rows for terms in the section\n for rterm in s:\n yield rterm\n for d in rterm.descendents:\n yield d"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nreturns a CSV representation as a string", "response": "def as_csv(self):\n \"\"\"Return a CSV representation as a string\"\"\"\n\n from io import StringIO\n\n s = StringIO()\n w = csv.writer(s)\n for row in self.rows:\n w.writerow(row)\n\n return s.getvalue()"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef as_lines(self):\n\n out_lines = []\n for t,v in self.lines:\n\n # Make the output prettier\n if t == 'Section':\n out_lines.append('')\n\n out_lines.append('{}: {}'.format(t,v if v is not None else '') )\n\n return '\\n'.join(out_lines)", "response": "Return a Lines representation as a string"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nreturn a factory for creating QWidgetComponents.", "response": "def QWidgetComponentFactory(ui_file=None, *args, **kwargs):\n \"\"\"\n Defines a class factory creating :class:`QWidgetComponent` classes using given ui file.\n\n :param ui_file: Ui file.\n :type ui_file: unicode\n :param \\*args: Arguments.\n :type \\*args: \\*\n :param \\*\\*kwargs: Keywords arguments.\n :type \\*\\*kwargs: \\*\\*\n :return: QWidgetComponent class.\n :rtype: QWidgetComponent\n \"\"\"\n\n class QWidgetComponent(foundations.ui.common.QWidget_factory(ui_file=ui_file)):\n\n \"\"\"\n Defines the base class for **Manager** package QWidget Components.\n \"\"\"\n\n component_activated = pyqtSignal()\n \"\"\"\n This signal is emited by the :class:`QObjectComponent` class when the Component is activated.\n \"\"\"\n\n component_deactivated = pyqtSignal()\n \"\"\"\n This signal is emited by the :class:`QObjectComponent` class when the Component is deactivated.\n \"\"\"\n\n component_initialized_ui = pyqtSignal()\n \"\"\"\n This signal is emited by the :class:`QObjectComponent` class when the Component ui is initialized.\n \"\"\"\n\n component_uninitialized_ui = pyqtSignal()\n \"\"\"\n This signal is emited by the :class:`QObjectComponent` class when the Component ui is uninitialized.\n \"\"\"\n\n def __init__(self, parent=None, name=None, *args, **kwargs):\n \"\"\"\n Initializes the class.\n\n :param parent: Object parent.\n :type parent: QObject\n :param name: Component name.\n :type name: unicode\n :param \\*args: Arguments.\n :type \\*args: \\*\n :param \\*\\*kwargs: Keywords arguments.\n :type \\*\\*kwargs: \\*\\*\n \"\"\"\n\n LOGGER.debug(\"> Initializing '{0}()' class.\".format(self.__class__.__name__))\n\n super(QWidgetComponent, self).__init__(parent, *args, **kwargs)\n\n # --- Setting class attributes. ---\n self.__name = None\n self.name = name\n\n self.__activated = False\n self.__initialized_ui = False\n self.__deactivatable = True\n\n @property\n def name(self):\n \"\"\"\n Property for **self.__name** attribute.\n\n :return: self.__name.\n :rtype: unicode\n \"\"\"\n\n return self.__name\n\n @name.setter\n @foundations.exceptions.handle_exceptions(AssertionError)\n def name(self, value):\n \"\"\"\n Setter for **self.__name** attribute.\n\n :param value: Attribute value.\n :type value: unicode\n \"\"\"\n\n if value is not None:\n assert type(value) is unicode, \"'{0}' attribute: '{1}' type is not 'unicode'!\".format(\n \"name\", value)\n self.__name = value\n\n @name.deleter\n @foundations.exceptions.handle_exceptions(foundations.exceptions.ProgrammingError)\n def name(self):\n \"\"\"\n Deleter for **self.__name** attribute.\n \"\"\"\n\n raise foundations.exceptions.ProgrammingError(\"{0} | '{1}' attribute is not deletable!\".format(\n self.__class__.__name__, \"name\"))\n\n @property\n def activated(self):\n \"\"\"\n Property for **self.__activated** attribute.\n\n :return: self.__activated.\n :rtype: unicode\n \"\"\"\n\n return self.__activated\n\n @activated.setter\n @foundations.exceptions.handle_exceptions(AssertionError)\n def activated(self, value):\n \"\"\"\n Setter for **self.__activated** attribute.\n\n :param value: Attribute value.\n :type value: unicode\n \"\"\"\n\n if value is not None:\n assert type(value) is bool, \"'{0}' attribute: '{1}' type is not 'bool'!\".format(\"activated\", value)\n self.component_activated.emit() if value else self.component_deactivated.emit()\n self.__activated = value\n\n @activated.deleter\n @foundations.exceptions.handle_exceptions(foundations.exceptions.ProgrammingError)\n def activated(self):\n \"\"\"\n Deleter for **self.__activated** attribute.\n \"\"\"\n\n raise foundations.exceptions.ProgrammingError(\"{0} | '{1}' attribute is not deletable!\".format(\n self.__class__.__name__, \"activated\"))\n\n @property\n def initialized_ui(self):\n \"\"\"\n Property for **self.__initialized_ui** attribute.\n\n :return: self.__initialized_ui.\n :rtype: bool\n \"\"\"\n\n return self.__initialized_ui\n\n @initialized_ui.setter\n @foundations.exceptions.handle_exceptions(AssertionError)\n def initialized_ui(self, value):\n \"\"\"\n Setter for **self.__initialized_ui** attribute.\n\n :param value: Attribute value.\n :type value: bool\n \"\"\"\n\n if value is not None:\n assert type(value) is bool, \"'{0}' attribute: '{1}' type is not 'bool'!\".format(\n \"initialized_ui\", value)\n self.component_initialized_ui.emit() if value else self.component_uninitialized_ui.emit()\n self.__initialized_ui = value\n\n @initialized_ui.deleter\n @foundations.exceptions.handle_exceptions(foundations.exceptions.ProgrammingError)\n def initialized_ui(self):\n \"\"\"\n Deleter for **self.__initialized_ui** attribute.\n \"\"\"\n\n raise foundations.exceptions.ProgrammingError(\n \"{0} | '{1}' attribute is not deletable!\".format(self.__class__.__name__, \"initialized_ui\"))\n\n @property\n def deactivatable(self):\n \"\"\"\n Property for **self.__deactivatable** attribute.\n\n :return: self.__deactivatable.\n :rtype: unicode\n \"\"\"\n\n return self.__deactivatable\n\n @deactivatable.setter\n @foundations.exceptions.handle_exceptions(AssertionError)\n def deactivatable(self, value):\n \"\"\"\n Setter for **self.__deactivatable** attribute.\n\n :param value: Attribute value.\n :type value: unicode\n \"\"\"\n\n if value is not None:\n assert type(value) is bool, \"'{0}' attribute: '{1}' type is not 'bool'!\".format(\"deactivatable\", value)\n self.__deactivatable = value\n\n @deactivatable.deleter\n @foundations.exceptions.handle_exceptions(foundations.exceptions.ProgrammingError)\n def deactivatable(self):\n \"\"\"\n Deleter for **self.__deactivatable** attribute.\n \"\"\"\n\n raise foundations.exceptions.ProgrammingError(\"{0} | '{1}' attribute is not deletable!\".format(\n self.__class__.__name__, \"deactivatable\"))\n\n @foundations.exceptions.handle_exceptions(NotImplementedError)\n def activate(self):\n \"\"\"\n Sets Component activation state.\n\n :return: Method success.\n :rtype: bool\n \"\"\"\n\n raise NotImplementedError(\"{0} | '{1}' must be implemented by '{2}' subclasses!\".format(\n self.__class__.__name__, self.activate.__name__, self.__class__.__name__))\n\n @foundations.exceptions.handle_exceptions(NotImplementedError)\n def deactivate(self):\n \"\"\"\n Unsets Component activation state.\n\n :return: Method success.\n :rtype: bool\n \"\"\"\n\n raise NotImplementedError(\"{0} | '{1}' must be implemented by '{2}' subclasses!\".format(\n self.__class__.__name__, self.deactivate.__name__, self.__class__.__name__))\n\n @foundations.exceptions.handle_exceptions(NotImplementedError)\n def initialize_ui(self):\n \"\"\"\n Initializes the Component ui.\n \"\"\"\n\n raise NotImplementedError(\"{0} | '{1}' must be implemented by '{2}' subclasses!\".format(\n self.__class__.__name__, self.deactivate.__name__, self.__class__.__name__))\n\n @foundations.exceptions.handle_exceptions(NotImplementedError)\n def add_widget(self):\n \"\"\"\n Adds the Component Widget ui.\n \"\"\"\n\n raise NotImplementedError(\"{0} | '{1}' must be implemented by '{2}' subclasses!\".format(\n self.__class__.__name__, self.deactivate.__name__, self.__class__.__name__))\n\n @foundations.exceptions.handle_exceptions(NotImplementedError)\n def remove_widget(self):\n \"\"\"\n Removes the Component Widget ui.\n \"\"\"\n\n raise NotImplementedError(\"{0} | '{1}' must be implemented by '{2}' subclasses!\".format(\n self.__class__.__name__, self.deactivate.__name__, self.__class__.__name__))\n\n @foundations.exceptions.handle_exceptions(NotImplementedError)\n def uninitialize_ui(self):\n \"\"\"\n Uninitializes the Component ui.\n \"\"\"\n\n raise NotImplementedError(\"{0} | '{1}' must be implemented by '{2}' subclasses!\".format(\n self.__class__.__name__, self.deactivate.__name__, self.__class__.__name__))\n\n return QWidgetComponent"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nremoving volume node if it follows a journal logically in the tree hierarchy.", "response": "def _restructure_if_volume_follows_journal(left, right):\n \"\"\"Remove volume node if it follows a journal logically in the tree hierarchy.\n\n Args:\n left (ast.ASTElement): The journal KeywordOp node.\n right (ast.ASTElement): The rest of the tree to be restructured.\n\n Return:\n (ast.ASTElement): The restructured tree, with the volume node removed.\n\n Notes:\n This happens to support queries like \"journal Phys.Rev. and vol d85\". Appends the value of KeywordOp with\n Keyword 'volume' and discards 'volume' KeywordOp node from the tree.\n \"\"\"\n def _get_volume_keyword_op_and_remaining_subtree(right_subtree):\n if isinstance(right_subtree, NotOp) and isinstance(right_subtree.op, KeywordOp) \\\n and right_subtree.op.left == Keyword('volume'):\n return None, None\n\n elif isinstance(right_subtree, AndOp) and isinstance(right_subtree.left, NotOp) \\\n and isinstance(right_subtree.left.op, KeywordOp) and right_subtree.left.op.left == Keyword('volume'):\n return None, right_subtree.right\n\n elif isinstance(right_subtree, KeywordOp) and right_subtree.left == Keyword('volume'):\n return right_subtree, None\n\n elif isinstance(right_subtree, AndOp) and right_subtree.left.left == Keyword('volume'):\n return right_subtree.left, right_subtree.right\n\n journal_value = left.right.value\n\n volume_and_remaining_subtree = _get_volume_keyword_op_and_remaining_subtree(right)\n if not volume_and_remaining_subtree:\n return\n\n volume_node, remaining_subtree = volume_and_remaining_subtree\n if volume_node:\n left.right.value = ','.join([journal_value, volume_node.right.value])\n\n return AndOp(left, remaining_subtree) if remaining_subtree else left"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nconverts a SimpleValueBooleanQuery to AndOp queries.", "response": "def _convert_simple_value_boolean_query_to_and_boolean_queries(tree, keyword):\n \"\"\"Chain SimpleValueBooleanQuery values into chained AndOp queries with the given current Keyword.\"\"\"\n\n def _create_operator_node(value_node):\n \"\"\"Creates a KeywordOp or a ValueOp node.\"\"\"\n base_node = value_node.op if isinstance(value_node, NotOp) else value_node\n updated_base_node = KeywordOp(keyword, base_node) if keyword else ValueOp(base_node)\n\n return NotOp(updated_base_node) if isinstance(value_node, NotOp) else updated_base_node\n\n def _get_bool_op_type(bool_op):\n return AndOp if isinstance(bool_op, And) else OrOp\n\n new_tree_root = _get_bool_op_type(tree.bool_op)(None, None)\n current_tree = new_tree_root\n previous_tree = tree\n\n while True: # Walk down the tree while building the new AndOp queries subtree.\n current_tree.left = _create_operator_node(previous_tree.left)\n\n if not isinstance(previous_tree.right, SimpleValueBooleanQuery):\n current_tree.right = _create_operator_node(previous_tree.right)\n break\n\n previous_tree = previous_tree.right\n current_tree.right = _get_bool_op_type(previous_tree.bool_op)(None, None)\n current_tree = current_tree.right\n\n return new_tree_root"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef visit_boolean_query(self, node):\n left = node.left.accept(self)\n right = node.right.accept(self)\n\n is_journal_keyword_op = isinstance(left, KeywordOp) and left.left == Keyword('journal')\n\n if is_journal_keyword_op:\n journal_and_volume_conjunction = _restructure_if_volume_follows_journal(left, right)\n\n if journal_and_volume_conjunction:\n return journal_and_volume_conjunction\n\n return AndOp(left, right) if isinstance(node.bool_op, And) else OrOp(left, right)", "response": "Convert BooleanRule into AndOp or OrOp nodes."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nvisits the SimpleValueBooleanQuery node and replace the node with the actual node type.", "response": "def visit_simple_value_boolean_query(self, node):\n \"\"\"\n Visits only the children of :class:`SimpleValueBooleanQuery` without substituting the actual node type.\n\n Notes:\n Defer conversion from :class:`SimpleValueBooleanQuery` to AndOp or OrOp.\n This transformation needs to occur higher in the tree, so that we don't lose the information that this is a\n boolean query among terminals and thus the associative rule needs to be applied if we reached here from a\n keyword query, or a conversion from :class:`SimpleValueBooleanQuery` to :class:`AndOp` or :class:`OrOp`,\n otherwise.\n \"\"\"\n node.left, node.right = node.left.accept(self), node.right.accept(self)\n return node"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef visit_spires_keyword_query(self, node):\n keyword = node.left.accept(self)\n value = node.right.accept(self)\n\n if isinstance(value, SimpleValueBooleanQuery):\n return _convert_simple_value_boolean_query_to_and_boolean_queries(value, keyword)\n\n return KeywordOp(keyword, value)", "response": "Visit a SpiresKeywordQuery node by returning a KeywordOp."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nvisit an InvenioKeywordQuery node by returning a KeywordOp.", "response": "def visit_invenio_keyword_query(self, node):\n \"\"\"Transform an :class:`InvenioKeywordQuery` into a :class:`KeywordOp`.\n\n Notes:\n In case the value being a :class:`SimpleValueBooleanQuery`, the subtree is transformed to chained\n :class:`AndOp` queries containing :class:`KeywordOp`, whose keyword is the keyword of the current node and\n values, all the :class:`SimpleValueBooleanQuery` values (either :class:`SimpleValues` or\n :class:`SimpleValueNegation`.)\n \"\"\"\n try:\n keyword = node.left.accept(self)\n except AttributeError:\n # The keywords whose values aren't an InspireKeyword are simple strings.\n keyword = Keyword(node.left)\n\n value = node.right.accept(self)\n\n if isinstance(value, SimpleValueBooleanQuery):\n return _convert_simple_value_boolean_query_to_and_boolean_queries(value, keyword)\n\n return KeywordOp(keyword, value)"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nconverting a node to one of ExactMatch PartialMatch and Regex Value nodes.", "response": "def visit_complex_value(self, node):\n \"\"\"Convert :class:`ComplexValue` to one of ExactMatch, PartialMatch and Regex Value nodes.\"\"\"\n if node.value.startswith(ComplexValue.EXACT_VALUE_TOKEN):\n value = node.value.strip(ComplexValue.EXACT_VALUE_TOKEN)\n return ExactMatchValue(value)\n\n elif node.value.startswith(ComplexValue.PARTIAL_VALUE_TOKEN):\n value = node.value.strip(ComplexValue.PARTIAL_VALUE_TOKEN)\n return PartialMatchValue(value, True if ast.GenericValue.WILDCARD_TOKEN in value else False)\n\n elif node.value.startswith(ComplexValue.REGEX_VALUE_TOKEN):\n return RegexValue(node.value.strip(ComplexValue.REGEX_VALUE_TOKEN))\n else:\n # Covering the case where ComplexValue supports more than ExactMatch, PartialMatch and Regex values.\n msg = self.__class__.__name__ + ': Unrecognized complex value'\n try:\n msg += ' lookahead token: \"' + node.value[0] + '\"'\n except IndexError:\n msg += ': \\\"' + repr(node.value) + '\"'\n msg += '.\\nUsing simple value instead: \"' + node.value + '\".'\n logger.warn(msg)\n return ast.Value(node.value)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\nasync def stop(self, force: bool=False) -> None:\n '''Cancel the task if it hasn't yet started, or tell it to\n gracefully stop running if it has.'''\n\n Log.debug('stopping task %s', self.name)\n self.running = False\n\n if force:\n self.task.cancel()", "response": "Cancel the task if it hasn t yet started."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nrecognize whether the author name contains full name parts and not initials or only lastname.", "response": "def author_name_contains_fullnames(author_name):\n \"\"\"Recognizes whether the name contains full name parts and not initials or only lastname.\n\n Returns:\n bool: True if name has only full name parts, e.g. 'Ellis John', False otherwise. So for example, False is\n returned for 'Ellis, J.' or 'Ellis'.\n \"\"\"\n def _is_initial(name_part):\n return len(name_part) == 1 or u'.' in name_part\n\n parsed_name = ParsedName(author_name)\n\n if len(parsed_name) == 1:\n return False\n elif any([_is_initial(name_part) for name_part in parsed_name]):\n return False\n\n return True"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\ndetects whether the name variation consists only from initials.", "response": "def _name_variation_has_only_initials(name):\n \"\"\"Detects whether the name variation consists only from initials.\"\"\"\n def _is_initial(name_variation):\n return len(name_variation) == 1 or u'.' in name_variation\n\n parsed_name = ParsedName.loads(name)\n\n return all([_is_initial(name_part) for name_part in parsed_name])"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef generate_minimal_name_variations(author_name):\n parsed_name = ParsedName.loads(unidecode(author_name))\n\n if len(parsed_name) > 1:\n lastnames = parsed_name.last.replace('-', ' ')\n\n non_lastnames = ' '.join(\n parsed_name.first_list + parsed_name.suffix_list\n )\n # Strip extra whitespace added if any of middle_list and suffix_list are empty.\n non_lastnames = non_lastnames.strip().replace('-', ' ')\n\n # Adding into a set first, so as to drop identical name variations.\n return list({\n name_variation.lower()\n for name_variation\n in [\n lastnames + ' ' + non_lastnames,\n lastnames + ' ' + non_lastnames[0],\n non_lastnames + ' ' + lastnames,\n non_lastnames + ' ' + lastnames[0],\n ]\n if not _name_variation_has_only_initials(name_variation)\n })\n else:\n return [parsed_name.dumps().replace('-', ' ').lower()]", "response": "Generates a small number of name variations."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\ntruncating wildcard from date parts.", "response": "def _truncate_wildcard_from_date(date_value):\n \"\"\"Truncate wildcard from date parts.\n\n Returns:\n (str) The truncated date.\n\n Raises:\n ValueError, on either unsupported date separator (currently only ' ' and '-' are supported), or if there's a\n wildcard in the year.\n\n Notes:\n Either whole date part is wildcard, in which we ignore it and do a range query on the\n remaining parts, or some numbers are wildcards, where again, we ignore this part.\n \"\"\"\n if ' ' in date_value:\n date_parts = date_value.split(' ')\n elif '-' in date_value:\n date_parts = date_value.split('-')\n else:\n # Either unsupported separators or wildcard in year, e.g. '201*'.\n raise ValueError(\"Erroneous date value: %s.\", date_value)\n\n if GenericValue.WILDCARD_TOKEN in date_parts[-1]:\n del date_parts[-1]\n\n return '-'.join(date_parts)"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\ntruncate the given date value according to the given date field.", "response": "def _truncate_date_value_according_on_date_field(field, date_value):\n \"\"\"Truncates date value (to year only) according to the given date field.\n\n Args:\n field (unicode): The field for which the date value will be used to query on.\n date_value (str): The date value that is going to be truncated to its year.\n\n Returns:\n PartialDate: The possibly truncated date, on success. None, otherwise.\n\n Notes:\n In case the fieldname is in `ES_MAPPING_HEP_DATE_ONLY_YEAR`, then the date is normalized and then only its year\n value is used. This is needed for ElasticSearch to be able to do comparisons on dates that have only year, which\n fails if being queried with a date with more .\n \"\"\"\n try:\n partial_date = PartialDate.parse(date_value)\n except ValueError:\n return None\n\n if field in ES_MAPPING_HEP_DATE_ONLY_YEAR:\n truncated_date = PartialDate.from_parts(partial_date.year)\n else:\n truncated_date = partial_date\n\n return truncated_date"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _get_next_date_from_partial_date(partial_date):\n relativedelta_arg = 'years'\n\n if partial_date.month:\n relativedelta_arg = 'months'\n if partial_date.day:\n relativedelta_arg = 'days'\n\n next_date = parse(partial_date.dumps()) + relativedelta(**{relativedelta_arg: 1})\n return PartialDate.from_parts(\n next_date.year,\n next_date.month if partial_date.month else None,\n next_date.day if partial_date.day else None\n )", "response": "Calculates the next date from the given partial date."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _get_proper_elastic_search_date_rounding_format(partial_date):\n es_date_math_unit = ES_DATE_MATH_ROUNDING_YEAR\n\n if partial_date.month:\n es_date_math_unit = ES_DATE_MATH_ROUNDING_MONTH\n if partial_date.day:\n es_date_math_unit = ES_DATE_MATH_ROUNDING_DAY\n\n return es_date_math_unit", "response": "Returns the proper ES date math unit according to the resolution of the partial_date."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nupdating the operator - value pairs for a given field.", "response": "def update_date_value_in_operator_value_pairs_for_fieldname(field, operator_value_pairs):\n \"\"\"Updates (operator, date value) pairs by normalizing the date value according to the given field.\n\n Args:\n field (unicode): The fieldname for which the operator-value pairs are being generated.\n operator_value_pairs (dict): ES range operator {'gt', 'gte', 'lt', 'lte'} along with a value.\n Additionally, if the operator is ``ES_RANGE_EQ_OPERATOR``, then it is indicated that the method should\n generate both a lower and an upper bound operator-value pairs, with the given date_value.\n\n Notes:\n On a ``ValueError`` an empty operator_value_pairs is returned.\n \"\"\"\n updated_operator_value_pairs = {}\n for operator, value in operator_value_pairs.items():\n modified_date = _truncate_date_value_according_on_date_field(field, value)\n if not modified_date:\n return {}\n\n if operator == ES_RANGE_EQ_OPERATOR:\n updated_operator_value_pairs['gte'] = \\\n modified_date.dumps() + _get_proper_elastic_search_date_rounding_format(modified_date)\n\n next_date = _get_next_date_from_partial_date(modified_date)\n updated_operator_value_pairs['lt'] = \\\n next_date.dumps() + _get_proper_elastic_search_date_rounding_format(next_date)\n else:\n updated_operator_value_pairs[operator] = \\\n modified_date.dumps() + _get_proper_elastic_search_date_rounding_format(modified_date)\n\n return updated_operator_value_pairs"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef generate_match_query(field, value, with_operator_and):\n parsed_value = None\n try:\n parsed_value = json.loads(value.lower())\n except (ValueError, TypeError, AttributeError):\n # Catch all possible exceptions\n # we are not interested if they will appear\n pass\n\n if isinstance(value, bool):\n return {'match': {field: value}}\n elif isinstance(parsed_value, bool):\n return {'match': {field: value.lower()}}\n\n if with_operator_and:\n return {\n 'match': {\n field: {\n 'query': value,\n 'operator': 'and'\n }\n }\n }\n\n return {'match': {field: value}}", "response": "Generates a match query for the given ES field and value."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nwrapping a query into a nested if the fields within the query are nested.", "response": "def wrap_query_in_nested_if_field_is_nested(query, field, nested_fields):\n \"\"\"Helper for wrapping a query into a nested if the fields within the query are nested\n\n Args:\n query : The query to be wrapped.\n field : The field that is being queried.\n nested_fields : List of fields which are nested.\n Returns:\n (dict): The nested query\n \"\"\"\n for element in nested_fields:\n match_pattern = r'^{}.'.format(element)\n if re.match(match_pattern, field):\n return generate_nested_query(element, query)\n\n return query"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nreturn True if the difference between date and any value in dates is less than or equal to limit.", "response": "def is_within_limits(self, limit, date, dates):\n \"\"\"\n Returns True if the difference between date and any value in dates\n is less than or equal to limit.\n \"\"\"\n return any((self.second_diff(date, d) <= limit for d in dates))"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef emit_tree_format(tree, verbose=False):\n if verbose:\n print(\"Converting: \" + repr(tree))\n ret_str = __recursive_formatter(tree)\n return ret_str", "response": "Returns a tree - like representation of a parse tree"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\ncalculates an exponential backoff for retry attempts with a small amount of jitter.", "response": "def calculate_retry_delay(attempt, max_delay=300):\n \"\"\"Calculates an exponential backoff for retry attempts with a small\n amount of jitter.\"\"\"\n delay = int(random.uniform(2, 4) ** attempt)\n if delay > max_delay:\n # After reaching the max delay, stop using expontential growth\n # and keep the delay nearby the max.\n delay = int(random.uniform(max_delay - 20, max_delay + 20))\n return delay"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nparse a given query string and returns a multi_match query.", "response": "def parse_query(query_str):\n \"\"\"\n Drives the whole logic, by parsing, restructuring and finally, generating an ElasticSearch query.\n\n Args:\n query_str (six.text_types): the given query to be translated to an ElasticSearch query\n\n Returns:\n six.text_types: Return an ElasticSearch query.\n\n Notes:\n In case there's an error, an ElasticSearch `multi_match` query is generated with its `query` value, being the\n query_str argument.\n \"\"\"\n def _generate_match_all_fields_query():\n # Strip colon character (special character for ES)\n stripped_query_str = ' '.join(query_str.replace(':', ' ').split())\n return {'multi_match': {'query': stripped_query_str, 'fields': ['_all'], 'zero_terms_query': 'all'}}\n\n if not isinstance(query_str, six.text_type):\n query_str = six.text_type(query_str.decode('utf-8'))\n\n logger.info('Parsing: \"' + query_str + '\\\".')\n\n parser = StatefulParser()\n rst_visitor = RestructuringVisitor()\n es_visitor = ElasticSearchVisitor()\n\n try:\n unrecognized_text, parse_tree = parser.parse(query_str, Query)\n\n if unrecognized_text: # Usually, should never happen.\n msg = 'Parser returned unrecognized text: \"' + unrecognized_text + \\\n '\" for query: \"' + query_str + '\".'\n\n if query_str == unrecognized_text and parse_tree is None:\n # Didn't recognize anything.\n logger.warn(msg)\n return _generate_match_all_fields_query()\n else:\n msg += 'Continuing with recognized parse tree.'\n logger.warn(msg)\n\n except SyntaxError as e:\n logger.warn('Parser syntax error (' + six.text_type(e) + ') with query: \"' + query_str +\n '\". Continuing with a match_all with the given query.')\n return _generate_match_all_fields_query()\n\n # Try-Catch-all exceptions for visitors, so that search functionality never fails for the user.\n try:\n restructured_parse_tree = parse_tree.accept(rst_visitor)\n logger.debug('Parse tree: \\n' + emit_tree_format(restructured_parse_tree))\n\n except Exception as e:\n logger.exception(\n RestructuringVisitor.__name__ + \" crashed\" + (\": \" + six.text_type(e) + \".\") if six.text_type(e) else '.'\n )\n return _generate_match_all_fields_query()\n\n try:\n es_query = restructured_parse_tree.accept(es_visitor)\n except Exception as e:\n logger.exception(\n ElasticSearchVisitor.__name__ + \" crashed\" + (\": \" + six.text_type(e) + \".\") if six.text_type(e) else '.'\n )\n return _generate_match_all_fields_query()\n\n if not es_query:\n # Case where an empty query was generated (i.e. date query with malformed date, e.g. \"d < 200\").\n return _generate_match_all_fields_query()\n\n return es_query"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nreturning a running Task object matching the given name or class.", "response": "def task(self, name_or_class: Any) -> Task:\n '''Return a running Task object matching the given name or class.'''\n\n if name_or_class in self.all_tasks:\n return self.all_tasks[name_or_class]\n\n try:\n return self.all_tasks.get(name_or_class.__class__.__name__, None)\n\n except AttributeError:\n return None"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\ninitializing configuration and start tasks.", "response": "async def init(self) -> None:\n '''Initialize configuration and start tasks.'''\n\n self.stats = await self.insert(self.stats)\n self.configuration = await self.insert(self.configuration)\n\n if not self.executor:\n try:\n max_workers = self.config.get('executor_workers')\n except Exception:\n max_workers = None\n\n self.executor = ThreadPoolExecutor(max_workers=max_workers)\n\n for task in self.initial_tasks:\n await self.insert(task)\n\n self.monitor = asyncio.ensure_future(self.monitor_tasks())\n self.counters['alive_since'] = time.time()"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\nasync def insert(self, task: Task) -> None:\n '''Insert the given task class into the Tasky event loop.'''\n\n if not isinstance(task, Task):\n task = task()\n\n if task.name not in self.all_tasks:\n task.tasky = self\n self.all_tasks[task.name] = task\n\n await task.init()\n\n elif task != self.all_tasks[task.name]:\n raise Exception('Duplicate task %s' % task.name)\n\n if task.enabled:\n task.task = asyncio.ensure_future(self.start_task(task))\n self.running_tasks.add(task)\n else:\n task.task = None\n\n return task", "response": "Insert the given task class into the Tasky event loop."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nexecute an arbitrary function outside the event loop using a shared Executor.", "response": "async def execute(self, fn, *args, **kwargs) -> None:\n '''Execute an arbitrary function outside the event loop using\n a shared Executor.'''\n\n fn = functools.partial(fn, *args, **kwargs)\n return await self.loop.run_in_executor(self.executor, fn)"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nexecute the tasky event loop until terminated.", "response": "def run_forever(self) -> None:\n '''Execute the tasky/asyncio event loop until terminated.'''\n\n Log.debug('running event loop until terminated')\n asyncio.ensure_future(self.init())\n self.loop.run_forever()\n self.loop.close()"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef run_until_complete(self) -> None:\n '''Execute the tasky/asyncio event loop until all tasks finish.'''\n\n Log.debug('running event loop until all tasks completed')\n self.terminate_on_finish = True\n asyncio.ensure_future(self.init())\n self.loop.run_forever()\n self.loop.close()", "response": "Execute the tasky event loop until all tasks finish."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nexecuting the tasky event loop for duration seconds.", "response": "def run_for_time(self, duration: float=10.0) -> None:\n '''Execute the tasky/asyncio event loop for `duration` seconds.'''\n\n Log.debug('running event loop for %.1f seconds', duration)\n try:\n asyncio.ensure_future(self.init())\n self.loop.run_until_complete(asyncio.sleep(duration))\n self.terminate()\n self.loop.run_forever()\n\n except RuntimeError as e:\n if not e.args[0].startswith('Event loop stopped'):\n raise\n\n finally:\n self.loop.close()"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nterminates all scheduled and executing tasks.", "response": "def terminate(self, *, force: bool=False, timeout: float=30.0,\n step: float=1.0) -> None:\n '''Stop all scheduled and/or executing tasks, first by asking nicely,\n and then by waiting up to `timeout` seconds before forcefully stopping\n the asyncio event loop.'''\n\n if isinstance(self.monitor, asyncio.Future):\n Log.debug('cancelling task monitor')\n self.monitor.cancel()\n\n Log.debug('stopping tasks')\n for task in list(self.running_tasks):\n if task.task.done():\n Log.debug('task %s already stopped', task.name)\n self.running_tasks.discard(task)\n else:\n Log.debug('asking %s to stop', task.name)\n asyncio.ensure_future(task.stop(force=force))\n\n if timeout > 0 and (self.monitor or self.running_tasks):\n Log.debug('waiting %.1f seconds for remaining tasks (%d)...',\n timeout, len(self.running_tasks))\n\n timeout -= step\n fn = functools.partial(self.terminate, force=force,\n timeout=timeout, step=step)\n return self.loop.call_later(step, fn)\n\n if timeout > 0:\n Log.debug('all tasks completed, stopping event loop')\n else:\n Log.debug('timed out waiting for tasks, stopping event loop')\n\n self.loop.stop()"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\ninitializing the task queue it for execution add the done callback and keep track of it for when tasks need to be stopped.", "response": "async def start_task(self, task: Task) -> None:\n '''Initialize the task, queue it for execution, add the done callback,\n and keep track of it for when tasks need to be stopped.'''\n\n try:\n Log.debug('task %s starting', task.name)\n before = time.time()\n task.counters['last_run'] = before\n\n task.running = True\n self.running_tasks.add(task)\n await task.run_task()\n Log.debug('task %s completed', task.name)\n\n except CancelledError:\n Log.debug('task %s cancelled', task.name)\n\n except Exception:\n Log.exception('unhandled exception in task %s', task.name)\n\n finally:\n self.running_tasks.discard(task)\n task.running = False\n task.task = None\n\n after = time.time()\n total = after - before\n task.counters['last_completed'] = after\n task.counters['duration'] = total"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nmonitor all known tasks for run state.", "response": "async def monitor_tasks(self, interval: float=1.0) -> None:\n '''Monitor all known tasks for run state. Ensure that enabled tasks\n are running, and that disabled tasks are stopped.'''\n\n Log.debug('monitor running')\n while True:\n try:\n await asyncio.sleep(interval)\n\n for name, task in self.all_tasks.items():\n if self.terminate_on_finish:\n if task in self.running_tasks and task.running:\n await task.stop()\n\n elif task.enabled:\n if task not in self.running_tasks:\n Log.debug('task %s enabled, restarting', task.name)\n await self.insert(task)\n\n else:\n if task in self.running_tasks:\n Log.debug('task %s disabled, stopping', task.name)\n await task.stop()\n\n if self.terminate_on_finish and not self.running_tasks:\n Log.debug('all tasks completed, terminating')\n break\n\n except CancelledError:\n Log.debug('monitor cancelled')\n break\n\n except Exception:\n Log.exception('monitoring exception')\n\n self.monitor = None\n self.loop.call_later(0, self.terminate)"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef exception(self, loop: asyncio.BaseEventLoop, context: dict) -> None:\n '''Log unhandled exceptions from anywhere in the event loop.'''\n\n Log.error('unhandled exception: %s', context['message'])\n Log.error('%s', context)\n if 'exception' in context:\n Log.error(' %s', context['exception'])", "response": "Log unhandled exceptions from anywhere in the event loop."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef sigint(self) -> None:\n '''Handle the user pressing Ctrl-C by stopping tasks nicely at first,\n then forcibly upon further presses.'''\n\n if self.stop_attempts < 1:\n Log.info('gracefully stopping tasks')\n self.stop_attempts += 1\n self.terminate()\n\n elif self.stop_attempts < 2:\n Log.info('forcefully cancelling tasks')\n self.stop_attempts += 1\n self.terminate(force=True)\n\n else:\n Log.info('forcefully stopping event loop')\n self.loop.stop()", "response": "Handle the user pressing Ctrl - C by stopping tasks nicely at first and forcibly upon further presses."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nhandles SIGTERM from the system by stopping tasks gracefully.", "response": "def sigterm(self) -> None:\n '''Handle SIGTERM from the system by stopping tasks gracefully.\n Repeated signals will be ignored while waiting for tasks to finish.'''\n\n if self.stop_attempts < 1:\n Log.info('received SIGTERM, gracefully stopping tasks')\n self.stop_attempts += 1\n self.terminate()\n\n else:\n Log.info('received SIGTERM, bravely waiting for tasks')"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nchecking if terminal token is a keyword after lower - casing it.", "response": "def parse(cls, parser, text, pos):\n \"\"\"Checks if terminal token is a keyword after lower-casing it.\"\"\"\n match = cls.regex.match(text)\n if match:\n # Check if match is is not in the grammar of the specific keyword class.\n if match.group(0).lower() not in cls.grammar:\n result = text, SyntaxError(repr(match.group(0)) + \" is not a member of \" + repr(cls.grammar))\n else:\n result = text[len(match.group(0)):], cls(match.group(0))\n else:\n result = text, SyntaxError(\"expecting \" + repr(cls.__name__))\n return result"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef parse(cls, parser, text, pos):\n try:\n remaining_text, keyword = parser.parse(text, cls.grammar)\n if keyword.lower() == 'texkey':\n parser._parsing_texkey_expression = True\n return remaining_text, InspireKeyword(keyword)\n except SyntaxError as e:\n parser._parsing_texkey_expression = False\n return text, e", "response": "Parse InspireKeyword.\n\n If the keyword is `texkey`, enable the parsing texkey expression flag, since its value contains ':' which\n normally isn't allowed."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nparse a terminal token that doesn t contain parentheses nor colon symbol.", "response": "def parse_terminal_token(cls, parser, text):\n \"\"\"Parses a terminal token that doesn't contain parentheses nor colon symbol.\n\n Note:\n Handles a special case of tokens where a ':' is needed (for `texkey` queries).\n\n If we're parsing text not in parentheses, then some DSL keywords (e.g. And, Or, Not, defined above) should\n not be recognized as terminals, thus we check if they are in the Keywords table (namespace like structure\n handled by PyPeg).\n This is done only when we are not parsing a parenthesized SimpleValue.\n\n Also, helps in supporting more implicit-and queries cases (last two checks).\n \"\"\"\n token_regex = cls.token_regex\n if parser._parsing_texkey_expression:\n token_regex = cls.texkey_token_regex\n parser._parsing_texkey_expression = False\n\n match = token_regex.match(text)\n if match:\n matched_token = match.group(0)\n\n # Check if token is a DSL keyword. Disable this check in the case where the parser isn't parsing a\n # parenthesized terminal.\n if not parser._parsing_parenthesized_terminal and matched_token.lower() in Keyword.table:\n return text, SyntaxError(\"found DSL keyword: \" + matched_token)\n\n remaining_text = text[len(matched_token):]\n\n # Attempt to recognize whether current terminal is followed by a \":\", which definitely signifies that\n # we are parsing a keyword, and we shouldn't.\n if cls.starts_with_colon.match(remaining_text):\n return text, \\\n SyntaxError(\"parsing a keyword (token followed by \\\":\\\"): \\\"\" + repr(matched_token) + \"\\\"\")\n\n # Attempt to recognize whether current terminal is a non shortened version of Inspire keywords. This is\n # done for supporting implicit-and in case of SPIRES style keyword queries. Using the non shortened version\n # of the keywords, makes this recognition not eager.\n if not parser._parsing_parenthesized_simple_values_expression \\\n and matched_token in INSPIRE_KEYWORDS_SET:\n return text, SyntaxError(\"parsing a keyword (non shortened INSPIRE keyword)\")\n\n result = remaining_text, matched_token\n else:\n result = text, SyntaxError(\"expecting match on \" + repr(cls.token_regex.pattern))\n return result"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nimitate parsing a list grammar.", "response": "def parse(cls, parser, text, pos):\n \"\"\"Imitates parsing a list grammar.\n\n Specifically, this\n grammar = [\n SimpleValueUnit.date_specifiers_regex,\n SimpleValueUnit.arxiv_token_regex,\n SimpleValueUnit.token_regex,\n SimpleValueUnit.parenthesized_token_grammar\n ].\n\n Parses plaintext which matches date specifiers or arxiv_identifier syntax, or is comprised of either 1) simple\n terminal (no parentheses) or 2) a parenthesized SimpleValue.\n\n For example, \"e(+)\" will be parsed in two steps, first, \"e\" token will be recognized and then \"(+)\", as a\n parenthesized SimpleValue.\n \"\"\"\n found = False\n\n # Attempt to parse date specifier\n match = cls.date_specifiers_regex.match(text)\n if match:\n remaining_text, token, found = text[len(match.group(0)):], match.group(0), True\n else:\n # Attempt to parse arxiv identifier\n match = cls.arxiv_token_regex.match(text)\n if match:\n remaining_text, token, found = text[len(match.group()):], match.group(2), True\n else:\n # Attempt to parse a terminal token\n remaining_text, token = SimpleValueUnit.parse_terminal_token(parser, text)\n if type(token) != SyntaxError:\n found = True\n else:\n # Attempt to parse a terminal with parentheses\n try:\n # Enable parsing a parenthesized terminal so that we can accept {+, -, |} as terminals.\n parser._parsing_parenthesized_terminal = True\n remaining_text, token = parser.parse(text, cls.parenthesized_token_grammar, pos)\n\n found = True\n except SyntaxError:\n pass\n except GrammarValueError:\n raise\n except ValueError:\n pass\n finally:\n parser._parsing_parenthesized_terminal = False\n\n if found:\n result = remaining_text, SimpleValueUnit(token)\n else:\n result = text, SyntaxError(\"expecting match on \" + cls.__name__)\n\n return result"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef unconsume_and_reconstruct_input(remaining_text, recognized_tokens, complex_value_idx):\n # Default slicing index: i.e. at most 3 elements will be unconsumed, Keyword, Whitespace and ComplexValue.\n slicing_start_idx = 2\n\n # Check whether the 3rd element from the end is an InspireKeyword. If not, a Value query with ComplexValue\n # was consumed.\n if not INSPIRE_PARSER_KEYWORDS.get(recognized_tokens[complex_value_idx - slicing_start_idx].value, None):\n slicing_start_idx = 1\n\n reconstructed_terminals = recognized_tokens[:complex_value_idx - slicing_start_idx]\n reconstructed_text = '{} {}'.format(\n ''.join([token.value for token in recognized_tokens[complex_value_idx - slicing_start_idx:]]),\n remaining_text\n )\n return reconstructed_text, reconstructed_terminals", "response": "Reconstructs input in case of consuming a keyword query or a value query with ComplexValue as value."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nparses the current state of the object.", "response": "def parse(cls, parser, text, pos):\n \"\"\"Using our own parse to enable the flag below.\"\"\"\n try:\n parser._parsing_parenthesized_simple_values_expression = True\n remaining_text, recognized_tokens = parser.parse(text, cls.grammar)\n return remaining_text, recognized_tokens\n except SyntaxError as e:\n return text, e\n finally:\n parser._parsing_parenthesized_simple_values_expression = False"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef _generate_fieldnames_if_bai_query(self, node_value, bai_field_variation, query_bai_field_if_dots_in_name):\n if bai_field_variation not in (FieldVariations.search, FieldVariations.raw):\n raise ValueError('Non supported field variation \"{}\".'.format(bai_field_variation))\n\n normalized_author_name = normalize_name(node_value).strip('.')\n\n if ElasticSearchVisitor.KEYWORD_TO_ES_FIELDNAME['author'] and \\\n ElasticSearchVisitor.BAI_REGEX.match(node_value):\n return [ElasticSearchVisitor.AUTHORS_BAI_FIELD + '.' + bai_field_variation]\n\n elif not whitespace.search(normalized_author_name) and \\\n query_bai_field_if_dots_in_name and \\\n ElasticSearchVisitor.KEYWORD_TO_ES_FIELDNAME['author'] and \\\n '.' in normalized_author_name:\n # Case of partial BAI, e.g. ``J.Smith``.\n return [ElasticSearchVisitor.AUTHORS_BAI_FIELD + '.' + bai_field_variation] + \\\n force_list(ElasticSearchVisitor.KEYWORD_TO_ES_FIELDNAME['author'])\n\n else:\n return None", "response": "Generates new fieldnames in case of BAI query."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\ngenerate a query handling specifically authors.", "response": "def _generate_author_query(self, author_name):\n \"\"\"Generates a query handling specifically authors.\n\n Notes:\n The match query is generic enough to return many results. Then, using the filter clause we truncate these\n so that we imitate legacy's behaviour on returning more \"exact\" results. E.g. Searching for `Smith, John`\n shouldn't return papers of 'Smith, Bob'.\n\n Additionally, doing a ``match`` with ``\"operator\": \"and\"`` in order to be even more exact in our search, by\n requiring that ``full_name`` field contains both\n \"\"\"\n name_variations = [name_variation.lower()\n for name_variation\n in generate_minimal_name_variations(author_name)]\n\n # When the query contains sufficient data, i.e. full names, e.g. ``Mele, Salvatore`` (and not ``Mele, S`` or\n # ``Mele``) we can improve our filtering in order to filter out results containing records with authors that\n # have the same non lastnames prefix, e.g. 'Mele, Samuele'.\n if author_name_contains_fullnames(author_name):\n specialized_author_filter = [\n {\n 'bool': {\n 'must': [\n {\n 'term': {ElasticSearchVisitor.AUTHORS_NAME_VARIATIONS_FIELD: names_variation[0]}\n },\n generate_match_query(\n ElasticSearchVisitor.KEYWORD_TO_ES_FIELDNAME['author'],\n names_variation[1],\n with_operator_and=True\n )\n ]\n }\n } for names_variation\n in product(name_variations, name_variations)\n ]\n\n else:\n # In the case of initials or even single lastname search, filter with only the name variations.\n specialized_author_filter = [\n {'term': {ElasticSearchVisitor.AUTHORS_NAME_VARIATIONS_FIELD: name_variation}}\n for name_variation in name_variations\n ]\n\n query = {\n 'bool': {\n 'filter': {\n 'bool': {\n 'should': specialized_author_filter\n }\n },\n 'must': {\n 'match': {\n ElasticSearchVisitor.KEYWORD_TO_ES_FIELDNAME['author']: author_name\n }\n }\n }\n }\n\n return generate_nested_query(ElasticSearchVisitor.AUTHORS_NESTED_QUERY_PATH, query)"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\ngenerate a term query handling authors and BAIs.", "response": "def _generate_exact_author_query(self, author_name_or_bai):\n \"\"\"Generates a term query handling authors and BAIs.\n\n Notes:\n If given value is a BAI, search for the provided value in the raw field variation of\n `ElasticSearchVisitor.AUTHORS_BAI_FIELD`.\n Otherwise, the value will be procesed in the same way as the indexed value (i.e. lowercased and normalized\n (inspire_utils.normalize_name and then NFKC normalization).\n E.g. Searching for 'Smith, J.' is the same as searching for: 'Smith, J', 'smith, j.', 'smith j', 'j smith',\n 'j. smith', 'J Smith', 'J. Smith'.\n \"\"\"\n if ElasticSearchVisitor.BAI_REGEX.match(author_name_or_bai):\n bai = author_name_or_bai.lower()\n query = self._generate_term_query(\n '.'.join((ElasticSearchVisitor.AUTHORS_BAI_FIELD, FieldVariations.search)),\n bai\n )\n else:\n author_name = normalize('NFKC', normalize_name(author_name_or_bai)).lower()\n query = self._generate_term_query(\n ElasticSearchVisitor.KEYWORD_TO_ES_FIELDNAME['exact-author'],\n author_name\n )\n\n return generate_nested_query(ElasticSearchVisitor.AUTHORS_NESTED_QUERY_PATH, query)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _generate_queries_for_title_symbols(title_field, query_value):\n values_tokenized_by_whitespace = query_value.split()\n\n symbol_queries = []\n for value in values_tokenized_by_whitespace:\n # Heuristic: If there's a symbol-indicating-character in the value, it signifies terms that should be\n # queried against the whitespace-tokenized title.\n if any(character in value for character in ElasticSearchVisitor.TITLE_SYMBOL_INDICATING_CHARACTER):\n symbol_queries.append(\n generate_match_query(\n '.'.join([title_field, FieldVariations.search]),\n value,\n with_operator_and=False\n )\n )\n\n return wrap_queries_in_bool_clauses_if_more_than_one(symbol_queries, use_must_clause=True)", "response": "Generates a list of queries for any symbols in the title against the whitespace tokenized field of titles."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef _generate_type_code_query(self, value):\n mapping_for_value = self.TYPECODE_VALUE_TO_FIELD_AND_VALUE_PAIRS_MAPPING.get(value, None)\n\n if mapping_for_value:\n return generate_match_query(*mapping_for_value, with_operator_and=True)\n else:\n return {\n 'bool': {\n 'minimum_should_match': 1,\n 'should': [\n generate_match_query('document_type', value, with_operator_and=True),\n generate_match_query('publication_type', value, with_operator_and=True),\n ]\n }\n }", "response": "Generate type - code queries."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef _generate_range_queries(self, fieldnames, operator_value_pairs):\n if ElasticSearchVisitor.KEYWORD_TO_ES_FIELDNAME['date'] == fieldnames:\n range_queries = []\n for fieldname in fieldnames:\n updated_operator_value_pairs = \\\n update_date_value_in_operator_value_pairs_for_fieldname(fieldname, operator_value_pairs)\n\n if not updated_operator_value_pairs:\n break # Malformed date\n else:\n range_query = {\n 'range': {\n fieldname: updated_operator_value_pairs\n }\n }\n\n range_queries.append(\n generate_nested_query(ElasticSearchVisitor.DATE_NESTED_QUERY_PATH, range_query)\n if fieldname in ElasticSearchVisitor.DATE_NESTED_FIELDS\n else range_query\n )\n else:\n range_queries = [{\n 'range': {\n fieldname: operator_value_pairs\n }\n }\n for fieldname in fieldnames\n ]\n\n return wrap_queries_in_bool_clauses_if_more_than_one(range_queries, use_must_clause=False)", "response": "Generates ElasticSearch range queries."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\ngenerates a query string that can be used to search for the content of the _all field.", "response": "def _generate_malformed_query(data):\n \"\"\"Generates a query on the ``_all`` field with all the query content.\n\n Args:\n data (six.text_type or list): The query in the format of ``six.text_type`` (when used from parsing driver)\n or ``list`` when used from withing the ES visitor.\n \"\"\"\n if isinstance(data, six.text_type):\n # Remove colon character (special character for ES)\n query_str = data.replace(':', ' ')\n else:\n query_str = ' '.join([word.strip(':') for word in data.children])\n\n return {\n 'simple_query_string': {\n 'fields': ['_all'],\n 'query': query_str\n }\n }"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef _preprocess_journal_query_value(third_journal_field, old_publication_info_values):\n # Prepare old publication info for :meth:`inspire_schemas.utils.convert_old_publication_info_to_new`.\n publication_info_keys = [\n ElasticSearchVisitor.JOURNAL_TITLE,\n ElasticSearchVisitor.JOURNAL_VOLUME,\n third_journal_field,\n ]\n values_list = [\n value.strip()\n for value\n in old_publication_info_values.split(',')\n if value\n ]\n\n old_publication_info = [\n {\n key: value\n for key, value\n in zip(publication_info_keys, values_list)\n if value\n }\n ]\n\n # We are always assuming that the returned list will not be empty. In the situation of a journal query with no\n # value, a malformed query will be generated instead.\n new_publication_info = convert_old_publication_info_to_new(old_publication_info)[0]\n\n return new_publication_info", "response": "Transforms the given journal query value to the new one."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef _generate_journal_nested_queries(self, value):\n # Abstract away which is the third field, we care only for its existence.\n third_journal_field = ElasticSearchVisitor.JOURNAL_PAGE_START\n\n new_publication_info = ElasticSearchVisitor._preprocess_journal_query_value(third_journal_field, value)\n\n # We always expect a journal title, otherwise query would be considered malformed, and thus this method would\n # not have been called.\n queries_for_each_field = [\n generate_match_query(ElasticSearchVisitor.JOURNAL_FIELDS_MAPPING[ElasticSearchVisitor.JOURNAL_TITLE],\n new_publication_info[ElasticSearchVisitor.JOURNAL_TITLE],\n with_operator_and=False)\n ]\n\n if ElasticSearchVisitor.JOURNAL_VOLUME in new_publication_info:\n queries_for_each_field.append(\n generate_match_query(\n ElasticSearchVisitor.JOURNAL_FIELDS_MAPPING[ElasticSearchVisitor.JOURNAL_VOLUME],\n new_publication_info[ElasticSearchVisitor.JOURNAL_VOLUME],\n with_operator_and=False\n )\n )\n\n if third_journal_field in new_publication_info:\n artid_or_page_start = new_publication_info[third_journal_field]\n match_queries = [\n generate_match_query(\n ElasticSearchVisitor.JOURNAL_FIELDS_MAPPING[third_field],\n artid_or_page_start,\n with_operator_and=False\n )\n for third_field\n in (ElasticSearchVisitor.JOURNAL_PAGE_START, ElasticSearchVisitor.JOURNAL_ART_ID)\n ]\n\n queries_for_each_field.append(\n wrap_queries_in_bool_clauses_if_more_than_one(match_queries, use_must_clause=False)\n )\n\n return generate_nested_query(\n ElasticSearchVisitor.JOURNAL_FIELDS_PREFIX,\n wrap_queries_in_bool_clauses_if_more_than_one(queries_for_each_field, use_must_clause=True)\n )", "response": "Generates ElasticSearch nested queries for the given value."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef visit_exact_match_value(self, node, fieldnames=None):\n if not fieldnames:\n fieldnames = ['_all']\n else:\n fieldnames = force_list(fieldnames)\n\n if ElasticSearchVisitor.KEYWORD_TO_ES_FIELDNAME['exact-author'] == fieldnames[0]:\n return self._generate_exact_author_query(node.value)\n\n elif ElasticSearchVisitor.KEYWORD_TO_ES_FIELDNAME['type-code'] == fieldnames[0]:\n return self._generate_type_code_query(node.value)\n\n elif ElasticSearchVisitor.KEYWORD_TO_ES_FIELDNAME['journal'] == fieldnames:\n return self._generate_journal_nested_queries(node.value)\n\n bai_fieldnames = self._generate_fieldnames_if_bai_query(\n node.value,\n bai_field_variation=FieldVariations.raw,\n query_bai_field_if_dots_in_name=False\n )\n\n if ElasticSearchVisitor.KEYWORD_TO_ES_FIELDNAME['date'] == fieldnames:\n term_queries = []\n for field in fieldnames:\n term_query = \\\n {'term': {field: _truncate_date_value_according_on_date_field(field, node.value).dumps()}}\n\n term_queries.append(\n generate_nested_query(ElasticSearchVisitor.DATE_NESTED_QUERY_PATH, term_query)\n if field in ElasticSearchVisitor.DATE_NESTED_FIELDS\n else term_query\n )\n elif ElasticSearchVisitor.KEYWORD_TO_ES_FIELDNAME['author'] in fieldnames:\n term_queries = [\n generate_nested_query(ElasticSearchVisitor.AUTHORS_NESTED_QUERY_PATH, {'term': {field: node.value}})\n for field in (bai_fieldnames or fieldnames)\n ]\n else:\n term_queries = [{'term': {field: node.value}} for field in (bai_fieldnames or fieldnames)]\n\n return wrap_queries_in_bool_clauses_if_more_than_one(term_queries, use_must_clause=False)", "response": "Generates a term query that matches the value of a node."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef visit_partial_match_value(self, node, fieldnames=None):\n if ElasticSearchVisitor.KEYWORD_TO_ES_FIELDNAME['date'] == fieldnames:\n # Date queries with partial values are transformed into range queries, among the given and the exact\n # next date, according to the granularity of the given date.\n if node.contains_wildcard:\n return self._generate_date_with_wildcard_query(node.value)\n\n return self._generate_range_queries(force_list(fieldnames), {ES_RANGE_EQ_OPERATOR: node.value})\n\n if ElasticSearchVisitor.KEYWORD_TO_ES_FIELDNAME['exact-author'] == fieldnames:\n return self._generate_exact_author_query(node.value)\n\n elif ElasticSearchVisitor.KEYWORD_TO_ES_FIELDNAME['type-code'] == fieldnames:\n return self._generate_type_code_query(node.value)\n\n elif ElasticSearchVisitor.KEYWORD_TO_ES_FIELDNAME['journal'] == fieldnames:\n return self._generate_journal_nested_queries(node.value)\n\n # Add wildcard token as prefix and suffix.\n value = \\\n ('' if node.value.startswith(ast.GenericValue.WILDCARD_TOKEN) else '*') + \\\n node.value + \\\n ('' if node.value.endswith(ast.GenericValue.WILDCARD_TOKEN) else '*')\n\n bai_fieldnames = self._generate_fieldnames_if_bai_query(\n node.value,\n bai_field_variation=FieldVariations.search,\n query_bai_field_if_dots_in_name=True\n )\n\n query = self._generate_query_string_query(value,\n fieldnames=bai_fieldnames or fieldnames,\n analyze_wildcard=True)\n if (bai_fieldnames and ElasticSearchVisitor.KEYWORD_TO_ES_FIELDNAME['author'] in bai_fieldnames) \\\n or (fieldnames and ElasticSearchVisitor.KEYWORD_TO_ES_FIELDNAME['author'] in fieldnames):\n return generate_nested_query(ElasticSearchVisitor.AUTHORS_NESTED_QUERY_PATH, query)\n\n return query", "response": "Generates a query which looks for a substring of the node s value in the given fieldname."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nsynchronises the schedule specified by the ID schedule_id to the scheduler service.", "response": "def run(self, schedule_id, **kwargs):\n \"\"\"\n Synchronises the schedule specified by the ID `schedule_id` to the\n scheduler service.\n\n Arguments:\n schedule_id {str} -- The ID of the schedule to sync\n \"\"\"\n log = self.get_logger(**kwargs)\n\n try:\n schedule = Schedule.objects.get(id=schedule_id)\n except Schedule.DoesNotExist:\n log.error(\"Missing Schedule %s\", schedule_id, exc_info=True)\n\n if schedule.scheduler_schedule_id is None:\n # Create the new schedule\n result = self.scheduler.create_schedule(schedule.scheduler_format)\n schedule.scheduler_schedule_id = result[\"id\"]\n # Disable update signal here to avoid calling twice\n post_save.disconnect(schedule_saved, sender=Schedule)\n schedule.save(update_fields=(\"scheduler_schedule_id\",))\n post_save.connect(schedule_saved, sender=Schedule)\n\n log.info(\n \"Created schedule %s on scheduler for schedule %s\",\n schedule.scheduler_schedule_id,\n schedule.id,\n )\n else:\n # Update the existing schedule\n result = self.scheduler.update_schedule(\n str(schedule.scheduler_schedule_id), schedule.scheduler_format\n )\n log.info(\n \"Updated schedule %s on scheduler for schedule %s\",\n schedule.scheduler_schedule_id,\n schedule.id,\n )"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\ndeactivates the schedule specified by the ID scheduler_schedule_id in the scheduler service.", "response": "def run(self, scheduler_schedule_id, **kwargs):\n \"\"\"\n Deactivates the schedule specified by the ID `scheduler_schedule_id` in\n the scheduler service.\n\n Arguments:\n scheduler_schedule_id {str} -- The ID of the schedule to deactivate\n \"\"\"\n log = self.get_logger(**kwargs)\n\n self.scheduler.update_schedule(scheduler_schedule_id, {\"active\": False})\n log.info(\n \"Deactivated schedule %s in the scheduler service\", scheduler_schedule_id\n )"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nreturning the tags for a specific model or content type.", "response": "def get_for_model(self, obj):\n \"\"\"Returns the tags for a specific model/content type.\"\"\"\n qs = Tag.objects.language(get_language())\n qs = qs.filter(\n tagged_items__content_type=ctype_models.ContentType.objects.get_for_model(obj)) # NOQA\n return qs.distinct()"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nreturning all tags for a whole queryset of objects.", "response": "def get_for_queryset(self, obj_queryset):\n \"\"\"Returns all tags for a whole queryset of objects.\"\"\"\n qs = Tag.objects.language(get_language())\n if obj_queryset.count() == 0:\n return qs.none()\n qs = qs.filter(\n tagged_items__object_id__in=[\n obj.id for obj in obj_queryset],\n tagged_items__content_type=ctype_models.ContentType.objects.get_for_model(obj_queryset[0])) # NOQA\n return qs.distinct()"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\ntask to ensure subscription is bumped or converted", "response": "def post_send_process(context):\n \"\"\"\n Task to ensure subscription is bumped or converted\n \"\"\"\n if \"error\" in context:\n return context\n\n [deserialized_subscription] = serializers.deserialize(\n \"json\", context[\"subscription\"]\n )\n subscription = deserialized_subscription.object\n [messageset] = serializers.deserialize(\"json\", context[\"messageset\"])\n messageset = messageset.object\n\n # Get set max\n set_max = messageset.messages.filter(lang=subscription.lang).count()\n logger.debug(\"set_max calculated - %s\" % set_max)\n\n # Compare user position to max\n if subscription.next_sequence_number == set_max:\n with transaction.atomic():\n # Mark current as completed\n logger.debug(\"marking current subscription as complete\")\n subscription.completed = True\n subscription.active = False\n subscription.process_status = 2 # Completed\n deserialized_subscription.save(\n update_fields=(\"completed\", \"active\", \"process_status\")\n )\n # If next set defined create new subscription\n if messageset.next_set:\n logger.info(\"Creating new subscription for next set\")\n newsub = Subscription.objects.create(\n identity=subscription.identity,\n lang=subscription.lang,\n messageset=messageset.next_set,\n schedule=messageset.next_set.default_schedule,\n )\n logger.debug(\"Created Subscription <%s>\" % newsub.id)\n else:\n # More in this set so increment by one\n logger.debug(\"incrementing next_sequence_number\")\n subscription.next_sequence_number = F(\"next_sequence_number\") + 1\n logger.debug(\"setting process status back to 0\")\n subscription.process_status = 0\n logger.debug(\"saving subscription\")\n deserialized_subscription.save(\n update_fields=(\"next_sequence_number\", \"process_status\")\n )\n # return response\n return \"Subscription for %s updated\" % str(subscription.id)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef calculate_subscription_lifecycle(subscription_id):\n subscription = Subscription.objects.select_related(\"messageset\", \"schedule\").get(\n id=subscription_id\n )\n behind = subscription.messages_behind()\n if behind == 0:\n return\n\n current_messageset = subscription.messageset\n current_sequence_number = subscription.next_sequence_number\n end_subscription = Subscription.fast_forward_lifecycle(subscription, save=False)[-1]\n BehindSubscription.objects.create(\n subscription=subscription,\n messages_behind=behind,\n current_messageset=current_messageset,\n current_sequence_number=current_sequence_number,\n expected_messageset=end_subscription.messageset,\n expected_sequence_number=end_subscription.next_sequence_number,\n )", "response": "Calculates the expected lifecycle position of the subscription in\n ."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef find_behind_subscriptions():\n subscriptions = Subscription.objects.filter(\n active=True, completed=False, process_status=0\n ).values_list(\"id\", flat=True)\n for subscription_id in subscriptions.iterator():\n calculate_subscription_lifecycle.delay(str(subscription_id))", "response": "Finds any subscriptions that are behind according to where they should be and creates a BehindSubscription entry for them."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef send(self, request, pk=None):\n schedule = self.get_object()\n queue_subscription_send.delay(str(schedule.id))\n\n return Response({}, status=status.HTTP_202_ACCEPTED)", "response": "Sends all the subscriptions for the specified schedule"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nreturn the configured value for the given key name or default.", "response": "def get(self, key: Any, default: Any=None) -> Any:\n '''Return the configured value for the given key name, or `default` if\n no value is available or key is invalid.'''\n\n return self.data.get(key, default)"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nreturns the task - specific configuration.", "response": "def task_config(self, task: Task) -> Any:\n '''Return the task-specific configuration.'''\n\n return self.get(task.__class__.__name__)"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nloading configuration in JSON format from either a file or a raw data string.", "response": "async def init(self) -> None:\n '''Load configuration in JSON format from either a file or\n a raw data string.'''\n\n if self.data:\n return\n\n if self.json_data:\n try:\n self.data = json.loads(self.json_data)\n\n except Exception:\n Log.exception('Falied to load raw configuration')\n\n else:\n try:\n with open(self.json_path, 'r') as f:\n self.data = json.load(f)\n\n except Exception:\n Log.exception('Failed to load configuration from %s',\n self.json_path)\n self.data = {}"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nexecuting the task inside the asyncio event loop after self. DELAY seconds.", "response": "async def run_task(self) -> None:\n '''Execute the task inside the asyncio event loop after `DELAY`\n seconds. Track the time it takes to run, and log when it starts/stops.\n If/when `reset()` is called, reset the wait time to `DELAY` seconds.'''\n\n self.last_run = 0.0\n self.target = self.time() + self.DELAY\n\n while self.running:\n try:\n now = self.time()\n\n if now < self.target:\n sleep = self.target - now\n await self.sleep(sleep)\n\n elif self.last_run < self.target:\n Log.debug('executing timer task %s', self.name)\n self.last_run = self.time()\n await self.run()\n total = self.time() - self.last_run\n Log.debug('finished timer task %s in %.1f seconds',\n self.name, total)\n\n else:\n sleep = min(5.0, self.DELAY)\n await self.sleep(sleep)\n\n except CancelledError:\n Log.debug('cancelled timer task %s', self.name)\n raise\n\n except Exception:\n Log.exception('exception in timer task %s', self.name)"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef reset(self) -> None:\n '''Reset task execution to `DELAY` seconds from now.'''\n\n Log.debug('resetting timer task %s')\n self.target = self.time() + self.DELAY", "response": "Reset the task execution to the current time."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef internal_only(view_func):\n\n @functools.wraps(view_func)\n def wrapper(request, *args, **kwargs):\n forwards = request.META.get(\"HTTP_X_FORWARDED_FOR\", \"\").split(\",\")\n # The nginx in the docker container adds the loadbalancer IP to the list inside\n # X-Forwarded-For, so if the list contains more than a single item, we know\n # that it went through our loadbalancer\n if len(forwards) > 1:\n raise PermissionDenied()\n return view_func(request, *args, **kwargs)\n\n return wrapper", "response": "A view decorator which blocks access for requests coming through the load balancer."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nvalidates subscription data before creating Outbound message", "response": "def post(self, request, *args, **kwargs):\n \"\"\" Validates subscription data before creating Outbound message\n \"\"\"\n schedule_disable.delay(kwargs[\"subscription_id\"])\n return Response({\"accepted\": True}, status=201)"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef post(self, request, *args, **kwargs):\n # Look up subscriber\n subscription_id = kwargs[\"subscription_id\"]\n if Subscription.objects.filter(id=subscription_id).exists():\n status = 202\n accepted = {\"accepted\": True}\n store_resend_request.apply_async(args=[subscription_id])\n else:\n status = 400\n accepted = {\n \"accepted\": False,\n \"reason\": \"Cannot find subscription with ID {}\".format(subscription_id),\n }\n return Response(accepted, status=status)", "response": "Validates subscription data before creating Outbound message\n "} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nvalidating subscription data before creating Subscription message", "response": "def post(self, request, *args, **kwargs):\n \"\"\" Validates subscription data before creating Subscription message\n \"\"\"\n # Ensure that we check for the 'data' key in the request object before\n # attempting to reference it\n if \"data\" in request.data:\n # This is a workaround for JSONField not liking blank/null refs\n if \"metadata\" not in request.data[\"data\"]:\n request.data[\"data\"][\"metadata\"] = {}\n\n if \"initial_sequence_number\" not in request.data[\"data\"]:\n request.data[\"data\"][\"initial_sequence_number\"] = request.data[\n \"data\"\n ].get(\"next_sequence_number\")\n\n subscription = SubscriptionSerializer(data=request.data[\"data\"])\n if subscription.is_valid():\n subscription.save()\n # Return\n status = 201\n accepted = {\"accepted\": True}\n return Response(accepted, status=status)\n else:\n status = 400\n return Response(subscription.errors, status=status)\n else:\n status = 400\n message = {\"data\": [\"This field is required.\"]}\n return Response(message, status=status)"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef find_behind_subscriptions(self, request):\n\n task_id = find_behind_subscriptions.delay()\n return Response(\n {\"accepted\": True, \"task_id\": str(task_id)}, status=status.HTTP_202_ACCEPTED\n )", "response": "Starts a celery task that looks through active subscriptions to find\n and adds a\n BehindSubscription for them."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef repl():\n while True:\n try:\n sys.stdout.write(\"Type in next query: \\n> \")\n import locale\n query_str = raw_input().decode(sys.stdin.encoding or locale.getpreferredencoding(True))\n except KeyboardInterrupt:\n break\n\n if u'quit' in query_str:\n break\n\n print_query_and_parse_tree(query_str)", "response": "Read - Eval - Print - Loop for reading the query printing it and its parse tree."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\nasync def run_task(self) -> None:\n '''Initialize the queue and spawn extra worker tasks if this if the\n first task. Then wait for work items to enter the task queue, and\n execute the `run()` method with the current work item.'''\n\n while self.running:\n try:\n item = self.QUEUE.get_nowait()\n\n Log.debug('%s processing work item', self.name)\n await self.run(item)\n\n Log.debug('%s completed work item', self.name)\n self.QUEUE.task_done()\n\n except asyncio.QueueEmpty:\n if self.OPEN:\n await self.sleep(0.05)\n\n else:\n Log.debug('%s queue closed and empty, stopping', self.name)\n return\n\n except CancelledError:\n Log.debug('%s cancelled, dropping work item')\n self.QUEUE.task_done()\n raise\n\n except Exception:\n Log.exception('%s failed work item', self.name)\n self.QUEUE.task_done()", "response": "Initialize the queue and spawn extra worker tasks."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef schedule_saved(sender, instance, **kwargs):\n from contentstore.tasks import sync_schedule\n\n sync_schedule.delay(str(instance.id))", "response": "Called when a new object is saved in the scheduler."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\ndeactivating the celery task that ensures that this schedule is deactivated", "response": "def schedule_deleted(sender, instance, **kwargs):\n \"\"\"\n Fires off the celery task to ensure that this schedule is deactivated\n\n Arguments:\n sender {class} -- The model class, always Schedule\n instance {Schedule} --\n The instance of the schedule that we want to deactivate\n \"\"\"\n from contentstore.tasks import deactivate_schedule\n\n deactivate_schedule.delay(str(instance.scheduler_schedule_id))"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nexecutes the task inside the asyncio event loop.", "response": "async def run_task(self) -> None:\n '''Execute the task inside the asyncio event loop. Track the time it\n takes to run, and log when it starts/stops. After `INTERVAL` seconds,\n if/once the task has finished running, run it again until `stop()`\n is called.'''\n\n while self.running:\n try:\n Log.debug('executing periodic task %s', self.name)\n before = self.time()\n await self.run()\n total = self.time() - before\n Log.debug('finished periodic task %s in %.1f seconds',\n self.name, total)\n\n sleep = self.INTERVAL - total\n if sleep > 0:\n await self.sleep(sleep)\n\n except CancelledError:\n Log.debug('cancelled periodic task %s', self.name)\n raise\n\n except Exception:\n Log.exception('exception in periodic task %s', self.name)"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef group_by(keys, values=None, reduction=None, axis=0):\n g = GroupBy(keys, axis)\n if values is None:\n return g\n groups = g.split(values)\n if reduction is None:\n return g.unique, groups\n return [(key, reduction(group)) for key, group in zip(g.unique, groups)]", "response": "construct a GroupBy object on the given keys optionally performing the given reduction on the given values"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef split_iterable_as_iterable(self, values):\n values = iter(enumerate(values))\n cache = dict()\n def get_value(ti):\n try:\n return cache.pop(ti)\n except:\n while True:\n i, v = next(values)\n if i==ti:\n return v\n cache[i] = v\n s = iter(self.index.sorter)\n for c in self.count:\n yield (get_value(i) for i in itertools.islice(s, int(c)))", "response": "Splits an iterable into lists of items in the order of the keys\n ."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nsplit an iterable into key - group tuples without regard for the ordering of self. index. unique .", "response": "def split_iterable_as_unordered_iterable(self, values):\n \"\"\"Group iterable into iterables, without regard for the ordering of self.index.unique\n key-group tuples are yielded as soon as they are complete\n\n Parameters\n ----------\n values : iterable of length equal to keys\n iterable of values to be grouped\n\n Yields\n ------\n tuple of key, and a list of corresponding items in values\n\n Notes\n -----\n This approach is lazy, insofar as grouped values are close in their iterable\n \"\"\"\n from collections import defaultdict\n cache = defaultdict(list)\n count = self.count\n unique = self.unique\n key = (lambda i: unique[i]) if isinstance(unique, np.ndarray) else (lambda i: tuple(c[i] for c in unique))\n for i,v in zip(self.inverse, values):\n cache[i].append(v)\n if len(cache[i]) == count[i]:\n yield key(i), cache.pop(i)"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef split_sequence_as_iterable(self, values):\n print(self.count)\n s = iter(self.index.sorter)\n for c in self.count:\n yield (values[i] for i in itertools.islice(s, int(c)))", "response": "Splits the given values into iterables of items in the same length as the keys\n ."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nsplitting the array into a single array by means of reshaping", "response": "def split_array_as_array(self, values):\n \"\"\"Group ndarray into ndarray by means of reshaping\n\n Parameters\n ----------\n values : ndarray_like, [index.size, ...]\n\n Returns\n -------\n ndarray, [groups, group_size, ...]\n values grouped by key\n\n Raises\n ------\n AssertionError\n This operation is only possible if index.uniform==True\n \"\"\"\n if not self.index.uniform:\n raise ValueError(\"Array can only be split as array if all groups have the same size\")\n values = np.asarray(values)\n values = values[self.index.sorter]\n return values.reshape(self.groups, -1, *values.shape[1:])"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef split_array_as_list(self, values):\n values = np.asarray(values)\n values = values[self.index.sorter]\n return np.split(values, self.index.slices[1:-1], axis=0)", "response": "Group values as a list of arrays or a jagged - array\n "} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nreducing the values over identical key groups using the given ufunc operator over the first axis.", "response": "def reduce(self, values, operator=np.add, axis=0, dtype=None):\n \"\"\"Reduce the values over identical key groups, using the given ufunc\n reduction is over the first axis, which should have elements corresponding to the keys\n all other axes are treated indepenently for the sake of this reduction\n\n Parameters\n ----------\n values : ndarray, [keys, ...]\n values to perform reduction over\n operator : numpy.ufunc\n a numpy ufunc, such as np.add or np.sum\n axis : int, optional\n the axis to reduce over\n dtype : output dtype\n\n Returns\n -------\n ndarray, [groups, ...]\n values reduced by operator over the key-groups\n \"\"\"\n values = np.take(values, self.index.sorter, axis=axis)\n return operator.reduceat(values, self.index.start, axis=axis, dtype=dtype)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef sum(self, values, axis=0, dtype=None):\n values = np.asarray(values)\n return self.unique, self.reduce(values, axis=axis, dtype=dtype)", "response": "compute the sum over each group\n "} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\ncomputes the product over each group", "response": "def prod(self, values, axis=0, dtype=None):\n \"\"\"compute the product over each group\n\n Parameters\n ----------\n values : array_like, [keys, ...]\n values to multiply per group\n axis : int, optional\n alternative reduction axis for values\n dtype : output dtype\n\n Returns\n -------\n unique: ndarray, [groups]\n unique keys\n reduced : ndarray, [groups, ...]\n value array, reduced over groups\n \"\"\"\n values = np.asarray(values)\n return self.unique, self.reduce(values, axis=axis, dtype=dtype, operator=np.multiply)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef mean(self, values, axis=0, weights=None, dtype=None):\n values = np.asarray(values)\n if weights is None:\n result = self.reduce(values, axis=axis, dtype=dtype)\n shape = [1] * values.ndim\n shape[axis] = self.groups\n weights = self.count.reshape(shape)\n else:\n weights = np.asarray(weights)\n result = self.reduce(values * weights, axis=axis, dtype=dtype)\n weights = self.reduce(weights, axis=axis, dtype=dtype)\n return self.unique, result / weights", "response": "compute the mean over each group\n "} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef var(self, values, axis=0, weights=None, dtype=None):\n values = np.asarray(values)\n unique, mean = self.mean(values, axis, weights, dtype)\n err = values - mean.take(self.inverse, axis)\n\n if weights is None:\n shape = [1] * values.ndim\n shape[axis] = self.groups\n group_weights = self.count.reshape(shape)\n var = self.reduce(err ** 2, axis=axis, dtype=dtype)\n else:\n weights = np.asarray(weights)\n group_weights = self.reduce(weights, axis=axis, dtype=dtype)\n var = self.reduce(weights * err ** 2, axis=axis, dtype=dtype)\n\n return unique, var / group_weights", "response": "compute the variance over each group\n "} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nreturn the standard deviation over each group", "response": "def std(self, values, axis=0, weights=None, dtype=None):\n \"\"\"standard deviation over each group\n\n Parameters\n ----------\n values : array_like, [keys, ...]\n values to take standard deviation of per group\n axis : int, optional\n alternative reduction axis for values\n\n Returns\n -------\n unique: ndarray, [groups]\n unique keys\n reduced : ndarray, [groups, ...]\n value array, reduced over groups\n \"\"\"\n unique, var = self.var(values, axis, weights, dtype)\n return unique, np.sqrt(var)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef median(self, values, axis=0, average=True):\n mid_2 = self.index.start + self.index.stop\n hi = (mid_2 ) // 2\n lo = (mid_2 - 1) // 2\n\n #need this indirection for lex-index compatibility\n sorted_group_rank_per_key = self.index.sorted_group_rank_per_key\n\n def median1d(slc):\n #place values at correct keys; preconditions the upcoming lexsort\n slc = slc[self.index.sorter]\n #refine value sorting within each keygroup\n sorter = np.lexsort((slc, sorted_group_rank_per_key))\n slc = slc[sorter]\n return (slc[lo]+slc[hi]) / 2 if average else slc[hi]\n\n values = np.asarray(values)\n if values.ndim>1: #is trying to skip apply_along_axis somewhat premature optimization?\n values = np.apply_along_axis(median1d, axis, values)\n else:\n values = median1d(values)\n return self.unique, values", "response": "compute the median value over each group"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef mode(self, values, weights=None):\n if weights is None:\n unique, weights = npi.count((self.index.sorted_group_rank_per_key, values))\n else:\n unique, weights = npi.group_by((self.index.sorted_group_rank_per_key, values)).sum(weights)\n\n x, bin = npi.group_by(unique[0]).argmax(weights)\n return x, unique[1][bin]", "response": "compute the mode of each entry in the set"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef min(self, values, axis=0):\n values = np.asarray(values)\n return self.unique, self.reduce(values, np.minimum, axis)", "response": "return the minimum of unique entries within each group\n "} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nreturn the maximum within each group", "response": "def max(self, values, axis=0):\n \"\"\"return the maximum within each group\n\n Parameters\n ----------\n values : array_like, [keys, ...]\n values to take maximum of per group\n axis : int, optional\n alternative reduction axis for values\n\n Returns\n -------\n unique: ndarray, [groups]\n unique keys\n reduced : ndarray, [groups, ...]\n value array, reduced over groups\n \"\"\"\n values = np.asarray(values)\n return self.unique, self.reduce(values, np.maximum, axis)"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nreturn values at first occurance of its associated key", "response": "def first(self, values, axis=0):\n \"\"\"return values at first occurance of its associated key\n\n Parameters\n ----------\n values : array_like, [keys, ...]\n values to pick the first value of per group\n axis : int, optional\n alternative reduction axis for values\n\n Returns\n -------\n unique: ndarray, [groups]\n unique keys\n reduced : ndarray, [groups, ...]\n value array, reduced over groups\n \"\"\"\n values = np.asarray(values)\n return self.unique, np.take(values, self.index.sorter[self.index.start], axis)"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef last(self, values, axis=0):\n values = np.asarray(values)\n return self.unique, np.take(values, self.index.sorter[self.index.stop-1], axis)", "response": "return values at last occurance of its associated key\n "} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef any(self, values, axis=0):\n values = np.asarray(values)\n if not values.dtype == np.bool:\n values = values != 0\n return self.unique, self.reduce(values, axis=axis) > 0", "response": "compute if any item evaluates to true in each group\n "} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef all(self, values, axis=0):\n values = np.asarray(values)\n return self.unique, self.reduce(values, axis=axis, operator=np.multiply) != 0", "response": "compute if all items evaluates to true in each group\n "} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nreturning the index into values corresponding to the minimum value of the group", "response": "def argmin(self, values):\n \"\"\"return the index into values corresponding to the minimum value of the group\n\n Parameters\n ----------\n values : array_like, [keys]\n values to pick the argmin of per group\n\n Returns\n -------\n unique: ndarray, [groups]\n unique keys\n argmin : ndarray, [groups]\n index into value array, representing the argmin per group\n \"\"\"\n keys, minima = self.min(values)\n minima = minima[self.inverse]\n # select the first occurence of the minimum in each group\n index = as_index((self.inverse, values == minima))\n return keys, index.sorter[index.start[-self.groups:]]"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nreturn the index into values corresponding to the maximum value of the group", "response": "def argmax(self, values):\n \"\"\"return the index into values corresponding to the maximum value of the group\n\n Parameters\n ----------\n values : array_like, [keys]\n values to pick the argmax of per group\n\n Returns\n -------\n unique: ndarray, [groups]\n unique keys\n argmax : ndarray, [groups]\n index into value array, representing the argmax per group\n \"\"\"\n keys, maxima = self.max(values)\n maxima = maxima[self.inverse]\n # select the first occurence of the maximum in each group\n index = as_index((self.inverse, values == maxima))\n return keys, index.sorter[index.start[-self.groups:]]"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef as_index(keys, axis=semantics.axis_default, base=False, stable=True, lex_as_struct=False):\n if isinstance(keys, Index):\n if type(keys) is BaseIndex and base==False:\n keys = keys.keys #need to upcast to an indirectly sorted index type\n else:\n return keys #already done here\n if isinstance(keys, tuple):\n if lex_as_struct:\n keys = as_struct_array(*keys)\n else:\n return LexIndex(keys, stable)\n\n try:\n keys = np.asarray(keys)\n except:\n raise TypeError('Given object does not form a valid set of keys')\n if axis is None:\n keys = keys.flatten()\n if keys.ndim==1:\n if base:\n return BaseIndex(keys)\n else:\n return Index(keys, stable=stable)\n else:\n return ObjectIndex(keys, axis, stable=stable)", "response": "Converts a given set of keys to an index."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nreturns index array that maps unique values back to original space. unique [ inverse ] == keys", "response": "def inverse(self):\n \"\"\"return index array that maps unique values back to original space. unique[inverse]==keys\"\"\"\n inv = np.empty(self.size, np.int)\n inv[self.sorter] = self.sorted_group_rank_per_key\n return inv"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef rank(self):\n r = np.empty(self.size, np.int)\n r[self.sorter] = np.arange(self.size)\n return r", "response": "how high in sorted list each key is. inverse permutation of sorter"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nreturns a tuple of unique key columns", "response": "def unique(self):\n \"\"\"returns a tuple of unique key columns\"\"\"\n return tuple(\n (array_as_typed(s, k.dtype, k.shape) if k.ndim>1 else s)[self.start]\n for s, k in zip(self.sorted, self._keys))"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\npack a sequence of columns into a recarray containing the input columns as struct fields", "response": "def as_struct_array(*columns):\n \"\"\"pack a sequence of columns into a recarray\n\n Parameters\n ----------\n columns : sequence of key objects\n\n Returns\n -------\n data : recarray\n recarray containing the input columns as struct fields\n \"\"\"\n columns = [np.asarray(c) for c in columns]\n rows = len(columns[0])\n\n names = ['f'+str(i) for i in range(len(columns))]\n dtype = [(names[i], c.dtype, c.shape[1:]) for i, c in enumerate(columns)]\n data = np.empty(rows, dtype)\n for i, c in enumerate(columns):\n data[names[i]] = c\n return data"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\ncasting the given axis of an array to a void object", "response": "def axis_as_object(arr, axis=-1):\n \"\"\"cast the given axis of an array to a void object\n if the axis to be cast is contiguous, a view is returned, otherwise a copy is made\n this is useful for efficiently sorting by the content of an axis, for instance\n\n Parameters\n ----------\n arr : ndarray\n array to view as void object type\n axis : int\n axis to view as a void object type\n\n Returns\n -------\n ndarray\n array with the given axis viewed as a void object\n \"\"\"\n shape = arr.shape\n # make axis to be viewed as a void object as contiguous items\n arr = np.ascontiguousarray(np.rollaxis(arr, axis, arr.ndim))\n # number of bytes in each void object\n nbytes = arr.dtype.itemsize * shape[axis]\n # void type with the correct number of bytes\n voidtype = np.dtype((np.void, nbytes))\n # return the view as such, with the reduced shape\n return arr.view(voidtype).reshape(np.delete(shape, axis))"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\ncasts an array of void objects to a typed axis", "response": "def object_as_axis(arr, dtype, axis=-1):\n \"\"\"\n cast an array of void objects to a typed axis\n\n Parameters\n ----------\n arr : ndarray, [ndim], void\n array of type np.void\n dtype : numpy dtype object\n the output dtype to cast the input array to\n axis : int\n position to insert the newly formed axis into\n\n Returns\n -------\n ndarray, [ndim+1], dtype\n output array cast to given dtype\n \"\"\"\n # view the void objects as typed elements\n arr = arr.view(dtype).reshape(arr.shape + (-1,))\n # put the axis in the specified location\n return np.rollaxis(arr, -1, axis)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\ncounting the number of times each key occurs in the input set", "response": "def count(keys, axis=semantics.axis_default):\n \"\"\"count the number of times each key occurs in the input set\n\n Arguments\n ---------\n keys : indexable object\n\n Returns\n -------\n unique : ndarray, [groups, ...]\n unique keys\n count : ndarray, [groups], int\n the number of times each key occurs in the input set\n\n Notes\n -----\n Can be seen as numpy work-alike of collections.Counter\n Alternatively, as sparse equivalent of count_table\n \"\"\"\n index = as_index(keys, axis, base=True)\n return index.unique, index.count"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\ncount the number of times each key occurs in the input set", "response": "def count_table(*keys):\n \"\"\"count the number of times each key occurs in the input set\n\n Arguments\n ---------\n keys : tuple of indexable objects, each having the same number of items\n\n Returns\n -------\n unique : tuple of ndarray, [groups, ...]\n unique keys for each input item\n they form the axes labels of the table\n table : ndarray, [keys[0].groups, ... keys[n].groups], int\n the number of times each key-combination occurs in the input set\n\n Notes\n -----\n Equivalent to R's pivot table or pandas 'crosstab'\n Alternatively, dense equivalent of the count function\n Should we add weights option?\n Or better yet; what about general reductions over key-grids?\n \"\"\"\n indices = [as_index(k, axis=0) for k in keys]\n uniques = [i.unique for i in indices]\n inverses = [i.inverse for i in indices]\n shape = [i.groups for i in indices]\n table = np.zeros(shape, np.int)\n np.add.at(table, inverses, 1)\n return tuple(uniques), table"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nperform binning over the given axes of the keys", "response": "def binning(keys, start, end, count, axes=None):\n \"\"\"Perform binning over the given axes of the keys\n\n Parameters\n ----------\n keys : indexable or tuple of indexable\n\n Examples\n --------\n binning(np.random.rand(100), 0, 1, 10)\n \"\"\"\n if isinstance(keys, tuple):\n n_keys = len(keys)\n else:\n n_keys = 1\n\n bins = np.linspace(start, end, count+1, endpoint=True)\n idx = np.searchsorted(bins, keys)\n if axes is None:\n axes = [-1]"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nreturns the multiplicity of each key or how often it occurs in the set", "response": "def multiplicity(keys, axis=semantics.axis_default):\n \"\"\"return the multiplicity of each key, or how often it occurs in the set\n\n Parameters\n ----------\n keys : indexable object\n\n Returns\n -------\n ndarray, [keys.size], int\n the number of times each input item occurs in the set\n \"\"\"\n index = as_index(keys, axis)\n return index.count[index.inverse]"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nreturn the rank of the object in the pecking order.", "response": "def rank(keys, axis=semantics.axis_default):\n \"\"\"where each item is in the pecking order.\n\n Parameters\n ----------\n keys : indexable object\n\n Returns\n -------\n ndarray, [keys.size], int\n unique integers, ranking the sorting order\n\n Notes\n -----\n we should have that index.sorted[index.rank] == keys\n \"\"\"\n index = as_index(keys, axis)\n return index.rank"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef mode(keys, axis=semantics.axis_default, weights=None, return_indices=False):\n index = as_index(keys, axis)\n if weights is None:\n unique, weights = count(index)\n else:\n unique, weights = group_by(index).sum(weights)\n bin = np.argmax(weights)\n _mode = unique[bin] # FIXME: replace with index.take for lexindex compatibility?\n if return_indices:\n indices = index.sorter[index.start[bin]: index.stop[bin]]\n return _mode, indices\n else:\n return _mode", "response": "compute the mode or most frequent occuring key in a set\n "} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef incidence(boundary):\n return GroupBy(boundary).split(np.arange(boundary.size) // boundary.shape[1])", "response": "compute indidence info matrix for a given boundary info between simplices"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nreturn true if all keys are unique", "response": "def all_unique(keys, axis=semantics.axis_default):\n \"\"\"Returns true if all keys are unique\"\"\"\n index = as_index(keys, axis)\n return index.groups == index.size"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef any_unique(keys, axis=semantics.axis_default):\n index = as_index(keys, axis)\n return np.any(index.count == 1)", "response": "returns true if any of the keys is unique"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef all_equal(keys, axis=semantics.axis_default):\n index = as_index(keys, axis)\n return index.groups == 1", "response": "returns true of all keys are equal"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nreturns true if all keys have equal multiplicity", "response": "def is_uniform(keys, axis=semantics.axis_default):\n \"\"\"returns true if all keys have equal multiplicity\"\"\"\n index = as_index(keys, axis)\n return index.uniform"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nreturns tuples of inverse indices of the given keys", "response": "def get_inverses(self, keys):\n \"\"\"\n Returns\n -------\n Tuple of inverse indices\n \"\"\"\n return tuple([as_index(k, axis=0).inverse for k in keys])"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef unique(self, values):\n _, count = self.count()\n if not np.array_equiv(count, 1):\n raise ValueError(\"Not every entry in the table is assigned a unique value\")\n return self.sum(values)", "response": "Place each entry in a table while asserting that each entry occurs once"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\ncomputes the set of unique keys within a node - level object", "response": "def unique(keys, axis=semantics.axis_default, return_index=False, return_inverse=False, return_count=False):\n \"\"\"compute the set of unique keys\n\n Parameters\n ----------\n keys : indexable key object\n keys object to find unique keys within\n axis : int\n if keys is a multi-dimensional array, the axis to regard as the sequence of key objects\n return_index : bool\n if True, return indexes such that keys[index] == unique\n return_inverse : bool\n if True, return the indices such that unique[inverse] == keys\n return_count : bool\n if True, return the number of times each unique key occurs in the input\n\n Notes\n -----\n The kwargs are there to provide a backwards compatible interface to numpy.unique, but arguably,\n it is cleaner to call index and its properties directly, should more than unique values be desired as output\n \"\"\"\n stable = return_index or return_inverse\n index = as_index(keys, axis, base = not stable, stable = stable)\n\n ret = index.unique,\n if return_index:\n ret = ret + (index.index,)\n if return_inverse:\n ret = ret + (index.inverse,)\n if return_count:\n ret = ret + (index.count,)\n return ret[0] if len(ret) == 1 else ret"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nreturning a boolean array indicating if each element of that is contained in this.", "response": "def contains(this, that, axis=semantics.axis_default):\n \"\"\"Returns bool for each element of `that`, indicating if it is contained in `this`\n\n Parameters\n ----------\n this : indexable key sequence\n sequence of items to test against\n that : indexable key sequence\n sequence of items to test for\n\n Returns\n -------\n ndarray, [that.size], bool\n returns a bool for each element in `that`, indicating if it is contained in `this`\n\n Notes\n -----\n Reads as 'this contains that'\n Similar to 'that in this', but with different performance characteristics\n \"\"\"\n this = as_index(this, axis=axis, lex_as_struct=True, base=True)\n that = as_index(that, axis=axis, lex_as_struct=True)\n\n left = np.searchsorted(that._keys, this._keys, sorter=that.sorter, side='left')\n right = np.searchsorted(that._keys, this._keys, sorter=that.sorter, side='right')\n\n flags = np.zeros(that.size + 1, dtype=np.int)\n np.add.at(flags, left, 1)\n np.add.at(flags, right, -1)\n\n return np.cumsum(flags)[:-1].astype(np.bool)[that.rank]"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nreturning a bool for each element of this indicating if it is present in that.", "response": "def in_(this, that, axis=semantics.axis_default):\n \"\"\"Returns bool for each element of `this`, indicating if it is present in `that`\n\n Parameters\n ----------\n this : indexable key sequence\n sequence of items to test for\n that : indexable key sequence\n sequence of items to test against\n\n Returns\n -------\n ndarray, [that.size], bool\n returns a bool for each element in `this`, indicating if it is present in `that`\n\n Notes\n -----\n Reads as 'this in that'\n Similar to 'that contains this', but with different performance characteristics\n \"\"\"\n this = as_index(this, axis=axis, lex_as_struct=True, base=True)\n that = as_index(that, axis=axis, lex_as_struct=True)\n\n left = np.searchsorted(that._keys, this._keys, sorter=that.sorter, side='left')\n right = np.searchsorted(that._keys, this._keys, sorter=that.sorter, side='right')\n\n return left != right"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef indices(this, that, axis=semantics.axis_default, missing='raise'):\n this = as_index(this, axis=axis, lex_as_struct=True)\n # use this for getting this.keys and that.keys organized the same way;\n # sorting is superfluous though. make sorting a cached property?\n # should we be working with cached properties generally?\n # or we should use sorted values, if searchsorted can exploit this knowledge?\n that = as_index(that, axis=axis, base=True, lex_as_struct=True)\n\n # use raw private keys here, rather than public unpacked keys\n insertion = np.searchsorted(this._keys, that._keys, sorter=this.sorter, side='left')\n indices = np.take(this.sorter, insertion, mode='clip')\n\n if missing != 'ignore':\n invalid = this._keys[indices] != that._keys\n if missing == 'raise':\n if np.any(invalid):\n raise KeyError('Not all keys in `that` are present in `this`')\n elif missing == 'mask':\n indices = np.ma.masked_array(indices, invalid)\n else:\n indices[invalid] = missing\n return indices", "response": "Find indices such that this [ indices ] == that [ indices ]"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef remap(input, keys, values, missing='ignore', inplace=False):\n input = np.asarray(input) # FIXME: currently instances of Index are not allowed\n values = np.asarray(values)\n if missing == 'ignore':\n idx = indices(keys, input, missing='mask')\n mask = np.logical_not(idx.mask)\n idx = idx.data\n elif missing == 'raise':\n idx = indices(keys, input, missing='raise')\n mask = Ellipsis\n else:\n raise ValueError(\"'missing' should be either 'ignore' or 'raise'\")\n output = input if inplace else input.copy()\n output[mask] = values[idx[mask]]\n return output", "response": "Given an array of keys and values remaps its entries corresponding to keys to values."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nconcatenate indexable objects. Parameters ---------- sets : iterable of indexable objects Returns ------- indexable object handles both arrays and tuples of arrays", "response": "def _set_concatenate(sets):\n \"\"\"concatenate indexable objects.\n\n Parameters\n ----------\n sets : iterable of indexable objects\n\n Returns\n -------\n indexable object\n\n handles both arrays and tuples of arrays\n \"\"\"\n def con(set):\n # if not all():\n # raise ValueError('concatenated keys must have the same dtype')\n try:\n return np.concatenate([s for s in sets if len(s)])\n except ValueError:\n return set[0]\n\n if any(not isinstance(s, tuple) for s in sets):\n #assume all arrays\n return con(sets)\n else:\n #assume all tuples\n return tuple(con(s) for s in zip(*sets))"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nreturn the elements which occur n times over the sequence of sets", "response": "def _set_count(sets, n, **kwargs):\n \"\"\"return the elements which occur n times over the sequence of sets\n\n Parameters\n ----------\n sets : iterable of indexable objects\n n : int\n number of sets the element should occur in\n\n Returns\n -------\n indexable\n indexable with all elements that occured in n of the sets\n\n Notes\n -----\n used by both exclusive and intersection\n \"\"\"\n sets = _set_preprocess(sets, **kwargs)\n i = as_index(_set_concatenate(sets), axis=0, base=True)\n # FIXME : this does not work for lex-keys\n return i.unique[i.count == n]"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef union(*sets, **kwargs):\n sets = _set_preprocess(sets, **kwargs)\n return as_index( _set_concatenate(sets), axis=0, base=True).unique", "response": "all unique items which occur in any one of the sets\n Returns a new indexable object containing the union of all unique items in any of the sets\n "} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef difference(*sets, **kwargs):\n head, tail = sets[0], sets[1:]\n idx = as_index(head, **kwargs)\n lhs = idx.unique\n rhs = [intersection(idx, s, **kwargs) for s in tail]\n return exclusive(lhs, *rhs, axis=0, assume_unique=True)", "response": "returns the difference of all items in the set in the head set and all other items in the tail set"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\ngenerate a random id for div chart.", "response": "def _generate_div_id_chart(prefix=\"chart_id\", digits=8):\n \"\"\"Generate a random id for div chart.\n \"\"\"\n choices = (random.randrange(0, 52) for _ in range(digits))\n return prefix + \"\".join((string.ascii_letters[x] for x in choices))"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\ndisplaying you DataFrame with Highcharts.", "response": "def display_charts(df, chart_type=\"default\", render_to=None, **kwargs):\n \"\"\"Display you DataFrame with Highcharts.\n\n df: DataFrame\n chart_type: str\n 'default' or 'stock'\n render_to: str\n div id for plotting your data\n \"\"\"\n if chart_type not in (\"default\", \"stock\"):\n raise ValueError(\"Wrong chart_type: accept 'default' or 'stock'.\")\n chart_id = render_to if render_to is not None else _generate_div_id_chart()\n json_data = serialize(df, render_to=chart_id, chart_type=chart_type,\n **kwargs)\n content = \"\"\"
\n \"\"\"\n return display(HTML(content.format(chart_id=chart_id,\n data=json_data)))"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef _series_data_filter(data):\n data = copy.deepcopy(data)\n if \"series\" in data:\n for series in data[\"series\"]:\n series[\"data\"] = \"[...]\"\n return data", "response": "Filter the data for the series key in the list stored under the key series by \"[...]\"."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef pretty_params(data, indent=2):\n data_to_print = _series_data_filter(data)\n print(json.dumps(data_to_print, indent=indent))", "response": "Pretty print your Highcharts params into a JSON."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef get_additional_params(self, **params):\n\n # TODO: Move these params to their own vertical if needed.\n polling_params = [\n 'locationschema',\n 'carrierschema',\n 'sorttype',\n 'sortorder',\n 'originairports',\n 'destinationairports',\n 'stops',\n 'outbounddeparttime',\n 'outbounddepartstarttime',\n 'outbounddepartendtime',\n 'inbounddeparttime',\n 'inbounddepartstarttime',\n 'inbounddepartendtime',\n 'duration',\n 'includecarriers',\n 'excludecarriers'\n ]\n\n additional_params = dict(\n (key, value) for key, value in params.items()\n if key in polling_params\n )\n\n return additional_params", "response": "Get the additional params needed for polling."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef get_result(self, errors=GRACEFUL, **params):\n additional_params = self.get_additional_params(**params)\n return self.poll_session(\n self.create_session(**params),\n errors=errors,\n **additional_params\n )", "response": "Get all results by creating and polling the session."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nmaking a request to the specified service URL and returns the response.", "response": "def make_request(self, service_url, method='get', headers=None, data=None,\n callback=None, errors=GRACEFUL, **params):\n \"\"\"\n Reusable method for performing requests.\n\n :param service_url - URL to request\n :param method - request method, default is 'get'\n :param headers - request headers\n :param data - post data\n :param callback - callback to be applied to response,\n default callback will parse response as json object.\n :param errors - specifies communication errors handling mode,\n possible values are:\n * strict (default) - throw an error as soon as one\n occurred\n * graceful - ignore certain errors, e.g. EmptyResponse\n * ignore - ignore all errors and return a result in\n any case.\n NOTE that it DOES NOT mean that\n no exceptions can be raised from\n this method, it mostly ignores\n communication related errors.\n * None or empty string equals to default\n :param params - additional query parameters for request\n \"\"\"\n error_modes = (STRICT, GRACEFUL, IGNORE)\n error_mode = errors or GRACEFUL\n if error_mode.lower() not in error_modes:\n raise ValueError(\n 'Possible values for errors argument are: %s' %\n ', '.join(error_modes)\n )\n\n if callback is None:\n callback = self._default_resp_callback\n\n if 'apikey' not in service_url.lower():\n params.update({\n 'apiKey': self.api_key\n })\n\n request = getattr(requests, method.lower())\n\n log.debug('* Request URL: %s' % service_url)\n log.debug('* Request method: %s' % method)\n log.debug('* Request query params: %s' % params)\n log.debug('* Request headers: %s' % headers)\n\n r = request(service_url, headers=headers, data=data, params=params)\n try:\n r.raise_for_status()\n return callback(r)\n except Exception as e:\n return self._with_error_handling(r, e, error_mode,\n self.response_format)"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef get_markets(self, market):\n url = \"{url}/{market}\".format(url=self.MARKET_SERVICE_URL,\n market=market)\n return self.make_request(url, headers=self._headers())", "response": "Get the list of markets available for a given market"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef location_autosuggest(self, **params):\n service_url = \"{url}/{params_path}\".format(\n url=self.LOCATION_AUTOSUGGEST_URL,\n params_path=self._construct_params(\n params, self.LOCATION_AUTOSUGGEST_PARAMS)\n )\n return self.make_request(\n service_url,\n headers=self._headers(),\n **params\n )", "response": "This method returns a response object containing the location of the object."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\npoll the URL :param poll_url - URL to poll, should be returned by 'create_session' call :param initial_delay - specifies how many seconds to wait before the first poll :param delay - specifies how many seconds to wait between the polls :param tries - number of polls to perform :param errors - errors handling mode, see corresponding parameter in 'make_request' method :param params - additional query params for each poll request", "response": "def poll_session(self, poll_url, initial_delay=2, delay=1, tries=20,\n errors=GRACEFUL, **params):\n \"\"\"\n Poll the URL\n :param poll_url - URL to poll,\n should be returned by 'create_session' call\n :param initial_delay - specifies how many seconds to wait before\n the first poll\n :param delay - specifies how many seconds to wait between the polls\n :param tries - number of polls to perform\n :param errors - errors handling mode,\n see corresponding parameter in 'make_request' method\n :param params - additional query params for each poll request\n \"\"\"\n time.sleep(initial_delay)\n poll_response = None\n for n in range(tries):\n poll_response = self.make_request(\n poll_url,\n headers=self._headers(),\n errors=errors, **params\n )\n\n if self.is_poll_complete(poll_response):\n return poll_response\n else:\n time.sleep(delay)\n\n if STRICT == errors:\n raise ExceededRetries(\n \"Failed to poll within {0} tries.\".format(tries))\n else:\n return poll_response"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nconstruct params list from required_keys and opt_keys.", "response": "def _construct_params(params, required_keys, opt_keys=None):\n \"\"\"\n Construct params list in order of given keys.\n \"\"\"\n try:\n params_list = [params.pop(key) for key in required_keys]\n except KeyError as e:\n raise MissingParameter(\n 'Missing expected request parameter: %s' % e)\n if opt_keys:\n params_list.extend([params.pop(key)\n for key in opt_keys if key in params])\n return '/'.join(str(p) for p in params_list)"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef create_session(self, **params):\n return self.make_request(self.PRICING_SESSION_URL,\n method='post',\n headers=self._session_headers(),\n callback=lambda resp: resp.headers[\n 'location'],\n data=params)", "response": "Create the session for the current user"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef request_booking_details(self, poll_url, **params):\n return self.make_request(\"%s/booking\" % poll_url,\n method='put',\n headers=self._headers(),\n callback=lambda resp: resp.headers[\n 'location'],\n **params)", "response": "Request for booking details for a specific resource."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef get_cheapest_price_by_date(self, **params):\n service_url = \"{url}/{params_path}\".format(\n url=self.BROWSE_DATES_SERVICE_URL,\n params_path=self._construct_params(\n params, self._REQ_PARAMS, self._OPT_PARAMS)\n )\n\n return self.make_request(\n service_url,\n headers=self._headers(),\n **params\n )", "response": "This method returns the cheapest price for the current date."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef get_cheapest_price_by_route(self, **params):\n service_url = \"{url}/{params_path}\".format(\n url=self.BROWSE_ROUTES_SERVICE_URL,\n params_path=self._construct_params(\n params, self._REQ_PARAMS, self._OPT_PARAMS)\n\n )\n return self.make_request(\n service_url,\n headers=self._headers(),\n **params\n )", "response": "This method returns the cheapest price by route."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef get_cheapest_quotes(self, **params):\n service_url = \"{url}/{params_path}\".format(\n url=self.BROWSE_QUOTES_SERVICE_URL,\n params_path=self._construct_params(\n params, self._REQ_PARAMS, self._OPT_PARAMS)\n )\n return self.make_request(\n service_url,\n headers=self._headers(),\n **params\n )", "response": "This method returns the cheapest quotes for the current locale."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\ncreates the session for the user.", "response": "def create_session(self, **params):\n \"\"\"\n Create the session\n date format: YYYY-MM-DDThh:mm\n location: ISO code\n \"\"\"\n\n required_keys = ('market', 'currency', 'locale', 'pickupplace',\n 'dropoffplace', 'pickupdatetime', 'dropoffdatetime',\n 'driverage')\n\n service_url = \"{url}/{params_path}\".format(\n url=self.PRICING_SESSION_URL,\n params_path=self._construct_params(params, required_keys)\n )\n\n poll_path = self.make_request(service_url,\n headers=self._session_headers(),\n callback=lambda resp: resp.headers[\n 'location'],\n userip=params['userip'])\n\n return \"{url}{path}\".format(url=self.API_HOST, path=poll_path)"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nformats keyboard /command field.", "response": "def keyboard_field(value, args=None):\n \"\"\"\n Format keyboard /command field. \n \"\"\"\n qs = QueryDict(args)\n per_line = qs.get('per_line', 1)\n field = qs.get(\"field\", \"slug\")\n command = qs.get(\"command\")\n convert = lambda element: \"/\" + command + \" \" + str(getattr(element, field))\n group = lambda flat, size: [flat[i:i+size] for i in range(0, len(flat), size)]\n grouped = group(value, int(per_line))\n new_list = []\n for line in grouped:\n new_list.append([convert(e) for e in line]) \n return str(new_list).encode('utf-8')"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef get_queryset(self):\n if self.queryset is None:\n if self.model:\n return self.model._default_manager.all()\n else:\n raise ImproperlyConfigured(\n \"%(cls)s is missing a QuerySet. Define \"\n \"%(cls)s.model, %(cls)s.queryset, or override \"\n \"%(cls)s.get_queryset().\" % {\n 'cls': self.__class__.__name__\n }\n )\n return self.queryset.all()", "response": "Returns the QuerySet that will be used to look up the object."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef login_required(view_func):\n @wraps(view_func)\n def wrapper(bot, update, **kwargs):\n chat = Chat.objects.get(id=update.message.chat.id)\n if chat.is_authenticated():\n return view_func(bot, update, **kwargs)\n from telegrambot.bot_views.login import LoginBotView\n login_command_view = LoginBotView.as_command_view()\n bot_model = Bot.objects.get(token=bot.token)\n kwargs['link'] = reverse('telegrambot:auth', kwargs={'bot': bot_model.user_api.username}) \n return login_command_view(bot, update, **kwargs)\n return wrapper", "response": "Decorator for command views that checks that the chat is authenticated and sends message with link for authenticated if necessary."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef data_structure_builder(func):\n @wraps(func)\n def ds_builder_wrapper(function, *args, **kwargs):\n try:\n function = DSBuilder(function)\n except NoBuilder:\n pass\n return func(function, *args, **kwargs)\n\n return ds_builder_wrapper", "response": "Decorator to handle automatic data structure creation for pipe - utils.\n "} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef regex_condition(func):\n @wraps(func)\n def regex_condition_wrapper(condition, *args, **kwargs):\n if isinstance(condition, string_types):\n condition = maybe | partial(re.match, condition)\n return func(condition, *args, **kwargs)\n return regex_condition_wrapper", "response": "Decorator that turns a condition string instead of a function into a regex - matching function."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nsort an incoming sequence by using the given function as key.", "response": "def sort_by(function):\n \"\"\"\n Sorts an incoming sequence by using the given `function` as key.\n\n >>> range(10) > sort_by(-X)\n [9, 8, 7, 6, 5, 4, 3, 2, 1, 0]\n\n Supports automatic data-structure creation::\n\n users > sort_by([X.last_name, X.first_name])\n\n There is also a shortcut for ``sort_by(X)`` called ``sort``:\n\n >>> [4, 5, 8, -3, 0] > sort\n [-3, 0, 4, 5, 8]\n\n And (as of ``0.2.3``) a shortcut for reversing the sort:\n\n >>> 'asdfaSfa' > sort_by(X.lower()).descending\n ['s', 'S', 'f', 'f', 'd', 'a', 'a', 'a']\n \"\"\"\n f = partial(sorted, key=function)\n f.attrs = {'descending': _descending_sort_by(function)}\n return f"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\ntake the first count items from the input.", "response": "def take_first(count):\n \"\"\"\n Assumes an iterable on the input, returns an iterable with first `count`\n items from the input (or possibly less, if there isn't that many).\n\n >>> range(9000) > where(X % 100 == 0) | take_first(5) | tuple\n (0, 100, 200, 300, 400)\n\n \"\"\"\n def _take_first(iterable):\n return islice(iterable, count)\n return pipe | set_name('take_first(%s)' % count, _take_first)"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef drop_first(count):\n def _drop_first(iterable):\n g = (x for x in range(1, count + 1))\n return dropwhile(\n lambda i: unless(StopIteration, lambda: next(g))(), iterable)\n return pipe | set_name('drop_first(%s)' % count, _drop_first)", "response": "Returns an iterable with identical items\n except for the first count items."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef unless(exception_class_or_tuple, func, *args, **kwargs):\n @pipe_util\n @auto_string_formatter\n @data_structure_builder\n def construct_unless(function):\n # a wrapper so we can re-use the decorators\n def _unless(*args, **kwargs):\n try:\n return function(*args, **kwargs)\n except exception_class_or_tuple:\n pass\n return _unless\n\n name = lambda: 'unless(%s, %s)' % (exception_class_or_tuple, ', '.join(\n filter(None, (get_name(func), repr_args(*args, **kwargs)))))\n\n return set_name(name, construct_unless(func, *args, **kwargs))", "response": "A decorator that returns a new object that will not be caught while executing func."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef group_by(function):\n def _group_by(seq):\n result = {}\n for item in seq:\n result.setdefault(function(item), []).append(item)\n return dict_items(result)\n\n return _group_by", "response": "Groups input sequence by function."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef take_until(condition):\n f = partial(takewhile, pipe | condition | operator.not_)\n f.attrs = {'including': take_until_including(condition)}\n return f", "response": "take_until returns a list of nodes where condition is true"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef take_until_including(condition):\n def take_until_including_(interable):\n for i in interable:\n if not condition(i):\n yield i\n else:\n yield i\n break\n return take_until_including_", "response": "Return a generator that yields the elements of the list of items in the list that satisfy the condition."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef xpartial(func, *xargs, **xkwargs):\n any_x = any(isinstance(a, XObject) for a in xargs + tuple(xkwargs.values()))\n use = lambda x, value: (~x)(value) if isinstance(x, XObject) else x\n\n @wraps(func, assigned=filter(partial(hasattr, func), WRAPPER_ASSIGNMENTS))\n def xpartially_applied(*func_args, **func_kwargs):\n if any_x:\n if not func_args:\n raise ValueError('Function \"%s\" partially applied with an '\n 'X placeholder but called with no positional arguments.'\n % get_name(func))\n first = func_args[0]\n rest = func_args[1:]\n args = tuple(use(x, first) for x in xargs) + rest\n kwargs = dict((k, use(x, first)) for k, x in dict_items(xkwargs))\n kwargs.update(func_kwargs)\n else:\n args = xargs + func_args\n kwargs = dict(xkwargs, **func_kwargs)\n return func(*args, **kwargs)\n\n name = lambda: '%s(%s)' % (get_name(func), repr_args(*xargs, **xkwargs))\n return set_name(name, xpartially_applied)", "response": "A function that returns a new object which is a partial of the function func with the given arguments and keyword arguments."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nreturning the ecliptic longitude of the sun at the given UTC time.", "response": "def sun_ecliptic_longitude(utc_time):\n \"\"\"Ecliptic longitude of the sun at *utc_time*.\n \"\"\"\n jdate = jdays2000(utc_time) / 36525.0\n # mean anomaly, rad\n m_a = np.deg2rad(357.52910 +\n 35999.05030 * jdate -\n 0.0001559 * jdate * jdate -\n 0.00000048 * jdate * jdate * jdate)\n # mean longitude, deg\n l_0 = 280.46645 + 36000.76983 * jdate + 0.0003032 * jdate * jdate\n d_l = ((1.914600 - 0.004817 * jdate - 0.000014 * jdate * jdate) * np.sin(m_a) +\n (0.019993 - 0.000101 * jdate) * np.sin(2 * m_a) + 0.000290 * np.sin(3 * m_a))\n # true longitude, deg\n l__ = l_0 + d_l\n return np.deg2rad(l__)"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef sun_ra_dec(utc_time):\n jdate = jdays2000(utc_time) / 36525.0\n eps = np.deg2rad(23.0 + 26.0 / 60.0 + 21.448 / 3600.0 -\n (46.8150 * jdate + 0.00059 * jdate * jdate -\n 0.001813 * jdate * jdate * jdate) / 3600)\n eclon = sun_ecliptic_longitude(utc_time)\n x__ = np.cos(eclon)\n y__ = np.cos(eps) * np.sin(eclon)\n z__ = np.sin(eps) * np.sin(eclon)\n r__ = np.sqrt(1.0 - z__ * z__)\n # sun declination\n declination = np.arctan2(z__, r__)\n # right ascension\n right_ascension = 2 * np.arctan2(y__, (x__ + r__))\n return right_ascension, declination", "response": "Right ascension and declination of the sun at *utc_time*."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef get_alt_az(utc_time, lon, lat):\n lon = np.deg2rad(lon)\n lat = np.deg2rad(lat)\n\n ra_, dec = sun_ra_dec(utc_time)\n h__ = _local_hour_angle(utc_time, lon, ra_)\n return (np.arcsin(np.sin(lat) * np.sin(dec) +\n np.cos(lat) * np.cos(dec) * np.cos(h__)),\n np.arctan2(-np.sin(h__), (np.cos(lat) * np.tan(dec) -\n np.sin(lat) * np.cos(h__))))", "response": "Return sun altitude and azimuth from the given time."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef cos_zen(utc_time, lon, lat):\n lon = np.deg2rad(lon)\n lat = np.deg2rad(lat)\n\n r_a, dec = sun_ra_dec(utc_time)\n h__ = _local_hour_angle(utc_time, lon, r_a)\n return (np.sin(lat) * np.sin(dec) + np.cos(lat) * np.cos(dec) * np.cos(h__))", "response": "Cosine of the sun - zenith angle for lon and lat at the UTC time."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nreturning sun - zonal angle at a given longitude and latitude.", "response": "def sun_zenith_angle(utc_time, lon, lat):\n \"\"\"Sun-zenith angle for *lon*, *lat* at *utc_time*.\n lon,lat in degrees.\n The angle returned is given in degrees\n \"\"\"\n return np.rad2deg(np.arccos(cos_zen(utc_time, lon, lat)))"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\ncalculates the sun earth distance correction relative to 1 AU.", "response": "def sun_earth_distance_correction(utc_time):\n \"\"\"Calculate the sun earth distance correction, relative to 1 AU.\n \"\"\"\n year = 365.256363004\n # This is computed from\n # http://curious.astro.cornell.edu/question.php?number=582\n # AU = 149597870700.0\n # a = 149598261000.0\n # theta = (jdays2000(utc_time) - 2) * (2 * np.pi) / year\n # e = 0.01671123\n # r = a*(1-e*e)/(1+e * np.cos(theta))\n # corr_me = (r / AU) ** 2\n\n # from known software.\n corr = 1 - 0.0334 * np.cos(2 * np.pi * (jdays2000(utc_time) - 2) / year)\n\n return corr"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef observer_position(time, lon, lat, alt):\n\n lon = np.deg2rad(lon)\n lat = np.deg2rad(lat)\n\n theta = (gmst(time) + lon) % (2 * np.pi)\n c = 1 / np.sqrt(1 + F * (F - 2) * np.sin(lat)**2)\n sq = c * (1 - F)**2\n\n achcp = (A * c + alt) * np.cos(lat)\n x = achcp * np.cos(theta) # kilometers\n y = achcp * np.sin(theta)\n z = (A * sq + alt) * np.sin(lat)\n\n vx = -MFACTOR * y # kilometers/second\n vy = MFACTOR * x\n vz = 0\n\n return (x, y, z), (vx, vy, vz)", "response": "Calculate observer ECI position."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\ncalculates the observer look angle to a satellite.", "response": "def get_observer_look(sat_lon, sat_lat, sat_alt, utc_time, lon, lat, alt):\n \"\"\"Calculate observers look angle to a satellite.\n http://celestrak.com/columns/v02n02/\n\n utc_time: Observation time (datetime object)\n lon: Longitude of observer position on ground in degrees east\n lat: Latitude of observer position on ground in degrees north\n alt: Altitude above sea-level (geoid) of observer position on ground in km\n\n Return: (Azimuth, Elevation)\n \"\"\"\n (pos_x, pos_y, pos_z), (vel_x, vel_y, vel_z) = astronomy.observer_position(\n utc_time, sat_lon, sat_lat, sat_alt)\n\n (opos_x, opos_y, opos_z), (ovel_x, ovel_y, ovel_z) = \\\n astronomy.observer_position(utc_time, lon, lat, alt)\n\n lon = np.deg2rad(lon)\n lat = np.deg2rad(lat)\n\n theta = (astronomy.gmst(utc_time) + lon) % (2 * np.pi)\n\n rx = pos_x - opos_x\n ry = pos_y - opos_y\n rz = pos_z - opos_z\n\n sin_lat = np.sin(lat)\n cos_lat = np.cos(lat)\n sin_theta = np.sin(theta)\n cos_theta = np.cos(theta)\n\n top_s = sin_lat * cos_theta * rx + \\\n sin_lat * sin_theta * ry - cos_lat * rz\n top_e = -sin_theta * rx + cos_theta * ry\n top_z = cos_lat * cos_theta * rx + \\\n cos_lat * sin_theta * ry + sin_lat * rz\n\n az_ = np.arctan(-top_e / top_s)\n\n if has_xarray and isinstance(az_, xr.DataArray):\n az_data = az_.data\n else:\n az_data = az_\n\n if has_dask and isinstance(az_data, da.Array):\n az_data = da.where(top_s > 0, az_data + np.pi, az_data)\n az_data = da.where(az_data < 0, az_data + 2 * np.pi, az_data)\n else:\n az_data[np.where(top_s > 0)] += np.pi\n az_data[np.where(az_data < 0)] += 2 * np.pi\n\n if has_xarray and isinstance(az_, xr.DataArray):\n az_.data = az_data\n else:\n az_ = az_data\n\n rg_ = np.sqrt(rx * rx + ry * ry + rz * rz)\n el_ = np.arcsin(top_z / rg_)\n\n return np.rad2deg(az_), np.rad2deg(el_)"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\ncalculate time of last ascending node relative to the timecluster.", "response": "def get_last_an_time(self, utc_time):\n \"\"\"Calculate time of last ascending node relative to the\n specified time\n \"\"\"\n\n # Propagate backwards to ascending node\n dt = np.timedelta64(10, 'm')\n t_old = utc_time\n t_new = t_old - dt\n pos0, vel0 = self.get_position(t_old, normalize=False)\n pos1, vel1 = self.get_position(t_new, normalize=False)\n while not (pos0[2] > 0 and pos1[2] < 0):\n pos0 = pos1\n t_old = t_new\n t_new = t_old - dt\n pos1, vel1 = self.get_position(t_new, normalize=False)\n\n # Return if z within 1 km of an\n if np.abs(pos0[2]) < 1:\n return t_old\n elif np.abs(pos1[2]) < 1:\n return t_new\n\n # Bisect to z within 1 km\n while np.abs(pos1[2]) > 1:\n # pos0, vel0 = pos1, vel1\n dt = (t_old - t_new) / 2\n t_mid = t_old - dt\n pos1, vel1 = self.get_position(t_mid, normalize=False)\n if pos1[2] > 0:\n t_old = t_mid\n else:\n t_new = t_mid\n\n return t_mid"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef get_position(self, utc_time, normalize=True):\n\n kep = self._sgdp4.propagate(utc_time)\n pos, vel = kep2xyz(kep)\n\n if normalize:\n pos /= XKMPER\n vel /= XKMPER * XMNPDA / SECDAY\n\n return pos, vel", "response": "Get the cartesian position and velocity from the satellite."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef get_lonlatalt(self, utc_time):\n (pos_x, pos_y, pos_z), (vel_x, vel_y, vel_z) = self.get_position(\n utc_time, normalize=True)\n\n lon = ((np.arctan2(pos_y * XKMPER, pos_x * XKMPER) - astronomy.gmst(utc_time))\n % (2 * np.pi))\n\n lon = np.where(lon > np.pi, lon - np.pi * 2, lon)\n lon = np.where(lon <= -np.pi, lon + np.pi * 2, lon)\n\n r = np.sqrt(pos_x ** 2 + pos_y ** 2)\n lat = np.arctan2(pos_z, r)\n e2 = F * (2 - F)\n while True:\n lat2 = lat\n c = 1 / (np.sqrt(1 - e2 * (np.sin(lat2) ** 2)))\n lat = np.arctan2(pos_z + c * e2 * np.sin(lat2), r)\n if np.all(abs(lat - lat2) < 1e-10):\n break\n alt = r / np.cos(lat) - c\n alt *= A\n return np.rad2deg(lon), np.rad2deg(lat), alt", "response": "Calculate lon lat and alt of the satellite."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef get_observer_look(self, utc_time, lon, lat, alt):\n\n utc_time = dt2np(utc_time)\n (pos_x, pos_y, pos_z), (vel_x, vel_y, vel_z) = self.get_position(\n utc_time, normalize=False)\n (opos_x, opos_y, opos_z), (ovel_x, ovel_y, ovel_z) = \\\n astronomy.observer_position(utc_time, lon, lat, alt)\n\n lon = np.deg2rad(lon)\n lat = np.deg2rad(lat)\n\n theta = (astronomy.gmst(utc_time) + lon) % (2 * np.pi)\n\n rx = pos_x - opos_x\n ry = pos_y - opos_y\n rz = pos_z - opos_z\n\n sin_lat = np.sin(lat)\n cos_lat = np.cos(lat)\n sin_theta = np.sin(theta)\n cos_theta = np.cos(theta)\n\n top_s = sin_lat * cos_theta * rx + \\\n sin_lat * sin_theta * ry - cos_lat * rz\n top_e = -sin_theta * rx + cos_theta * ry\n top_z = cos_lat * cos_theta * rx + \\\n cos_lat * sin_theta * ry + sin_lat * rz\n\n az_ = np.arctan(-top_e / top_s)\n\n az_ = np.where(top_s > 0, az_ + np.pi, az_)\n az_ = np.where(az_ < 0, az_ + 2 * np.pi, az_)\n\n rg_ = np.sqrt(rx * rx + ry * ry + rz * rz)\n el_ = np.arcsin(top_z / rg_)\n\n return np.rad2deg(az_), np.rad2deg(el_)", "response": "Calculate the observer look angle to a satellite."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\ncalculate the orbit number at the specified time.", "response": "def get_orbit_number(self, utc_time, tbus_style=False):\n \"\"\"Calculate orbit number at specified time.\n Optionally use TBUS-style orbit numbering (TLE orbit number + 1)\n \"\"\"\n utc_time = np.datetime64(utc_time)\n try:\n dt = astronomy._days(utc_time - self.orbit_elements.an_time)\n orbit_period = astronomy._days(self.orbit_elements.an_period)\n except AttributeError:\n pos_epoch, vel_epoch = self.get_position(self.tle.epoch,\n normalize=False)\n if np.abs(pos_epoch[2]) > 1 or not vel_epoch[2] > 0:\n # Epoch not at ascending node\n self.orbit_elements.an_time = self.get_last_an_time(\n self.tle.epoch)\n else:\n # Epoch at ascending node (z < 1 km) and positive v_z\n self.orbit_elements.an_time = self.tle.epoch\n\n self.orbit_elements.an_period = self.orbit_elements.an_time - \\\n self.get_last_an_time(self.orbit_elements.an_time\n - np.timedelta64(10, 'm'))\n\n dt = astronomy._days(utc_time - self.orbit_elements.an_time)\n orbit_period = astronomy._days(self.orbit_elements.an_period)\n\n orbit = int(self.tle.orbit + dt / orbit_period +\n self.tle.mean_motion_derivative * dt**2 +\n self.tle.mean_motion_sec_derivative * dt**3)\n\n if tbus_style:\n orbit += 1\n return orbit"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef get_next_passes(self, utc_time, length, lon, lat, alt, tol=0.001, horizon=0):\n\n def elevation(minutes):\n \"\"\"Compute the elevation.\"\"\"\n return self.get_observer_look(utc_time +\n timedelta(\n minutes=np.float64(minutes)),\n lon, lat, alt)[1] - horizon\n\n def elevation_inv(minutes):\n \"\"\"Compute the inverse of elevation.\"\"\"\n return -elevation(minutes)\n\n def get_root(fun, start, end, tol=0.01):\n \"\"\"Root finding scheme\"\"\"\n x_0 = end\n x_1 = start\n fx_0 = fun(end)\n fx_1 = fun(start)\n if abs(fx_0) < abs(fx_1):\n fx_0, fx_1 = fx_1, fx_0\n x_0, x_1 = x_1, x_0\n\n x_n = optimize.brentq(fun, x_0, x_1)\n return x_n\n\n def get_max_parab(fun, start, end, tol=0.01):\n \"\"\"Successive parabolic interpolation.\"\"\"\n a = float(start)\n c = float(end)\n b = (a + c) / 2.0\n\n f_a = fun(a)\n f_b = fun(b)\n f_c = fun(c)\n\n x = b\n\n while True:\n x = x - 0.5 * (((b - a) ** 2 * (f_b - f_c)\n - (b - c) ** 2 * (f_b - f_a)) /\n ((b - a) * (f_b - f_c) - (b - c) * (f_b - f_a)))\n if np.isnan(x):\n return b\n if abs(b - x) <= tol:\n return x\n\n a, b, c = (a + x) / 2.0, x, (x + c) / 2.0\n f_a, f_b, f_c = fun(a), fun(b), fun(c)\n\n # every minute\n times = utc_time + np.array([timedelta(minutes=minutes)\n for minutes in range(length * 60)])\n elev = self.get_observer_look(times, lon, lat, alt)[1] - horizon\n zcs = np.where(np.diff(np.sign(elev)))[0]\n res = []\n risetime = None\n falltime = None\n for guess in zcs:\n horizon_mins = get_root(\n elevation, guess, guess + 1.0, tol=tol / 60.0)\n horizon_time = utc_time + timedelta(minutes=horizon_mins)\n if elev[guess] < 0:\n risetime = horizon_time\n risemins = horizon_mins\n falltime = None\n else:\n falltime = horizon_time\n fallmins = horizon_mins\n if risetime:\n int_start = max(0, int(np.floor(risemins)))\n int_end = min(len(elev), int(np.ceil(fallmins) + 1))\n middle = int_start + np.argmax(elev[int_start:int_end])\n highest = utc_time + \\\n timedelta(minutes=get_max_parab(\n elevation_inv,\n max(risemins, middle - 1), min(fallmins, middle + 1),\n tol=tol / 60.0\n ))\n res += [(risetime, falltime, highest)]\n risetime = None\n return res", "response": "Calculate the next passes for a given observer."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef _get_time_at_horizon(self, utc_time, obslon, obslat, **kwargs):\n warnings.warn(\"_get_time_at_horizon is replaced with get_next_passes\",\n DeprecationWarning)\n if \"precision\" in kwargs:\n precision = kwargs['precision']\n else:\n precision = timedelta(seconds=0.001)\n if \"max_iterations\" in kwargs:\n nmax_iter = kwargs[\"max_iterations\"]\n else:\n nmax_iter = 100\n\n sec_step = 0.5\n t_step = timedelta(seconds=sec_step / 2.0)\n\n # Local derivative:\n def fprime(timex):\n el0 = self.get_observer_look(timex - t_step,\n obslon, obslat, 0.0)[1]\n el1 = self.get_observer_look(timex + t_step,\n obslon, obslat, 0.0)[1]\n return el0, (abs(el1) - abs(el0)) / sec_step\n\n tx0 = utc_time - timedelta(seconds=1.0)\n tx1 = utc_time\n idx = 0\n # eps = 500.\n eps = 100.\n while abs(tx1 - tx0) > precision and idx < nmax_iter:\n tx0 = tx1\n fpr = fprime(tx0)\n # When the elevation is high the scale is high, and when\n # the elevation is low the scale is low\n # var_scale = np.abs(np.sin(fpr[0] * np.pi/180.))\n # var_scale = np.sqrt(var_scale)\n var_scale = np.abs(fpr[0])\n tx1 = tx0 - timedelta(seconds=(eps * var_scale * fpr[1]))\n idx = idx + 1\n # print idx, tx0, tx1, var_scale, fpr\n if abs(tx1 - utc_time) < precision and idx < 2:\n tx1 = tx1 + timedelta(seconds=1.0)\n\n if abs(tx1 - tx0) <= precision and idx < nmax_iter:\n return tx1\n else:\n return None", "response": "Get the time closest in time to utc_time when the satellite is at the horizon relative to the position of an observer on the base of the satellite."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef subpoint(query_point, a=A, b=B):\n x, y, z = query_point\n\n lat = geodetic_lat(query_point)\n lon = np.arctan2(y, x)\n e2_ = (a * a - b * b) / (a * a)\n n__ = a / np.sqrt(1 - e2_ * np.sin(lat)**2)\n nx_ = n__ * np.cos(lat) * np.cos(lon)\n ny_ = n__ * np.cos(lat) * np.sin(lon)\n nz_ = (1 - e2_) * n__ * np.sin(lat)\n\n return np.stack([nx_, ny_, nz_], axis=0)", "response": "Get the point on the ellipsoid under the query_point."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nrotates vector around axis by angle.", "response": "def qrotate(vector, axis, angle):\n \"\"\"Rotate *vector* around *axis* by *angle* (in radians).\n\n *vector* is a matrix of column vectors, as is *axis*.\n This function uses quaternion rotation.\n \"\"\"\n n_axis = axis / vnorm(axis)\n sin_angle = np.expand_dims(np.sin(angle / 2), 0)\n if np.ndim(n_axis) == 1:\n n_axis = np.expand_dims(n_axis, 1)\n p__ = np.dot(n_axis, sin_angle)[:, np.newaxis]\n else:\n p__ = n_axis * sin_angle\n\n q__ = Quaternion(np.cos(angle / 2), p__)\n shape = vector.shape\n return np.einsum(\"kj, ikj->ij\",\n vector.reshape((3, -1)),\n q__.rotation_matrix()[:3, :3]).reshape(shape)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef compute_pixels(orb, sgeom, times, rpy=(0.0, 0.0, 0.0)):\n if isinstance(orb, (list, tuple)):\n tle1, tle2 = orb\n orb = Orbital(\"mysatellite\", line1=tle1, line2=tle2)\n\n # get position and velocity for each time of each pixel\n pos, vel = orb.get_position(times, normalize=False)\n\n # now, get the vectors pointing to each pixel\n vectors = sgeom.vectors(pos, vel, *rpy)\n\n # compute intersection of lines (directed by vectors and passing through\n # (0, 0, 0)) and ellipsoid. Derived from:\n # http://en.wikipedia.org/wiki/Line%E2%80%93sphere_intersection\n\n # do the computation between line and ellipsoid (WGS 84)\n # NB: AAPP uses GRS 80...\n centre = -pos\n a__ = 6378.137 # km\n # b__ = 6356.75231414 # km, GRS80\n b__ = 6356.752314245 # km, WGS84\n radius = np.array([[1 / a__, 1 / a__, 1 / b__]]).T\n shape = vectors.shape\n\n xr_ = vectors.reshape([3, -1]) * radius\n cr_ = centre.reshape([3, -1]) * radius\n ldotc = np.einsum(\"ij,ij->j\", xr_, cr_)\n lsq = np.einsum(\"ij,ij->j\", xr_, xr_)\n csq = np.einsum(\"ij,ij->j\", cr_, cr_)\n\n d1_ = (ldotc - np.sqrt(ldotc ** 2 - csq * lsq + lsq)) / lsq\n\n # return the actual pixel positions\n return vectors * d1_.reshape(shape[1:]) - centre", "response": "Compute the cartesian coordinates of the pixels in instrument scan."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nnorms of a matrix of vectors stacked along the axis * dimension.", "response": "def mnorm(m, axis=None):\n \"\"\"norm of a matrix of vectors stacked along the *axis* dimension.\"\"\"\n if axis is None:\n axis = np.ndim(m) - 1\n return np.sqrt((m**2).sum(axis))"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\ngetting the unit vectors pointing to the different pixels.", "response": "def vectors(self, pos, vel, roll=0.0, pitch=0.0, yaw=0.0):\n \"\"\"Get unit vectors pointing to the different pixels.\n\n *pos* and *vel* are column vectors, or matrices of column\n vectors. Returns vectors as stacked rows.\n \"\"\"\n # TODO: yaw steering mode !\n\n # Fake nadir: This is the intersection point between the satellite\n # looking down at the centre of the ellipsoid and the surface of the\n # ellipsoid. Nadir on the other hand is the point which vertical goes\n # through the satellite...\n # nadir = -pos / vnorm(pos)\n\n nadir = subpoint(-pos)\n nadir /= vnorm(nadir)\n\n # x is along track (roll)\n x = vel / vnorm(vel)\n\n # y is cross track (pitch)\n y = np.cross(nadir, vel, 0, 0, 0)\n y /= vnorm(y)\n\n # rotate first around x\n x_rotated = qrotate(nadir, x, self.fovs[0] + roll)\n # then around y\n xy_rotated = qrotate(x_rotated, y, self.fovs[1] + pitch)\n # then around z\n return qrotate(xy_rotated, nadir, yaw)"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef avhrr(scans_nb, scan_points,\n scan_angle=55.37, frequency=1 / 6.0, apply_offset=True):\n \"\"\"Definition of the avhrr instrument.\n\n Source: NOAA KLM User's Guide, Appendix J\n http://www.ncdc.noaa.gov/oa/pod-guide/ncdc/docs/klm/html/j/app-j.htm\n \"\"\"\n # build the avhrr instrument (scan angles)\n avhrr_inst = np.vstack(((scan_points / 1023.5 - 1)\n * np.deg2rad(-scan_angle),\n np.zeros((len(scan_points),))))\n\n avhrr_inst = np.tile(\n avhrr_inst[:, np.newaxis, :], [1, np.int(scans_nb), 1])\n\n # building the corresponding times array\n # times = (np.tile(scan_points * 0.000025 + 0.0025415, [scans_nb, 1])\n # + np.expand_dims(offset, 1))\n\n times = np.tile(scan_points * 0.000025, [np.int(scans_nb), 1])\n if apply_offset:\n offset = np.arange(np.int(scans_nb)) * frequency\n times += np.expand_dims(offset, 1)\n\n return ScanGeometry(avhrr_inst, times)", "response": "Build the avhrr instrument for the given number of scans."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef avhrr_gac(scan_times, scan_points,\n scan_angle=55.37, frequency=0.5):\n \"\"\"Definition of the avhrr instrument, gac version\n\n Source: NOAA KLM User's Guide, Appendix J\n http://www.ncdc.noaa.gov/oa/pod-guide/ncdc/docs/klm/html/j/app-j.htm\n \"\"\"\n try:\n offset = np.array([(t - scan_times[0]).seconds +\n (t - scan_times[0]).microseconds / 1000000.0 for t in scan_times])\n except TypeError:\n offset = np.arange(scan_times) * frequency\n scans_nb = len(offset)\n\n avhrr_inst = np.vstack(((scan_points / 1023.5 - 1)\n * np.deg2rad(-scan_angle),\n np.zeros((len(scan_points),))))\n\n avhrr_inst = np.tile(\n avhrr_inst[:, np.newaxis, :], [1, np.int(scans_nb), 1])\n # building the corresponding times array\n times = (np.tile(scan_points * 0.000025, [scans_nb, 1])\n + np.expand_dims(offset, 1))\n return ScanGeometry(avhrr_inst, times)", "response": "Return the avhrr instrument and gac version of the avhrr archive."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\ndescribes the VIIRS instrument geometry I - band by default.", "response": "def viirs(scans_nb, scan_indices=slice(0, None),\n chn_pixels=6400, scan_lines=32, scan_step=1):\n \"\"\"Describe VIIRS instrument geometry, I-band by default.\n VIIRS scans several lines simultaneously (there are 16 detectors for each\n M-band, 32 detectors for each I-band) so the scan angles (and times) are\n two-dimensional arrays, contrary to AVHRR for example.\n\n scan_step: The increment in number of scans. E.g. if scan_step is 100 and\n the number of scans (scans_nb) is 10 then these 10 scans are\n distributed over the swath so that between each scan there are\n 99 emtpy (excluded) scans\n\n \"\"\"\n\n entire_width = np.arange(chn_pixels)\n scan_points = entire_width[scan_indices.astype('int')]\n scan_pixels = len(scan_points)\n\n ''' initial angle 55.84 deg replaced with 56.28 deg found in\n VIIRS User's Guide from NESDIS, version 1.2 (09/10/2013).\n Ref : NOAA Technical Report NESDIS 142.\n Seems to be better (not quantified)'''\n across_track = \\\n (scan_points / (chn_pixels / 2. - 0.5) - 1) * np.deg2rad(-56.28)\n y_max_angle = np.arctan2(11.87 / 2, 824.0)\n along_track = \\\n -(np.arange(scan_lines) / (scan_lines / 2. - 0.5) - 1) * \\\n y_max_angle\n scan = np.dstack((np.tile(across_track, (scan_lines, 1)).T,\n np.tile(along_track, (scan_pixels, 1))))\n npp = np.tile(scan, [scans_nb, 1]).T\n\n # from the timestamp in the filenames, a granule takes 1:25.400 to record\n # (85.4 seconds) so 1.779166667 would be the duration of 1 scanline (48\n # scans per granule) dividing the duration of a single scan by a width of\n # 6400 pixels results in 0.0002779947917 seconds for each column of 32\n # pixels in the scanline\n\n # the individual times per pixel are probably wrong, unless the scanning\n # behaves the same as for AVHRR, The VIIRS sensor rotates to allow internal\n # calibration before each scanline. This would imply that the scanline\n # always moves in the same direction. more info @\n # http://www.eoportal.org/directory/pres_NPOESSNationalPolarorbitingOperationalEnvironmentalSatelliteSystem.html\n\n SEC_EACH_SCANCOLUMN = 0.0002779947917\n sec_scan_duration = 1.779166667\n times = np.tile(scan_points * SEC_EACH_SCANCOLUMN,\n [np.int(scans_nb*scan_lines), 1])\n offset = np.repeat(np.arange(scans_nb) *\n sec_scan_duration*scan_step, scan_lines)\n times += np.expand_dims(offset, 1)\n\n # build the scan geometry object\n return ScanGeometry(npp, times)"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\ndescribes HIRS 4 instrument geometry.", "response": "def hirs4(scans_nb, edges_only=False):\n \"\"\"Describe HIRS/4 instrument geometry.\n\n See:\n - https://www.eumetsat.int/website/home/Satellites/CurrentSatellites/Metop/MetopDesign/HIRS/index.html\n - https://www1.ncdc.noaa.gov/pub/data/satellite/publications/podguides/\n N-15%20thru%20N-19/pdf/0.0%20NOAA%20KLM%20Users%20Guide.pdf\n (NOAA KLM Users Guide \u2013August 2014 Revision)\n\n Parameters:\n scans_nb | int - number of scan lines\n\n Keywords:\n * edges_only - use only edge pixels\n\n Returns:\n pyorbital.geoloc.ScanGeometry object\n\n \"\"\"\n\n scan_len = 56 # 56 samples per scan\n scan_rate = 6.4 # single scan, seconds\n scan_angle = -49.5 # swath, degrees\n sampling_interval = abs(scan_rate) / scan_len # single view, seconds\n\n if edges_only:\n scan_points = np.array([0, scan_len - 1])\n else:\n scan_points = np.arange(0, scan_len)\n\n # build the instrument (scan angles)\n samples = np.vstack(((scan_points / (scan_len * 0.5 - 0.5) - 1)\n * np.deg2rad(scan_angle),\n np.zeros((len(scan_points),))))\n samples = np.tile(samples[:, np.newaxis, :], [1, np.int(scans_nb), 1])\n\n # building the corresponding times array\n offset = np.arange(scans_nb) * scan_rate\n times = (np.tile(scan_points * sampling_interval, [np.int(scans_nb), 1])\n + np.expand_dims(offset, 1))\n\n # build the scan geometry object\n return ScanGeometry(samples, times)"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef olci(scans_nb, scan_points=None):\n\n if scan_points is None:\n scan_len = 4000 # samples per scan\n scan_points = np.arange(4000)\n else:\n scan_len = len(scan_points)\n # scan_rate = 0.044 # single scan, seconds\n scan_angle_west = 46.5 # swath, degrees\n scan_angle_east = -22.1 # swath, degrees\n # sampling_interval = 18e-3 # single view, seconds\n # build the olci instrument scan line angles\n scanline_angles = np.linspace(np.deg2rad(scan_angle_west),\n np.deg2rad(scan_angle_east), scan_len)\n inst = np.vstack((scanline_angles, np.zeros(scan_len,)))\n\n inst = np.tile(inst[:, np.newaxis, :], [1, np.int(scans_nb), 1])\n\n # building the corresponding times array\n # times = (np.tile(scan_points * 0.000025 + 0.0025415, [scans_nb, 1])\n # + np.expand_dims(offset, 1))\n\n times = np.tile(np.zeros_like(scanline_angles), [np.int(scans_nb), 1])\n # if apply_offset:\n # offset = np.arange(np.int(scans_nb)) * frequency\n # times += np.expand_dims(offset, 1)\n\n return ScanGeometry(inst, times)", "response": "Return a ScanGeometry object for the OLCI instrument."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nreading platform numbers from the PPP_CONFIG_DIR.", "response": "def read_platform_numbers(in_upper=False, num_as_int=False):\n \"\"\"Read platform numbers from $PPP_CONFIG_DIR/platforms.txt if available.\"\"\"\n out_dict = {}\n os.getenv('PPP_CONFIG_DIR', PKG_CONFIG_DIR)\n platform_file = None\n if 'PPP_CONFIG_DIR' in os.environ:\n platform_file = os.path.join(os.environ['PPP_CONFIG_DIR'], 'platforms.txt')\n if not platform_file or not os.path.isfile(platform_file):\n platform_file = os.path.join(PKG_CONFIG_DIR, 'platforms.txt')\n\n try:\n fid = open(platform_file, 'r')\n except IOError:\n LOGGER.error(\"Platform file %s not found.\", platform_file)\n return out_dict\n for row in fid:\n # skip comment lines\n if not row.startswith('#'):\n parts = row.split()\n if len(parts) < 2:\n continue\n if in_upper:\n parts[0] = parts[0].upper()\n if num_as_int:\n parts[1] = int(parts[1])\n out_dict[parts[0]] = parts[1]\n fid.close()\n\n return out_dict"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nread TLE from tle_file line1 and line2.", "response": "def read(platform, tle_file=None, line1=None, line2=None):\n \"\"\"Read TLE for `platform` from `tle_file`\n\n File is read from `line1` to `line2`, from the newest file provided in the\n TLES pattern, or from internet if none is provided.\n \"\"\"\n return Tle(platform, tle_file=tle_file, line1=line1, line2=line2)"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nfetches TLE from internet and save it to destination.", "response": "def fetch(destination):\n \"\"\"Fetch TLE from internet and save it to `destination`.\"\"\"\n with io.open(destination, mode=\"w\", encoding=\"utf-8\") as dest:\n for url in TLE_URLS:\n response = urlopen(url)\n dest.write(response.read().decode(\"utf-8\"))"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nperforming the checksum for the current TLE.", "response": "def _checksum(self):\n \"\"\"Performs the checksum for the current TLE.\"\"\"\n for line in [self._line1, self._line2]:\n check = 0\n for char in line[:-1]:\n if char.isdigit():\n check += int(char)\n if char == \"-\":\n check += 1\n\n if (check % 10) != int(line[-1]):\n raise ChecksumError(self._platform + \" \" + line)"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nparse the TLE data from the Aleph file.", "response": "def _parse_tle(self):\n \"\"\"Parse values from TLE data.\"\"\"\n\n def _read_tle_decimal(rep):\n \"\"\"Convert *rep* to decimal value.\"\"\"\n if rep[0] in [\"-\", \" \", \"+\"]:\n digits = rep[1:-2].strip()\n val = rep[0] + \".\" + digits + \"e\" + rep[-2:]\n else:\n digits = rep[:-2].strip()\n val = \".\" + digits + \"e\" + rep[-2:]\n\n return float(val)\n\n self.satnumber = self._line1[2:7]\n self.classification = self._line1[7]\n self.id_launch_year = self._line1[9:11]\n self.id_launch_number = self._line1[11:14]\n self.id_launch_piece = self._line1[14:17]\n self.epoch_year = self._line1[18:20]\n self.epoch_day = float(self._line1[20:32])\n self.epoch = \\\n np.datetime64(datetime.datetime.strptime(self.epoch_year, \"%y\") +\n datetime.timedelta(days=self.epoch_day - 1), 'us')\n self.mean_motion_derivative = float(self._line1[33:43])\n self.mean_motion_sec_derivative = _read_tle_decimal(self._line1[44:52])\n self.bstar = _read_tle_decimal(self._line1[53:61])\n try:\n self.ephemeris_type = int(self._line1[62])\n except ValueError:\n self.ephemeris_type = 0\n self.element_number = int(self._line1[64:68])\n\n self.inclination = float(self._line2[8:16])\n self.right_ascension = float(self._line2[17:25])\n self.excentricity = int(self._line2[26:33]) * 10 ** -7\n self.arg_perigee = float(self._line2[34:42])\n self.mean_anomaly = float(self._line2[43:51])\n self.mean_motion = float(self._line2[52:63])\n self.orbit = int(self._line2[63:68])"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef fix_chrome_mac_platform(platform):\n ver = platform.split('OS X ')[1]\n build_range = range(*MACOSX_CHROME_BUILD_RANGE[ver])\n build = choice(build_range)\n mac_ver = ver.replace('.', '_') + '_' + str(build)\n return 'Macintosh; Intel Mac OS X %s' % mac_ver", "response": "Fix Chrome on Mac OS X 10. 11. 6 platform."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nbuilds random system components for given os_id and navigator_id.", "response": "def build_system_components(device_type, os_id, navigator_id):\n \"\"\"\n For given os_id build random platform and oscpu\n components\n\n Returns dict {platform_version, platform, ua_platform, oscpu}\n\n platform_version is OS name used in different places\n ua_platform goes to navigator.platform\n platform is used in building navigator.userAgent\n oscpu goes to navigator.oscpu\n \"\"\"\n\n if os_id == 'win':\n platform_version = choice(OS_PLATFORM['win'])\n cpu = choice(OS_CPU['win'])\n if cpu:\n platform = '%s; %s' % (platform_version, cpu)\n else:\n platform = platform_version\n res = {\n 'platform_version': platform_version,\n 'platform': platform,\n 'ua_platform': platform,\n 'oscpu': platform,\n }\n elif os_id == 'linux':\n cpu = choice(OS_CPU['linux'])\n platform_version = choice(OS_PLATFORM['linux'])\n platform = '%s %s' % (platform_version, cpu)\n res = {\n 'platform_version': platform_version,\n 'platform': platform,\n 'ua_platform': platform,\n 'oscpu': 'Linux %s' % cpu,\n }\n elif os_id == 'mac':\n cpu = choice(OS_CPU['mac'])\n platform_version = choice(OS_PLATFORM['mac'])\n platform = platform_version\n if navigator_id == 'chrome':\n platform = fix_chrome_mac_platform(platform)\n res = {\n 'platform_version': platform_version,\n 'platform': 'MacIntel',\n 'ua_platform': platform,\n 'oscpu': 'Intel Mac OS X %s' % platform.split(' ')[-1],\n }\n elif os_id == 'android':\n assert navigator_id in ('firefox', 'chrome')\n assert device_type in ('smartphone', 'tablet')\n platform_version = choice(OS_PLATFORM['android'])\n if navigator_id == 'firefox':\n if device_type == 'smartphone':\n ua_platform = '%s; Mobile' % platform_version\n elif device_type == 'tablet':\n ua_platform = '%s; Tablet' % platform_version\n elif navigator_id == 'chrome':\n device_id = choice(SMARTPHONE_DEV_IDS)\n ua_platform = 'Linux; %s; %s' % (platform_version, device_id)\n oscpu = 'Linux %s' % choice(OS_CPU['android'])\n res = {\n 'platform_version': platform_version,\n 'ua_platform': ua_platform,\n 'platform': oscpu,\n 'oscpu': oscpu,\n }\n return res"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nbuild app components for given navigator_id", "response": "def build_app_components(os_id, navigator_id):\n \"\"\"\n For given navigator_id build app features\n\n Returns dict {name, product_sub, vendor, build_version, build_id}\n \"\"\"\n\n if navigator_id == 'firefox':\n build_version, build_id = get_firefox_build()\n if os_id in ('win', 'linux', 'mac'):\n geckotrail = '20100101'\n else:\n geckotrail = build_version\n res = {\n 'name': 'Netscape',\n 'product_sub': '20100101',\n 'vendor': '',\n 'build_version': build_version,\n 'build_id': build_id,\n 'geckotrail': geckotrail,\n }\n elif navigator_id == 'chrome':\n res = {\n 'name': 'Netscape',\n 'product_sub': '20030107',\n 'vendor': 'Google Inc.',\n 'build_version': get_chrome_build(),\n 'build_id': None,\n }\n elif navigator_id == 'ie':\n num_ver, build_version, trident_version = get_ie_build()\n if num_ver >= 11:\n app_name = 'Netscape'\n else:\n app_name = 'Microsoft Internet Explorer'\n res = {\n 'name': app_name,\n 'product_sub': None,\n 'vendor': '',\n 'build_version': build_version,\n 'build_id': None,\n 'trident_version': trident_version,\n }\n return res"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef get_option_choices(opt_name, opt_value, default_value, all_choices):\n\n choices = []\n if isinstance(opt_value, six.string_types):\n choices = [opt_value]\n elif isinstance(opt_value, (list, tuple)):\n choices = list(opt_value)\n elif opt_value is None:\n choices = default_value\n else:\n raise InvalidOption('Option %s has invalid'\n ' value: %s' % (opt_name, opt_value))\n if 'all' in choices:\n choices = all_choices\n for item in choices:\n if item not in all_choices:\n raise InvalidOption('Choices of option %s contains invalid'\n ' item: %s' % (opt_name, item))\n return choices", "response": "Generates possible choices for the option opt_name opt_value with default value as default_value as all_choices"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nselects one random pair of config ids from the given os and navigator filters.", "response": "def pick_config_ids(device_type, os, navigator):\n \"\"\"\n Select one random pair (device_type, os_id, navigator_id) from\n all possible combinations matching the given os and\n navigator filters.\n\n :param os: allowed os(es)\n :type os: string or list/tuple or None\n :param navigator: allowed browser engine(s)\n :type navigator: string or list/tuple or None\n :param device_type: limit possible oses by device type\n :type device_type: list/tuple or None, possible values:\n \"desktop\", \"smartphone\", \"tablet\", \"all\"\n \"\"\"\n\n\n if os is None:\n default_dev_types = ['desktop']\n else:\n default_dev_types = list(DEVICE_TYPE_OS.keys())\n dev_type_choices = get_option_choices(\n 'device_type', device_type, default_dev_types,\n list(DEVICE_TYPE_OS.keys())\n )\n os_choices = get_option_choices('os', os, list(OS_NAVIGATOR.keys()),\n list(OS_NAVIGATOR.keys()))\n nav_choices = get_option_choices('navigator', navigator,\n list(NAVIGATOR_OS.keys()),\n list(NAVIGATOR_OS.keys()))\n\n variants = []\n for dev, os, nav in product(dev_type_choices, os_choices,\n nav_choices):\n\n if (os in DEVICE_TYPE_OS[dev]\n and nav in DEVICE_TYPE_NAVIGATOR[dev]\n and nav in OS_NAVIGATOR[os]):\n variants.append((dev, os, nav))\n if not variants:\n raise InvalidOption('Options device_type, os and navigator'\n ' conflicts with each other')\n device_type, os_id, navigator_id = choice(variants)\n\n assert os_id in OS_PLATFORM\n assert navigator_id in NAVIGATOR_OS\n assert device_type in DEVICE_TYPE_OS\n\n return device_type, os_id, navigator_id"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\ngenerate a web navigator s config.", "response": "def generate_navigator(os=None, navigator=None, platform=None,\n device_type=None):\n \"\"\"\n Generates web navigator's config\n\n :param os: limit list of oses for generation\n :type os: string or list/tuple or None\n :param navigator: limit list of browser engines for generation\n :type navigator: string or list/tuple or None\n :param device_type: limit possible oses by device type\n :type device_type: list/tuple or None, possible values:\n \"desktop\", \"smartphone\", \"tablet\", \"all\"\n\n :return: User-Agent config\n :rtype: dict with keys (os, name, platform, oscpu, build_version,\n build_id, app_version, app_name, app_code_name,\n product, product_sub, vendor, vendor_sub,\n user_agent)\n :raises InvalidOption: if could not generate user-agent for\n any combination of allowed platforms and navigators\n :raise InvalidOption: if any of passed options is invalid\n \"\"\"\n\n if platform is not None:\n os = platform\n warn('The `platform` option is deprecated.'\n ' Use `os` option instead.', stacklevel=3)\n device_type, os_id, navigator_id = (\n pick_config_ids(device_type, os, navigator)\n )\n system = build_system_components(\n device_type, os_id, navigator_id)\n app = build_app_components(os_id, navigator_id)\n ua_template = choose_ua_template(\n device_type, navigator_id, app)\n user_agent = ua_template.format(system=system, app=app)\n app_version = build_navigator_app_version(\n os_id, navigator_id, system['platform_version'], user_agent)\n return {\n # ids\n 'os_id': os_id,\n 'navigator_id': navigator_id,\n # system components\n 'platform': system['platform'],\n 'oscpu': system['oscpu'],\n # app components\n 'build_version': app['build_version'],\n 'build_id': app['build_id'],\n 'app_version': app_version,\n 'app_name': app['name'],\n 'app_code_name': 'Mozilla',\n 'product': 'Gecko',\n 'product_sub': app['product_sub'],\n 'vendor': app['vendor'],\n 'vendor_sub': '',\n # compiled user agent\n 'user_agent': user_agent,\n }"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\ngenerates HTTP User - Agent header for the current language.", "response": "def generate_user_agent(os=None, navigator=None, platform=None,\n device_type=None):\n \"\"\"\n Generates HTTP User-Agent header\n\n :param os: limit list of os for generation\n :type os: string or list/tuple or None\n :param navigator: limit list of browser engines for generation\n :type navigator: string or list/tuple or None\n :param device_type: limit possible oses by device type\n :type device_type: list/tuple or None, possible values:\n \"desktop\", \"smartphone\", \"tablet\", \"all\"\n :return: User-Agent string\n :rtype: string\n :raises InvalidOption: if could not generate user-agent for\n any combination of allowed oses and navigators\n :raise InvalidOption: if any of passed options is invalid\n \"\"\"\n return generate_navigator(os=os, navigator=navigator,\n platform=platform,\n device_type=device_type)['user_agent']"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef generate_navigator_js(os=None, navigator=None, platform=None,\n device_type=None):\n \"\"\"\n Generates web navigator's config with keys corresponding\n to keys of `windows.navigator` JavaScript object.\n\n :param os: limit list of oses for generation\n :type os: string or list/tuple or None\n :param navigator: limit list of browser engines for generation\n :type navigator: string or list/tuple or None\n :param device_type: limit possible oses by device type\n :type device_type: list/tuple or None, possible values:\n \"desktop\", \"smartphone\", \"tablet\", \"all\"\n :return: User-Agent config\n :rtype: dict with keys (TODO)\n :raises InvalidOption: if could not generate user-agent for\n any combination of allowed oses and navigators\n :raise InvalidOption: if any of passed options is invalid\n \"\"\"\n\n config = generate_navigator(os=os, navigator=navigator,\n platform=platform,\n device_type=device_type)\n return {\n 'appCodeName': config['app_code_name'],\n 'appName': config['app_name'],\n 'appVersion': config['app_version'],\n 'platform': config['platform'],\n 'userAgent': config['user_agent'],\n 'oscpu': config['oscpu'],\n 'product': config['product'],\n 'productSub': config['product_sub'],\n 'vendor': config['vendor'],\n 'vendorSub': config['vendor_sub'],\n 'buildID': config['build_id'],\n }", "response": "Generates a web navigator s config with keys corresponding\n to keys of Windows. navigator JavaScript object."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nchecking if there is more data left on the pipe_out", "response": "def more_data(pipe_out):\n \"\"\"Check if there is more data left on the pipe\n\n :param pipe_out: The os pipe_out\n :rtype: bool\n \"\"\"\n r, _, _ = select.select([pipe_out], [], [], 0)\n return bool(r)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nread data on a pipe and return it as a unicode string", "response": "def read_pipe(pipe_out):\n \"\"\"Read data on a pipe\n\n Used to capture stdout data produced by libiperf\n\n :param pipe_out: The os pipe_out\n :rtype: unicode string\n \"\"\"\n out = b''\n while more_data(pipe_out):\n out += os.read(pipe_out, 1024)\n\n return out.decode('utf-8')"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef role(self):\n try:\n self._role = c_char(\n self.lib.iperf_get_test_role(self._test)\n ).value.decode('utf-8')\n except TypeError:\n self._role = c_char(\n chr(self.lib.iperf_get_test_role(self._test))\n ).value.decode('utf-8')\n return self._role", "response": "The iperf3 instance role"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef json_output(self):\n enabled = self.lib.iperf_get_test_json_output(self._test)\n\n if enabled:\n self._json_output = True\n else:\n self._json_output = False\n\n return self._json_output", "response": "Toggles json output of libiperf\n\n Turning this off will output the iperf3 instance results to stdout and stderr"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef verbose(self):\n enabled = self.lib.iperf_get_verbose(self._test)\n\n if enabled:\n self._verbose = True\n else:\n self._verbose = False\n\n return self._verbose", "response": "Toggles verbose output for the iperf3 instance"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef iperf_version(self):\n # TODO: Is there a better way to get the const char than allocating 30?\n VersionType = c_char * 30\n return VersionType.in_dll(self.lib, \"version\").value.decode('utf-8')", "response": "Returns the version of the libiperf library"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nreturning an error string from libiperf", "response": "def _error_to_string(self, error_id):\n \"\"\"Returns an error string from libiperf\n\n :param error_id: The error_id produced by libiperf\n :rtype: string\n \"\"\"\n strerror = self.lib.iperf_strerror\n strerror.restype = c_char_p\n return strerror(error_id).decode('utf-8')"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nreturn the server hostname to connect to.", "response": "def server_hostname(self):\n \"\"\"The server hostname to connect to.\n\n Accepts DNS entries or IP addresses.\n\n :rtype: string\n \"\"\"\n result = c_char_p(\n self.lib.iperf_get_test_server_hostname(self._test)\n ).value\n if result:\n self._server_hostname = result.decode('utf-8')\n else:\n self._server_hostname = None\n return self._server_hostname"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef protocol(self):\n proto_id = self.lib.iperf_get_test_protocol_id(self._test)\n\n if proto_id == SOCK_STREAM:\n self._protocol = 'tcp'\n elif proto_id == SOCK_DGRAM:\n self._protocol = 'udp'\n\n return self._protocol", "response": "The iperf3 instance protocol"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef omit(self):\n self._omit = self.lib.iperf_get_test_omit(self._test)\n return self._omit", "response": "The test startup duration to omit in seconds."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\ntargets bandwidth in bits / sec", "response": "def bandwidth(self):\n \"\"\"Target bandwidth in bits/sec\"\"\"\n self._bandwidth = self.lib.iperf_get_test_rate(self._test)\n return self._bandwidth"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef reverse(self):\n enabled = self.lib.iperf_get_test_reverse(self._test)\n\n if enabled:\n self._reverse = True\n else:\n self._reverse = False\n\n return self._reverse", "response": "Toggles direction of test in the order of the test"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nruns the current test client.", "response": "def run(self):\n \"\"\"Run the current test client.\n\n :rtype: instance of :class:`TestResult`\n \"\"\"\n if self.json_output:\n output_to_pipe(self._pipe_in) # Disable stdout\n error = self.lib.iperf_run_client(self._test)\n\n if not self.iperf_version.startswith('iperf 3.1'):\n data = read_pipe(self._pipe_out)\n if data.startswith('Control connection'):\n data = '{' + data.split('{', 1)[1]\n else:\n data = c_char_p(\n self.lib.iperf_get_test_json_output_string(self._test)\n ).value\n if data:\n data = data.decode('utf-8')\n\n output_to_screen(self._stdout_fd, self._stderr_fd) # enable stdout\n\n if not data or error:\n data = '{\"error\": \"%s\"}' % self._error_to_string(self._errno)\n\n return TestResult(data)"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nrun the iperf3 server instance.", "response": "def run(self):\n \"\"\"Run the iperf3 server instance.\n\n :rtype: instance of :class:`TestResult`\n \"\"\"\n\n def _run_in_thread(self, data_queue):\n \"\"\"Runs the iperf_run_server\n\n :param data_queue: thread-safe queue\n \"\"\"\n output_to_pipe(self._pipe_in) # disable stdout\n error = self.lib.iperf_run_server(self._test)\n output_to_screen(self._stdout_fd, self._stderr_fd) # enable stdout\n\n # TODO json_output_string not available on earlier iperf3 builds\n # have to build in a version check using self.iperf_version\n # The following line should work on later versions:\n # data = c_char_p(\n # self.lib.iperf_get_test_json_output_string(self._test)\n # ).value\n data = read_pipe(self._pipe_out)\n\n if not data or error:\n data = '{\"error\": \"%s\"}' % self._error_to_string(self._errno)\n\n self.lib.iperf_reset_test(self._test)\n data_queue.put(data)\n\n if self.json_output:\n data_queue = Queue()\n\n t = threading.Thread(\n target=_run_in_thread, args=[self, data_queue]\n )\n t.daemon = True\n\n t.start()\n while t.is_alive():\n t.join(.1)\n\n return TestResult(data_queue.get())\n else:\n # setting json_output to False will output test to screen only\n self.lib.iperf_run_server(self._test)\n self.lib.iperf_reset_test(self._test)\n\n return None"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef subplots(nrows=1, ncols=1, sharex=False, sharey=False, squeeze=True,\n subplot_kw=None, hemisphere='lower', projection='equal_area',\n **fig_kw):\n \"\"\"\n Identical to matplotlib.pyplot.subplots, except that this will default to\n producing equal-area stereonet axes.\n\n This prevents constantly doing:\n\n >>> fig, ax = plt.subplot(subplot_kw=dict(projection='stereonet'))\n\n or\n\n >>> fig = plt.figure()\n >>> ax = fig.add_subplot(111, projection='stereonet')\n\n Using this function also avoids having ``mplstereonet`` continually appear\n to be an unused import when one of the above methods are used.\n\n Parameters\n -----------\n nrows : int\n Number of rows of the subplot grid. Defaults to 1.\n\n ncols : int\n Number of columns of the subplot grid. Defaults to 1.\n\n hemisphere : string\n Currently this has no effect. When upper hemisphere and dual\n hemisphere plots are implemented, this will control which hemisphere\n is displayed.\n\n projection : string\n The projection for the axes. Defaults to 'equal_area'--an equal-area\n (a.k.a. \"Schmidtt\") stereonet. May also be 'equal_angle' for an\n equal-angle (a.k.a. \"Wulff\") stereonet or any other valid matplotlib\n projection (e.g. 'polar' or 'rectilinear' for a \"normal\" axes).\n\n The following parameters are identical to matplotlib.pyplot.subplots:\n\n sharex : string or bool\n If *True*, the X axis will be shared amongst all subplots. If\n *True* and you have multiple rows, the x tick labels on all but\n the last row of plots will have visible set to *False*\n If a string must be one of \"row\", \"col\", \"all\", or \"none\".\n \"all\" has the same effect as *True*, \"none\" has the same effect\n as *False*.\n If \"row\", each subplot row will share a X axis.\n If \"col\", each subplot column will share a X axis and the x tick\n labels on all but the last row will have visible set to *False*.\n\n sharey : string or bool\n If *True*, the Y axis will be shared amongst all subplots. If\n *True* and you have multiple columns, the y tick labels on all but\n the first column of plots will have visible set to *False*\n If a string must be one of \"row\", \"col\", \"all\", or \"none\".\n \"all\" has the same effect as *True*, \"none\" has the same effect\n as *False*.\n If \"row\", each subplot row will share a Y axis.\n If \"col\", each subplot column will share a Y axis and the y tick\n labels on all but the last row will have visible set to *False*.\n\n *squeeze* : bool\n If *True*, extra dimensions are squeezed out from the\n returned axis object:\n\n - if only one subplot is constructed (nrows=ncols=1), the\n resulting single Axis object is returned as a scalar.\n\n - for Nx1 or 1xN subplots, the returned object is a 1-d numpy\n object array of Axis objects are returned as numpy 1-d\n arrays.\n\n - for NxM subplots with N>1 and M>1 are returned as a 2d\n array.\n\n If *False*, no squeezing at all is done: the returned axis\n object is always a 2-d array contaning Axis instances, even if it\n ends up being 1x1.\n\n *subplot_kw* : dict\n Dict with keywords passed to the\n :meth:`~matplotlib.figure.Figure.add_subplot` call used to\n create each subplots.\n\n *fig_kw* : dict\n Dict with keywords passed to the :func:`figure` call. Note that all\n keywords not recognized above will be automatically included here.\n\n Returns\n --------\n fig, ax : tuple\n\n - *fig* is the :class:`matplotlib.figure.Figure` object\n\n - *ax* can be either a single axis object or an array of axis\n objects if more than one supblot was created. The dimensions\n of the resulting array can be controlled with the squeeze\n keyword, see above.\n \"\"\"\n import matplotlib.pyplot as plt\n if projection in ['equal_area', 'equal_angle']:\n projection += '_stereonet'\n if subplot_kw == None:\n subplot_kw = {}\n subplot_kw['projection'] = projection\n return plt.subplots(nrows, ncols, sharex=sharex, sharey=sharey,\n squeeze=squeeze, subplot_kw=subplot_kw, **fig_kw)", "response": "Create a set of subplots for the specified number of rows and columns."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef fault_and_striae_plot(ax, strikes, dips, rakes):\n # Plot the planes\n lines = ax.plane(strikes, dips, 'k-', lw=0.5)\n\n # Calculate the position of the rake of the lineations, but don't plot yet\n x, y = mplstereonet.rake(strikes, dips, rakes)\n\n # Calculate the direction the arrows should point\n # These are all normal faults, so the arrows point away from the center\n # For thrusts, it would just be u, v = -x/mag, -y/mag\n mag = np.hypot(x, y)\n u, v = x / mag, y / mag\n\n # Plot the arrows at the rake locations...\n arrows = ax.quiver(x, y, u, v, width=1, headwidth=4, units='dots')\n return lines, arrows", "response": "Makes a fault - and - striae plot for normal faults\n with the given strikes dips and rakes."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef tangent_lineation_plot(ax, strikes, dips, rakes):\n # Calculate the position of the rake of the lineations, but don't plot yet\n rake_x, rake_y = mplstereonet.rake(strikes, dips, rakes)\n\n # Calculate the direction the arrows should point\n # These are all normal faults, so the arrows point away from the center\n # Because we're plotting at the pole location, however, we need to flip this\n # from what we plotted with the \"ball of string\" plot.\n mag = np.hypot(rake_x, rake_y)\n u, v = -rake_x / mag, -rake_y / mag\n\n # Calculate the position of the poles\n pole_x, pole_y = mplstereonet.pole(strikes, dips)\n\n # Plot the arrows centered on the pole locations...\n arrows = ax.quiver(pole_x, pole_y, u, v, width=1, headwidth=4, units='dots',\n pivot='middle')\n return arrows", "response": "Makes a tangent lineation plot for normal faults with the given strikes dips and rakes."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef load():\n # Get the data file relative to this file's location...\n datadir = os.path.dirname(__file__)\n filename = os.path.join(datadir, 'angelier_data.txt')\n\n data = []\n with open(filename, 'r') as infile:\n for line in infile:\n # Skip comments\n if line.startswith('#'):\n continue\n\n # First column: strike, second: dip, third: rake.\n strike, dip, rake = line.strip().split()\n\n if rake[-1].isalpha():\n # If there's a directional letter on the rake column, parse it\n # normally.\n strike, dip, rake = mplstereonet.parse_rake(strike, dip, rake)\n else:\n # Otherwise, it's actually an azimuthal measurement of the\n # slickenslide directions, so we need to convert it to a rake.\n strike, dip = mplstereonet.parse_strike_dip(strike, dip)\n azimuth = float(rake)\n rake = mplstereonet.azimuth2rake(strike, dip, azimuth)\n\n data.append([strike, dip, rake])\n\n # Separate the columns back out\n strike, dip, rake = zip(*data)\n return strike, dip, rake", "response": "Read data from a text file on disk."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nreturn a regular grid of point density estimates for the given linear orientation measurements.", "response": "def density_grid(*args, **kwargs):\n \"\"\"\n Estimates point density of the given linear orientation measurements\n (Interpreted as poles, lines, rakes, or \"raw\" longitudes and latitudes\n based on the `measurement` keyword argument.). Returns a regular (in\n lat-long space) grid of density estimates over a hemispherical surface.\n\n Parameters\n ----------\n\n *args : 2 or 3 sequences of measurements\n By default, this will be expected to be ``strike`` & ``dip``, both\n array-like sequences representing poles to planes. (Rake measurements\n require three parameters, thus the variable number of arguments.) The\n ``measurement`` kwarg controls how these arguments are interpreted.\n\n measurement : string, optional\n Controls how the input arguments are interpreted. Defaults to\n ``\"poles\"``. May be one of the following:\n\n ``\"poles\"`` : strikes, dips\n Arguments are assumed to be sequences of strikes and dips of\n planes. Poles to these planes are used for contouring.\n ``\"lines\"`` : plunges, bearings\n Arguments are assumed to be sequences of plunges and bearings\n of linear features.\n ``\"rakes\"`` : strikes, dips, rakes\n Arguments are assumed to be sequences of strikes, dips, and\n rakes along the plane.\n ``\"radians\"`` : lon, lat\n Arguments are assumed to be \"raw\" longitudes and latitudes in\n the stereonet's underlying coordinate system.\n\n method : string, optional\n The method of density estimation to use. Defaults to\n ``\"exponential_kamb\"``. May be one of the following:\n\n ``\"exponential_kamb\"`` : Kamb with exponential smoothing\n A modified Kamb method using exponential smoothing [1]_. Units are\n in numbers of standard deviations by which the density estimate\n differs from uniform.\n ``\"linear_kamb\"`` : Kamb with linear smoothing\n A modified Kamb method using linear smoothing [1]_. Units are in\n numbers of standard deviations by which the density estimate\n differs from uniform.\n ``\"kamb\"`` : Kamb with no smoothing\n Kamb's method [2]_ with no smoothing. Units are in numbers of\n standard deviations by which the density estimate differs from\n uniform.\n ``\"schmidt\"`` : 1% counts\n The traditional \"Schmidt\" (a.k.a. 1%) method. Counts points within\n a counting circle comprising 1% of the total area of the\n hemisphere. Does not take into account sample size. Units are in\n points per 1% area.\n\n sigma : int or float, optional\n The number of standard deviations defining the expected number of\n standard deviations by which a random sample from a uniform\n distribution of points would be expected to vary from being evenly\n distributed across the hemisphere. This controls the size of the\n counting circle, and therefore the degree of smoothing. Higher sigmas\n will lead to more smoothing of the resulting density distribution. This\n parameter only applies to Kamb-based methods. Defaults to 3.\n\n gridsize : int or 2-item tuple of ints, optional\n The size of the grid that the density is estimated on. If a single int\n is given, it is interpreted as an NxN grid. If a tuple of ints is given\n it is interpreted as (nrows, ncols). Defaults to 100.\n\n weights : array-like, optional\n The relative weight to be applied to each input measurement. The array\n will be normalized to sum to 1, so absolute value of the weights do not\n affect the result. Defaults to None.\n\n Returns\n -------\n xi, yi, zi : 2D arrays\n The longitude, latitude and density values of the regularly gridded\n density estimates. Longitude and latitude are in radians.\n\n See Also\n ---------\n mplstereonet.StereonetAxes.density_contourf\n mplstereonet.StereonetAxes.density_contour\n\n References\n ----------\n .. [1] Vollmer, 1995. C Program for Automatic Contouring of Spherical\n Orientation Data Using a Modified Kamb Method. Computers &\n Geosciences, Vol. 21, No. 1, pp. 31--49.\n\n .. [2] Kamb, 1959. Ice Petrofabric Observations from Blue Glacier,\n Washington, in Relation to Theory and Experiment. Journal of\n Geophysical Research, Vol. 64, No. 11, pp. 1891--1909.\n \"\"\"\n def do_nothing(x, y):\n return x, y\n measurement = kwargs.get('measurement', 'poles')\n gridsize = kwargs.get('gridsize', 100)\n weights = kwargs.get('weights', None)\n try:\n gridsize = int(gridsize)\n gridsize = (gridsize, gridsize)\n except TypeError:\n pass\n\n func = {'poles':stereonet_math.pole,\n 'lines':stereonet_math.line,\n 'rakes':stereonet_math.rake,\n 'radians':do_nothing}[measurement]\n lon, lat = func(*args)\n\n method = kwargs.get('method', 'exponential_kamb')\n sigma = kwargs.get('sigma', 3)\n func = {'linear_kamb':_linear_inverse_kamb,\n 'square_kamb':_square_inverse_kamb,\n 'schmidt':_schmidt_count,\n 'kamb':_kamb_count,\n 'exponential_kamb':_exponential_kamb,\n }[method]\n lon, lat, z = _count_points(lon, lat, func, sigma, gridsize, weights)\n return lon, lat, z"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _exponential_kamb(cos_dist, sigma=3):\n n = float(cos_dist.size)\n f = 2 * (1.0 + n / sigma**2)\n count = np.exp(f * (cos_dist - 1))\n units = np.sqrt(n * (f/2.0 - 1) / f**2)\n return count, units", "response": "Kernel function from Vollmer for exponential smoothing."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nkerneling function from Vollmer for linear smoothing.", "response": "def _linear_inverse_kamb(cos_dist, sigma=3):\n \"\"\"Kernel function from Vollmer for linear smoothing.\"\"\"\n n = float(cos_dist.size)\n radius = _kamb_radius(n, sigma)\n f = 2 / (1 - radius)\n cos_dist = cos_dist[cos_dist >= radius]\n count = (f * (cos_dist - radius))\n return count, _kamb_units(n, radius)"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef _kamb_count(cos_dist, sigma=3):\n n = float(cos_dist.size)\n dist = _kamb_radius(n, sigma)\n count = (cos_dist >= dist).astype(float)\n return count, _kamb_units(n, dist)", "response": "Original Kamb kernel function ( raw count within radius )."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef _schmidt_count(cos_dist, sigma=None):\n radius = 0.01\n count = ((1 - cos_dist) <= radius).astype(float)\n # To offset the count.sum() - 0.5 required for the kamb methods...\n count = 0.5 / count.size + count\n return count, (cos_dist.size * radius)", "response": "Schmidt counting kernel function."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _get_affine_transform(self):\n # How big is the projected globe?\n # In the case of a stereonet, it's actually constant.\n xscale = yscale = self._scale\n # Create an affine transform to stretch the projection from 0-1\n return Affine2D() \\\n .rotate(np.radians(self.rotation)) \\\n .scale(0.5 / xscale, 0.5 / yscale) \\\n .translate(0.5, 0.5)", "response": "The affine portion of the base transform. This is called by _set_lim_and_transforms."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef _set_lim_and_transforms(self):\n # Most of the transforms are set up correctly by LambertAxes\n LambertAxes._set_lim_and_transforms(self)\n\n # Transform for latitude ticks. These are typically unused, but just\n # in case we need them...\n yaxis_stretch = Affine2D().scale(4 * self.horizon, 1.0)\n yaxis_stretch = yaxis_stretch.translate(-self.horizon, 0.0)\n\n # These are identical to LambertAxes._set_lim_and_transforms, but we\n # need to update things to reflect the new \"yaxis_stretch\"\n yaxis_space = Affine2D().scale(1.0, 1.1)\n self._yaxis_transform = \\\n yaxis_stretch + \\\n self.transData\n yaxis_text_base = \\\n yaxis_stretch + \\\n self.transProjection + \\\n (yaxis_space + \\\n self.transAffine + \\\n self.transAxes)\n self._yaxis_text1_transform = \\\n yaxis_text_base + \\\n Affine2D().translate(-8.0, 0.0)\n self._yaxis_text2_transform = \\\n yaxis_text_base + \\\n Affine2D().translate(8.0, 0.0)", "response": "Setup the key transforms for the axes."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef set_longitude_grid(self, degrees):\n number = (360.0 / degrees) + 1\n locs = np.linspace(-np.pi, np.pi, number, True)[1:]\n locs[-1] -= 0.01 # Workaround for \"back\" gridlines showing.\n self.xaxis.set_major_locator(FixedLocator(locs))\n self._logitude_degrees = degrees\n self.xaxis.set_major_formatter(self.ThetaFormatter(degrees))", "response": "Set the number of degrees between each longitude gridline."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nset the rotation of the stereonet in degrees clockwise from North.", "response": "def set_rotation(self, rotation):\n \"\"\"Set the rotation of the stereonet in degrees clockwise from North.\"\"\"\n self._rotation = np.radians(rotation)\n self._polar.set_theta_offset(self._rotation + np.pi / 2.0)\n self.transData.invalidate()\n self.transAxes.invalidate()\n self._set_lim_and_transforms()"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef format_coord(self, x, y):\n p, b = stereonet_math.geographic2plunge_bearing(x, y)\n s, d = stereonet_math.geographic2pole(x, y)\n pb = u'P/B={:0.0f}\\u00b0/{:03.0f}\\u00b0'.format(p[0], b[0])\n sd = u'S/D={:03.0f}\\u00b0/{:0.0f}\\u00b0'.format(s[0], d[0])\n return u'{}, {}'.format(pb, sd)", "response": "Format displayed coordinates during mouseover of axes."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef grid(self, b=None, which='major', axis='both', kind='arbitrary',\n center=None, **kwargs):\n \"\"\"\n Usage is identical to a normal axes grid except for the ``kind`` and\n ``center`` kwargs. ``kind=\"polar\"`` will add a polar overlay.\n\n The ``center`` and ``kind`` arguments allow you to add a grid from a\n differently-centered stereonet. This is useful for making \"polar\n stereonets\" that still use the same coordinate system as a standard\n stereonet. (i.e. a plane/line/whatever will have the same\n representation on both, but the grid is displayed differently.)\n\n To display a polar grid on a stereonet, use ``kind=\"polar\"``.\n\n It is also often useful to display a grid relative to an arbitrary\n measurement (e.g. a lineation axis). In that case, use the\n ``lon_center`` and ``lat_center`` arguments. Note that these are in\n radians in \"stereonet coordinates\". Therefore, you'll often want to\n use one of the functions in ``stereonet_math`` to convert a\n line/plane/rake into the longitude and latitude you'd input here. For\n example: ``add_overlay(center=stereonet_math.line(plunge, bearing))``.\n\n If no parameters are specified, this is equivalent to turning on the\n standard grid.\n \"\"\"\n grid_on = self._gridOn\n Axes.grid(self, False)\n\n if kind == 'polar':\n center = 0, 0\n\n if self._overlay_axes is not None:\n self._overlay_axes.remove()\n self._overlay_axes = None\n\n if not b and b is not None:\n return\n\n if b is None:\n if grid_on:\n return\n\n if center is None or np.allclose(center, (np.pi/2, 0)):\n return Axes.grid(self, b, which, axis, **kwargs)\n\n self._add_overlay(center)\n self._overlay_axes.grid(True, which, axis, **kwargs)\n self._gridOn = True", "response": "This is a simple grid of the components of the current object."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef _add_overlay(self, center):\n plunge, bearing = stereonet_math.geographic2plunge_bearing(*center)\n lon0, lat0 = center\n fig = self.get_figure()\n self._overlay_axes = fig.add_axes(self.get_position(True),\n frameon=False, projection=self.name,\n center_longitude=0,\n center_latitude=np.radians(plunge),\n label='overlay',\n rotation=bearing)\n self._overlay_axes._polar.remove()\n self._overlay_axes.format_coord = self._overlay_format_coord\n self._overlay_axes.grid(True)", "response": "Adds an overlay to the current figure."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nsetting the azimuthal tick locations.", "response": "def set_azimuth_ticks(self, angles, labels=None, frac=None, **kwargs):\n \"\"\"\n Sets the azimuthal tick locations (Note: tick lines are not currently\n drawn or supported.).\n\n Parameters\n ----------\n angles : sequence of numbers\n The tick locations in degrees.\n labels : sequence of strings\n The tick label at each location. Defaults to a formatted version\n of the specified angles.\n frac : number\n The radial location of the tick labels. 1.0 is the along the edge,\n 1.1 would be outside, and 0.9 would be inside.\n **kwargs\n Additional parameters are text properties for the labels.\n \"\"\"\n return self._polar.set_thetagrids(angles, labels, frac, **kwargs)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef set_azimuth_ticklabels(self, labels, fontdict=None, **kwargs):\n return self._polar.set_xticklabels(labels, fontdict, **kwargs)", "response": "Sets the labels for the azimuthal ticks."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nplot lines representing planes on the axes.", "response": "def plane(self, strike, dip, *args, **kwargs):\n \"\"\"\n Plot lines representing planes on the axes. Additional arguments and\n keyword arguments are passed on to `ax.plot`.\n\n Parameters\n ----------\n strike, dip : number or sequences of numbers\n The strike and dip of the plane(s) in degrees. The dip direction is\n defined by the strike following the \"right-hand rule\".\n segments : int, optional\n The number of vertices to use for the line. Defaults to 100.\n **kwargs\n Additional parameters are passed on to `plot`.\n\n Returns\n -------\n A sequence of Line2D artists representing the lines specified by\n `strike` and `dip`.\n \"\"\"\n segments = kwargs.pop('segments', 100)\n center = self._center_latitude, self._center_longitude\n lon, lat = stereonet_math.plane(strike, dip, segments, center)\n return self.plot(lon, lat, *args, **kwargs)"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nplotting points representing poles to planes on the axes.", "response": "def pole(self, strike, dip, *args, **kwargs):\n \"\"\"\n Plot points representing poles to planes on the axes. Additional\n arguments and keyword arguments are passed on to `ax.plot`.\n\n Parameters\n ----------\n strike, dip : numbers or sequences of numbers\n The strike and dip of the plane(s) in degrees. The dip direction is\n defined by the strike following the \"right-hand rule\".\n **kwargs\n Additional parameters are passed on to `plot`.\n\n Returns\n -------\n A sequence of Line2D artists representing the point(s) specified by\n `strike` and `dip`.\n \"\"\"\n lon, lat = stereonet_math.pole(strike, dip)\n args, kwargs = self._point_plot_defaults(args, kwargs)\n return self.plot(lon, lat, *args, **kwargs)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef rake(self, strike, dip, rake_angle, *args, **kwargs):\n lon, lat = stereonet_math.rake(strike, dip, rake_angle)\n args, kwargs = self._point_plot_defaults(args, kwargs)\n return self.plot(lon, lat, *args, **kwargs)", "response": "Plot points along planes on the axes."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef line(self, plunge, bearing, *args, **kwargs):\n lon, lat = stereonet_math.line(plunge, bearing)\n args, kwargs = self._point_plot_defaults(args, kwargs)\n return self.plot([lon], [lat], *args, **kwargs)", "response": "Plot points representing linear features on the axes."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef _point_plot_defaults(self, args, kwargs):\n if args:\n return args, kwargs\n\n if 'ls' not in kwargs and 'linestyle' not in kwargs:\n kwargs['linestyle'] = 'none'\n if 'marker' not in kwargs:\n kwargs['marker'] = 'o'\n return args, kwargs", "response": "This function is used to set the default values for the point plot."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nunifying defaults and common functionality of density_contour and .", "response": "def _contour_helper(self, args, kwargs):\n \"\"\"Unify defaults and common functionality of ``density_contour`` and\n ``density_contourf``.\"\"\"\n contour_kwargs = {}\n contour_kwargs['measurement'] = kwargs.pop('measurement', 'poles')\n contour_kwargs['method'] = kwargs.pop('method', 'exponential_kamb')\n contour_kwargs['sigma'] = kwargs.pop('sigma', 3)\n contour_kwargs['gridsize'] = kwargs.pop('gridsize', 100)\n contour_kwargs['weights'] = kwargs.pop('weights', None)\n lon, lat, totals = contouring.density_grid(*args, **contour_kwargs)\n return lon, lat, totals, kwargs"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nestimates point density of the given linear orientation measurements (Interpreted as poles, lines, rakes, or \"raw\" longitudes and latitudes based on the `measurement` keyword argument.) and plots contour lines of the resulting density distribution. Parameters ---------- *args : A variable number of sequences of measurements. By default, this will be expected to be ``strike`` & ``dip``, both array-like sequences representing poles to planes. (Rake measurements require three parameters, thus the variable number of arguments.) The ``measurement`` kwarg controls how these arguments are interpreted. measurement : string, optional Controls how the input arguments are interpreted. Defaults to ``\"poles\"``. May be one of the following: ``\"poles\"`` : strikes, dips Arguments are assumed to be sequences of strikes and dips of planes. Poles to these planes are used for contouring. ``\"lines\"`` : plunges, bearings Arguments are assumed to be sequences of plunges and bearings of linear features. ``\"rakes\"`` : strikes, dips, rakes Arguments are assumed to be sequences of strikes, dips, and rakes along the plane. ``\"radians\"`` : lon, lat Arguments are assumed to be \"raw\" longitudes and latitudes in the stereonet's underlying coordinate system. method : string, optional The method of density estimation to use. Defaults to ``\"exponential_kamb\"``. May be one of the following: ``\"exponential_kamb\"`` : Kamb with exponential smoothing A modified Kamb method using exponential smoothing [1]_. Units are in numbers of standard deviations by which the density estimate differs from uniform. ``\"linear_kamb\"`` : Kamb with linear smoothing A modified Kamb method using linear smoothing [1]_. Units are in numbers of standard deviations by which the density estimate differs from uniform. ``\"kamb\"`` : Kamb with no smoothing Kamb's method [2]_ with no smoothing. Units are in numbers of standard deviations by which the density estimate differs from uniform. ``\"schmidt\"`` : 1% counts The traditional \"Schmidt\" (a.k.a. 1%) method. Counts points within a counting circle comprising 1% of the total area of the hemisphere. Does not take into account sample size. Units are in points per 1% area. sigma : int or float, optional The number of standard deviations defining the expected number of standard deviations by which a random sample from a uniform distribution of points would be expected to vary from being evenly distributed across the hemisphere. This controls the size of the counting circle, and therefore the degree of smoothing. Higher sigmas will lead to more smoothing of the resulting density distribution. This parameter only applies to Kamb-based methods. Defaults to 3. gridsize : int or 2-item tuple of ints, optional The size of the grid that the density is estimated on. If a single int is given, it is interpreted as an NxN grid. If a tuple of ints is given it is interpreted as (nrows, ncols). Defaults to 100. weights : array-like, optional The relative weight to be applied to each input measurement. The array will be normalized to sum to 1, so absolute value of the weights do not affect the result. Defaults to None. **kwargs Additional keyword arguments are passed on to matplotlib's `contour` function. Returns ------- A matplotlib ContourSet. See Also -------- mplstereonet.density_grid mplstereonet.StereonetAxes.density_contourf matplotlib.pyplot.contour matplotlib.pyplot.clabel Examples -------- Plot density contours of poles to the specified planes using a modified Kamb method with exponential smoothing [1]_. >>> strikes, dips = [120, 315, 86], [22, 85, 31] >>> ax.density_contour(strikes, dips) Plot density contours of a set of linear orientation measurements. >>> plunges, bearings = [-10, 20, -30], [120, 315, 86] >>> ax.density_contour(plunges, bearings, measurement='lines') Plot density contours of a set of rake measurements. >>> strikes, dips, rakes = [120, 315, 86], [22, 85, 31], [-5, 20, 9] >>> ax.density_contour(strikes, dips, rakes, measurement='rakes') Plot density contours of a set of \"raw\" longitudes and latitudes. >>> lon, lat = np.radians([-40, 30, -85]), np.radians([21, -59, 45]) >>> ax.density_contour(lon, lat, measurement='radians') Plot density contours of poles to planes using a Kamb method [2]_ with the density estimated on a 10x10 grid (in long-lat space) >>> strikes, dips = [120, 315, 86], [22, 85, 31] >>> ax.density_contour(strikes, dips, method='kamb', gridsize=10) Plot density contours of poles to planes with contours at [1,2,3] standard deviations. >>> strikes, dips = [120, 315, 86], [22, 85, 31] >>> ax.density_contour(strikes, dips, levels=[1,2,3]) References ---------- .. [1] Vollmer, 1995. C Program for Automatic Contouring of Spherical Orientation Data Using a Modified Kamb Method. Computers & Geosciences, Vol. 21, No. 1, pp. 31--49. .. [2] Kamb, 1959. Ice Petrofabric Observations from Blue Glacier, Washington, in Relation to Theory and Experiment. Journal of Geophysical Research, Vol. 64, No. 11, pp. 1891--1909.", "response": "def density_contour(self, *args, **kwargs):\n \"\"\"\n Estimates point density of the given linear orientation measurements\n (Interpreted as poles, lines, rakes, or \"raw\" longitudes and latitudes\n based on the `measurement` keyword argument.) and plots contour lines of\n the resulting density distribution.\n\n Parameters\n ----------\n *args : A variable number of sequences of measurements.\n By default, this will be expected to be ``strike`` & ``dip``, both\n array-like sequences representing poles to planes. (Rake\n measurements require three parameters, thus the variable number of\n arguments.) The ``measurement`` kwarg controls how these arguments\n are interpreted.\n\n measurement : string, optional\n Controls how the input arguments are interpreted. Defaults to\n ``\"poles\"``. May be one of the following:\n\n ``\"poles\"`` : strikes, dips\n Arguments are assumed to be sequences of strikes and dips\n of planes. Poles to these planes are used for contouring.\n ``\"lines\"`` : plunges, bearings\n Arguments are assumed to be sequences of plunges and\n bearings of linear features.\n ``\"rakes\"`` : strikes, dips, rakes\n Arguments are assumed to be sequences of strikes, dips, and\n rakes along the plane.\n ``\"radians\"`` : lon, lat\n Arguments are assumed to be \"raw\" longitudes and latitudes\n in the stereonet's underlying coordinate system.\n\n method : string, optional\n The method of density estimation to use. Defaults to\n ``\"exponential_kamb\"``. May be one of the following:\n\n ``\"exponential_kamb\"`` : Kamb with exponential smoothing\n A modified Kamb method using exponential smoothing [1]_. Units\n are in numbers of standard deviations by which the density\n estimate differs from uniform.\n ``\"linear_kamb\"`` : Kamb with linear smoothing\n A modified Kamb method using linear smoothing [1]_. Units are\n in numbers of standard deviations by which the density estimate\n differs from uniform.\n ``\"kamb\"`` : Kamb with no smoothing\n Kamb's method [2]_ with no smoothing. Units are in numbers of\n standard deviations by which the density estimate differs from\n uniform.\n ``\"schmidt\"`` : 1% counts\n The traditional \"Schmidt\" (a.k.a. 1%) method. Counts points\n within a counting circle comprising 1% of the total area of the\n hemisphere. Does not take into account sample size. Units are\n in points per 1% area.\n\n sigma : int or float, optional\n The number of standard deviations defining the expected number of\n standard deviations by which a random sample from a uniform\n distribution of points would be expected to vary from being evenly\n distributed across the hemisphere. This controls the size of the\n counting circle, and therefore the degree of smoothing. Higher\n sigmas will lead to more smoothing of the resulting density\n distribution. This parameter only applies to Kamb-based methods.\n Defaults to 3.\n\n gridsize : int or 2-item tuple of ints, optional\n The size of the grid that the density is estimated on. If a single\n int is given, it is interpreted as an NxN grid. If a tuple of ints\n is given it is interpreted as (nrows, ncols). Defaults to 100.\n\n weights : array-like, optional\n The relative weight to be applied to each input measurement. The\n array will be normalized to sum to 1, so absolute value of the\n weights do not affect the result. Defaults to None.\n\n **kwargs\n Additional keyword arguments are passed on to matplotlib's\n `contour` function.\n\n Returns\n -------\n A matplotlib ContourSet.\n\n See Also\n --------\n mplstereonet.density_grid\n mplstereonet.StereonetAxes.density_contourf\n matplotlib.pyplot.contour\n matplotlib.pyplot.clabel\n\n Examples\n --------\n Plot density contours of poles to the specified planes using a\n modified Kamb method with exponential smoothing [1]_.\n\n >>> strikes, dips = [120, 315, 86], [22, 85, 31]\n >>> ax.density_contour(strikes, dips)\n\n Plot density contours of a set of linear orientation measurements.\n\n >>> plunges, bearings = [-10, 20, -30], [120, 315, 86]\n >>> ax.density_contour(plunges, bearings, measurement='lines')\n\n Plot density contours of a set of rake measurements.\n\n >>> strikes, dips, rakes = [120, 315, 86], [22, 85, 31], [-5, 20, 9]\n >>> ax.density_contour(strikes, dips, rakes, measurement='rakes')\n\n Plot density contours of a set of \"raw\" longitudes and latitudes.\n\n >>> lon, lat = np.radians([-40, 30, -85]), np.radians([21, -59, 45])\n >>> ax.density_contour(lon, lat, measurement='radians')\n\n\n Plot density contours of poles to planes using a Kamb method [2]_\n with the density estimated on a 10x10 grid (in long-lat space)\n\n >>> strikes, dips = [120, 315, 86], [22, 85, 31]\n >>> ax.density_contour(strikes, dips, method='kamb', gridsize=10)\n\n Plot density contours of poles to planes with contours at [1,2,3]\n standard deviations.\n\n >>> strikes, dips = [120, 315, 86], [22, 85, 31]\n >>> ax.density_contour(strikes, dips, levels=[1,2,3])\n\n References\n ----------\n .. [1] Vollmer, 1995. C Program for Automatic Contouring of Spherical\n Orientation Data Using a Modified Kamb Method. Computers &\n Geosciences, Vol. 21, No. 1, pp. 31--49.\n\n .. [2] Kamb, 1959. Ice Petrofabric Observations from Blue Glacier,\n Washington, in Relation to Theory and Experiment. Journal of\n Geophysical Research, Vol. 64, No. 11, pp. 1891--1909.\n \"\"\"\n lon, lat, totals, kwargs = self._contour_helper(args, kwargs)\n return self.contour(lon, lat, totals, **kwargs)"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nestimating point density of the given linear orientation measurements (Interpreted as poles, lines, rakes, or \"raw\" longitudes and latitudes based on the `measurement` keyword argument.) and plots filled contours of the resulting density distribution. Parameters ---------- *args : A variable number of sequences of measurements. By default, this will be expected to be ``strike`` & ``dip``, both array-like sequences representing poles to planes. (Rake measurements require three parameters, thus the variable number of arguments.) The ``measurement`` kwarg controls how these arguments are interpreted. measurement : string, optional Controls how the input arguments are interpreted. Defaults to ``\"poles\"``. May be one of the following: ``\"poles\"`` : strikes, dips Arguments are assumed to be sequences of strikes and dips of planes. Poles to these planes are used for contouring. ``\"lines\"`` : plunges, bearings Arguments are assumed to be sequences of plunges and bearings of linear features. ``\"rakes\"`` : strikes, dips, rakes Arguments are assumed to be sequences of strikes, dips, and rakes along the plane. ``\"radians\"`` : lon, lat Arguments are assumed to be \"raw\" longitudes and latitudes in the stereonet's underlying coordinate system. method : string, optional The method of density estimation to use. Defaults to ``\"exponential_kamb\"``. May be one of the following: ``\"exponential_kamb\"`` : Kamb with exponential smoothing A modified Kamb method using exponential smoothing [1]_. Units are in numbers of standard deviations by which the density estimate differs from uniform. ``\"linear_kamb\"`` : Kamb with linear smoothing A modified Kamb method using linear smoothing [1]_. Units are in numbers of standard deviations by which the density estimate differs from uniform. ``\"kamb\"`` : Kamb with no smoothing Kamb's method [2]_ with no smoothing. Units are in numbers of standard deviations by which the density estimate differs from uniform. ``\"schmidt\"`` : 1% counts The traditional \"Schmidt\" (a.k.a. 1%) method. Counts points within a counting circle comprising 1% of the total area of the hemisphere. Does not take into account sample size. Units are in points per 1% area. sigma : int or float, optional The number of standard deviations defining the expected number of standard deviations by which a random sample from a uniform distribution of points would be expected to vary from being evenly distributed across the hemisphere. This controls the size of the counting circle, and therefore the degree of smoothing. Higher sigmas will lead to more smoothing of the resulting density distribution. This parameter only applies to Kamb-based methods. Defaults to 3. gridsize : int or 2-item tuple of ints, optional The size of the grid that the density is estimated on. If a single int is given, it is interpreted as an NxN grid. If a tuple of ints is given it is interpreted as (nrows, ncols). Defaults to 100. weights : array-like, optional The relative weight to be applied to each input measurement. The array will be normalized to sum to 1, so absolute value of the weights do not affect the result. Defaults to None. **kwargs Additional keyword arguments are passed on to matplotlib's `contourf` function. Returns ------- A matplotlib `QuadContourSet`. See Also -------- mplstereonet.density_grid mplstereonet.StereonetAxes.density_contour matplotlib.pyplot.contourf matplotlib.pyplot.clabel Examples -------- Plot filled density contours of poles to the specified planes using a modified Kamb method with exponential smoothing [1]_. >>> strikes, dips = [120, 315, 86], [22, 85, 31] >>> ax.density_contourf(strikes, dips) Plot filled density contours of a set of linear orientation measurements. >>> plunges, bearings = [-10, 20, -30], [120, 315, 86] >>> ax.density_contourf(plunges, bearings, measurement='lines') Plot filled density contours of a set of rake measurements. >>> strikes, dips, rakes = [120, 315, 86], [22, 85, 31], [-5, 20, 9] >>> ax.density_contourf(strikes, dips, rakes, measurement='rakes') Plot filled density contours of a set of \"raw\" longitudes and latitudes. >>> lon, lat = np.radians([-40, 30, -85]), np.radians([21, -59, 45]) >>> ax.density_contourf(lon, lat, measurement='radians') Plot filled density contours of poles to planes using a Kamb method [2]_ with the density estimated on a 10x10 grid (in long-lat space) >>> strikes, dips = [120, 315, 86], [22, 85, 31] >>> ax.density_contourf(strikes, dips, method='kamb', gridsize=10) Plot filled density contours of poles to planes with contours at [1,2,3] standard deviations. >>> strikes, dips = [120, 315, 86], [22, 85, 31] >>> ax.density_contourf(strikes, dips, levels=[1,2,3]) References ---------- .. [1] Vollmer, 1995. C Program for Automatic Contouring of Spherical Orientation Data Using a Modified Kamb Method. Computers & Geosciences, Vol. 21, No. 1, pp. 31--49. .. [2] Kamb, 1959. Ice Petrofabric Observations from Blue Glacier, Washington, in Relation to Theory and Experiment. Journal of Geophysical Research, Vol. 64, No. 11, pp. 1891--1909.", "response": "def density_contourf(self, *args, **kwargs):\n \"\"\"\n Estimates point density of the given linear orientation measurements\n (Interpreted as poles, lines, rakes, or \"raw\" longitudes and latitudes\n based on the `measurement` keyword argument.) and plots filled contours\n of the resulting density distribution.\n\n Parameters\n ----------\n *args : A variable number of sequences of measurements.\n By default, this will be expected to be ``strike`` & ``dip``, both\n array-like sequences representing poles to planes. (Rake\n measurements require three parameters, thus the variable number of\n arguments.) The ``measurement`` kwarg controls how these arguments\n are interpreted.\n\n measurement : string, optional\n Controls how the input arguments are interpreted. Defaults to\n ``\"poles\"``. May be one of the following:\n\n ``\"poles\"`` : strikes, dips\n Arguments are assumed to be sequences of strikes and dips\n of planes. Poles to these planes are used for contouring.\n ``\"lines\"`` : plunges, bearings\n Arguments are assumed to be sequences of plunges and\n bearings of linear features.\n ``\"rakes\"`` : strikes, dips, rakes\n Arguments are assumed to be sequences of strikes, dips, and\n rakes along the plane.\n ``\"radians\"`` : lon, lat\n Arguments are assumed to be \"raw\" longitudes and latitudes\n in the stereonet's underlying coordinate system.\n\n method : string, optional\n The method of density estimation to use. Defaults to\n ``\"exponential_kamb\"``. May be one of the following:\n\n ``\"exponential_kamb\"`` : Kamb with exponential smoothing\n A modified Kamb method using exponential smoothing [1]_. Units\n are in numbers of standard deviations by which the density\n estimate differs from uniform.\n ``\"linear_kamb\"`` : Kamb with linear smoothing\n A modified Kamb method using linear smoothing [1]_. Units are\n in numbers of standard deviations by which the density estimate\n differs from uniform.\n ``\"kamb\"`` : Kamb with no smoothing\n Kamb's method [2]_ with no smoothing. Units are in numbers of\n standard deviations by which the density estimate differs from\n uniform.\n ``\"schmidt\"`` : 1% counts\n The traditional \"Schmidt\" (a.k.a. 1%) method. Counts points\n within a counting circle comprising 1% of the total area of the\n hemisphere. Does not take into account sample size. Units are\n in points per 1% area.\n\n sigma : int or float, optional\n The number of standard deviations defining the expected number of\n standard deviations by which a random sample from a uniform\n distribution of points would be expected to vary from being evenly\n distributed across the hemisphere. This controls the size of the\n counting circle, and therefore the degree of smoothing. Higher\n sigmas will lead to more smoothing of the resulting density\n distribution. This parameter only applies to Kamb-based methods.\n Defaults to 3.\n\n gridsize : int or 2-item tuple of ints, optional\n The size of the grid that the density is estimated on. If a single\n int is given, it is interpreted as an NxN grid. If a tuple of ints\n is given it is interpreted as (nrows, ncols). Defaults to 100.\n\n weights : array-like, optional\n The relative weight to be applied to each input measurement. The\n array will be normalized to sum to 1, so absolute value of the\n weights do not affect the result. Defaults to None.\n\n **kwargs\n Additional keyword arguments are passed on to matplotlib's\n `contourf` function.\n\n Returns\n -------\n A matplotlib `QuadContourSet`.\n\n See Also\n --------\n mplstereonet.density_grid\n mplstereonet.StereonetAxes.density_contour\n matplotlib.pyplot.contourf\n matplotlib.pyplot.clabel\n\n Examples\n --------\n Plot filled density contours of poles to the specified planes using\n a modified Kamb method with exponential smoothing [1]_.\n\n >>> strikes, dips = [120, 315, 86], [22, 85, 31]\n >>> ax.density_contourf(strikes, dips)\n\n Plot filled density contours of a set of linear orientation\n measurements.\n\n >>> plunges, bearings = [-10, 20, -30], [120, 315, 86]\n >>> ax.density_contourf(plunges, bearings, measurement='lines')\n\n Plot filled density contours of a set of rake measurements.\n\n >>> strikes, dips, rakes = [120, 315, 86], [22, 85, 31], [-5, 20, 9]\n >>> ax.density_contourf(strikes, dips, rakes, measurement='rakes')\n\n Plot filled density contours of a set of \"raw\" longitudes and\n latitudes.\n\n >>> lon, lat = np.radians([-40, 30, -85]), np.radians([21, -59, 45])\n >>> ax.density_contourf(lon, lat, measurement='radians')\n\n\n Plot filled density contours of poles to planes using a Kamb method\n [2]_ with the density estimated on a 10x10 grid (in long-lat space)\n\n >>> strikes, dips = [120, 315, 86], [22, 85, 31]\n >>> ax.density_contourf(strikes, dips, method='kamb', gridsize=10)\n\n Plot filled density contours of poles to planes with contours at\n [1,2,3] standard deviations.\n\n >>> strikes, dips = [120, 315, 86], [22, 85, 31]\n >>> ax.density_contourf(strikes, dips, levels=[1,2,3])\n\n References\n ----------\n .. [1] Vollmer, 1995. C Program for Automatic Contouring of Spherical\n Orientation Data Using a Modified Kamb Method. Computers &\n Geosciences, Vol. 21, No. 1, pp. 31--49.\n\n .. [2] Kamb, 1959. Ice Petrofabric Observations from Blue Glacier,\n Washington, in Relation to Theory and Experiment. Journal of\n Geophysical Research, Vol. 64, No. 11, pp. 1891--1909.\n \"\"\"\n lon, lat, totals, kwargs = self._contour_helper(args, kwargs)\n return self.contourf(lon, lat, totals, **kwargs)"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nsets up a basic stereonet and plot the same data each time.", "response": "def basic():\n \"\"\"Set up a basic stereonet and plot the same data each time.\"\"\"\n fig, ax = mplstereonet.subplots()\n\n strike, dip = 315, 30\n ax.plane(strike, dip, color='lightblue')\n ax.pole(strike, dip, color='green', markersize=15)\n ax.rake(strike, dip, 40, marker='*', markersize=20, color='green')\n\n # Make a bit of room for the title...\n fig.subplots_adjust(top=0.8)\n\n return ax"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef stereonet_projection_explanation(ax):\n ax.set_title('Dip and Azimuth', y=1.1, size=18)\n\n # Set the azimuth ticks to be just \"N\", \"E\", etc.\n ax.set_azimuth_ticks(range(0, 360, 10))\n\n # Hackishly set some of the azimuth labels to North, East, etc...\n fmt = ax.yaxis.get_major_formatter()\n labels = [fmt(item) for item in ax.get_azimuth_ticks()]\n labels[0] = 'North'\n labels[9] = 'East'\n labels[18] = 'South'\n labels[27] = 'West'\n ax.set_azimuth_ticklabels(labels)\n\n # Unhide the xticklabels and use them for dip labels\n ax.xaxis.set_tick_params(label1On=True)\n labels = list(range(10, 100, 10)) + list(range(80, 0, -10))\n ax.set_xticks(np.radians(np.arange(-80, 90, 10)))\n ax.set_xticklabels([fmt(np.radians(item)) for item in labels])\n\n ax.set_xlabel('Dip or Plunge')\n\n xlabel_halo(ax)\n return ax", "response": "Example to explain azimuth and dip on a lower - helisphere stereonet."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef native_projection_explanation(ax):\n ax.set_title('Longitude and Latitude', size=18, y=1.1)\n\n # Hide the azimuth labels\n ax.set_azimuth_ticklabels([])\n\n # Make the axis tick labels visible:\n ax.set_xticks(np.radians(np.arange(-80, 90, 10)))\n ax.tick_params(label1On=True)\n\n ax.set_xlabel('Longitude')\n\n xlabel_halo(ax)\n return ax", "response": "Example showing how the native longitude and latitude relate to the\n stereonet projection."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef xlabel_halo(ax):\n import matplotlib.patheffects as effects\n for tick in ax.get_xticklabels() + [ax.xaxis.label]:\n tick.set_path_effects([effects.withStroke(linewidth=4, foreground='w')])", "response": "Add a white halo around the xlabels."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef parse_strike_dip(strike, dip):\n strike = parse_azimuth(strike)\n dip, direction = split_trailing_letters(dip)\n\n if direction is not None:\n expected_direc = strike + 90\n if opposite_end(expected_direc, direction):\n strike += 180\n\n if strike > 360:\n strike -= 360\n\n return strike, dip", "response": "Parses the strike and dip measurement strings and returns the strike and dip measurements."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nparsing a strike and dip measurement and returns a strike dip and rake.", "response": "def parse_rake(strike, dip, rake):\n \"\"\"\n Parses strings of strike, dip, and rake and returns a strike, dip, and rake\n measurement following the right-hand-rule, with the \"end\" of the strike\n that the rake is measured from indicated by the sign of the rake (positive\n rakes correspond to the strike direction, negative rakes correspond to the\n opposite end).\n\n Accepts either quadrant-formatted or azimuth-formatted strikes.\n\n For example, this would convert a strike of \"N30E\", dip of \"45NW\", with a\n rake of \"10NE\" to a strike of 210, dip of 45, and rake of 170.\n\n Rake angles returned by this function will always be between 0 and 180\n\n If no directions are specified, the measuriement is assumed to follow the\n usual right-hand-rule convention.\n\n Parameters\n ----------\n strike : string\n A strike measurement. May be in azimuth or quadrant format.\n dip : string\n The dip angle and direction of a plane.\n rake : string\n The rake angle and direction that the rake is measured from.\n\n Returns\n -------\n strike, dip, rake : floats\n Measurements of strike, dip, and rake following the conventions\n outlined above.\n \"\"\"\n strike, dip = parse_strike_dip(strike, dip)\n rake, direction = split_trailing_letters(rake)\n\n if direction is not None:\n if opposite_end(strike, direction):\n rake = -rake\n\n if rake < 0:\n rake += 180\n elif rake > 180:\n rake -= 180\n\n return strike, dip, rake"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nparsing a string of plunge and bearing and returns a consistent plunge and bearing measurement as floats.", "response": "def parse_plunge_bearing(plunge, bearing):\n \"\"\"\n Parses strings of plunge and bearing and returns a consistent plunge and\n bearing measurement as floats. Plunge angles returned by this function will\n always be between 0 and 90.\n\n If no direction letter(s) is present, the plunge is assumed to be measured\n from the end specified by the bearing. If a direction letter(s) is present,\n the bearing will be switched to the opposite (180 degrees) end if the\n specified direction corresponds to the opposite end specified by the\n bearing.\n\n Parameters\n ----------\n plunge : string\n A plunge measurement.\n bearing : string\n A bearing measurement. May be in azimuth or quadrant format.\n\n Returns\n -------\n plunge, bearing: floats\n The plunge and bearing following the conventions outlined above.\n\n Examples\n ---------\n\n >>> parse_plunge_bearing(\"30NW\", 160)\n ... (30, 340)\n\n \"\"\"\n bearing = parse_azimuth(bearing)\n plunge, direction = split_trailing_letters(plunge)\n\n if direction is not None:\n if opposite_end(bearing, direction):\n bearing +=180\n\n if plunge < 0:\n bearing += 180\n plunge = -plunge\n if plunge > 90:\n bearing += 180\n plunge = 180 - plunge\n\n if bearing > 360:\n bearing -= 360\n\n return plunge, bearing"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nconverts a planar measurment of dip direction using the dip - azimuth convention into a strike using the right - hand - rule.", "response": "def dip_direction2strike(azimuth):\n \"\"\"\n Converts a planar measurment of dip direction using the dip-azimuth\n convention into a strike using the right-hand-rule.\n\n Parameters\n ----------\n azimuth : number or string\n The dip direction of the plane in degrees. This can be either a\n numerical azimuth in the 0-360 range or a string representing a quadrant\n measurement (e.g. N30W).\n\n Returns\n -------\n strike : number\n The strike of the plane in degrees following the right-hand-rule.\n \"\"\"\n azimuth = parse_azimuth(azimuth)\n strike = azimuth - 90\n if strike < 0:\n strike += 360\n return strike"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef strike2dip_direction(strike):\n strike = parse_azimuth(strike)\n dip_direction = strike + 90\n if dip_direction > 360:\n dip_direction -= 360\n return dip_direction", "response": "Converts a planar measurement of strike using the right - hand - rule into the dip - direction that the plane dips."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef parse_azimuth(azimuth):\n try:\n azimuth = float(azimuth)\n except ValueError:\n if not azimuth[0].isalpha():\n raise ValueError('Ambiguous azimuth: {}'.format(azimuth))\n azimuth = parse_quadrant_measurement(azimuth)\n return azimuth", "response": "Parses an azimuth measurement in azimuth or quadrant format."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nparses a measurement of the form AxxB or AxxB into an azimuth of the next 260 degrees.", "response": "def parse_quadrant_measurement(quad_azimuth):\n \"\"\"\n Parses a quadrant measurement of the form \"AxxB\", where A and B are cardinal\n directions and xx is an angle measured relative to those directions.\n\n In other words, it converts a measurement such as E30N into an azimuth of\n 60 degrees, or W10S into an azimuth of 260 degrees.\n\n For ambiguous quadrant measurements such as \"N30S\", a ValueError is raised.\n\n Parameters\n -----------\n quad_azimuth : string\n An azimuth measurement in quadrant form.\n\n Returns\n -------\n azi : float\n An azimuth in degrees clockwise from north.\n\n See Also\n --------\n parse_azimuth\n \"\"\"\n def rotation_direction(first, second):\n return np.cross(_azimuth2vec(first), _azimuth2vec(second))\n\n # Parse measurement\n quad_azimuth = quad_azimuth.strip()\n try:\n first_dir = quadrantletter_to_azimuth(quad_azimuth[0].upper())\n sec_dir = quadrantletter_to_azimuth(quad_azimuth[-1].upper())\n except KeyError:\n raise ValueError('{} is not a valid azimuth'.format(quad_azimuth))\n\n angle = float(quad_azimuth[1:-1])\n\n # Convert quadrant measurement into an azimuth\n direc = rotation_direction(first_dir, sec_dir)\n azi = first_dir + direc * angle\n\n # Catch ambiguous measurements such as N10S and raise an error\n if abs(direc) < 0.9:\n raise ValueError('{} is not a valid azimuth'.format(quad_azimuth))\n\n # Ensure that 0 <= azi <= 360\n if azi < 0:\n azi += 360\n elif azi > 360:\n azi -= 360\n\n return azi"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nreturns the inverse of the transform.", "response": "def inverted(self):\n \"\"\"Return the inverse of the transform.\"\"\"\n # This is a bit of hackery so that we can put a single \"inverse\"\n # function here. If we just made \"self._inverse_type\" point to the class\n # in question, it wouldn't be defined yet. This way, it's done at\n # at runtime and we avoid the definition problem. Hackish, but better\n # than repeating code everywhere or making a relatively complex\n # metaclass.\n inverse_type = globals()[self._inverse_type]\n return inverse_type(self._center_longitude, self._center_latitude,\n self._resolution)"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef sph2cart(lon, lat):\n x = np.cos(lat)*np.cos(lon)\n y = np.cos(lat)*np.sin(lon)\n z = np.sin(lat)\n return x, y, z", "response": "Converts a longitude and latitude given in\n _radians_ to cartesian coordinates."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nconvert cartesian coordinates x y z into a longitude and latitude in radians.", "response": "def cart2sph(x, y, z):\n \"\"\"\n Converts cartesian coordinates `x`, `y`, `z` into a longitude and latitude.\n x=0, y=0, z=0 is assumed to correspond to the center of the globe.\n Returns lon and lat in radians.\n\n Parameters\n ----------\n `x`, `y`, `z` : Arrays of cartesian coordinates\n\n Returns\n -------\n lon : Longitude in radians\n lat : Latitude in radians\n \"\"\"\n r = np.sqrt(x**2 + y**2 + z**2)\n lat = np.arcsin(z/r)\n lon = np.arctan2(y, x)\n return lon, lat"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nrotates a node on a given axis by the given angle.", "response": "def _rotate(lon, lat, theta, axis='x'):\n \"\"\"\n Rotate \"lon\", \"lat\" coords (in _degrees_) about the X-axis by \"theta\"\n degrees. This effectively simulates rotating a physical stereonet.\n Returns rotated lon, lat coords in _radians_).\n \"\"\"\n # Convert input to numpy arrays in radians\n lon, lat = np.atleast_1d(lon, lat)\n lon, lat = map(np.radians, [lon, lat])\n theta = np.radians(theta)\n\n # Convert to cartesian coords for the rotation\n x, y, z = sph2cart(lon, lat)\n\n lookup = {'x':_rotate_x, 'y':_rotate_y, 'z':_rotate_z}\n X, Y, Z = lookup[axis](x, y, z, theta)\n\n # Now convert back to spherical coords (longitude and latitude, ignore R)\n lon, lat = cart2sph(X,Y,Z)\n return lon, lat"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\ncalculate the antipode of the given point or sequence of numbers lon lat.", "response": "def antipode(lon, lat):\n \"\"\"\n Calculates the antipode (opposite point on the globe) of the given point or\n points. Input and output is expected to be in radians.\n\n Parameters\n ----------\n lon : number or sequence of numbers\n Longitude in radians\n lat : number or sequence of numbers\n Latitude in radians\n\n Returns\n -------\n lon, lat : arrays\n Sequences (regardless of whether or not the input was a single value or\n a sequence) of longitude and latitude in radians.\n \"\"\"\n x, y, z = sph2cart(lon, lat)\n return cart2sph(-x, -y, -z)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\ncalculating the longitude and latitude of each plane in the base set of points.", "response": "def plane(strike, dip, segments=100, center=(0, 0)):\n \"\"\"\n Calculates the longitude and latitude of `segments` points along the\n stereonet projection of each plane with a given `strike` and `dip` in\n degrees. Returns points for one hemisphere only.\n\n Parameters\n ----------\n strike : number or sequence of numbers\n The strike of the plane(s) in degrees, with dip direction indicated by\n the azimuth (e.g. 315 vs. 135) specified following the \"right hand\n rule\".\n dip : number or sequence of numbers\n The dip of the plane(s) in degrees.\n segments : number or sequence of numbers\n The number of points in the returned `lon` and `lat` arrays. Defaults\n to 100 segments.\n center : sequence of two numbers (lon, lat)\n The longitude and latitude of the center of the hemisphere that the\n returned points will be in. Defaults to 0,0 (approriate for a typical\n stereonet).\n\n Returns\n -------\n lon, lat : arrays\n `num_segments` x `num_strikes` arrays of longitude and latitude in\n radians.\n \"\"\"\n lon0, lat0 = center\n strikes, dips = np.atleast_1d(strike, dip)\n lons = np.zeros((segments, strikes.size), dtype=np.float)\n lats = lons.copy()\n for i, (strike, dip) in enumerate(zip(strikes, dips)):\n # We just plot a line of constant longitude and rotate it by the strike.\n dip = 90 - dip\n lon = dip * np.ones(segments)\n lat = np.linspace(-90, 90, segments)\n lon, lat = _rotate(lon, lat, strike)\n\n if lat0 != 0 or lon0 != 0:\n dist = angular_distance([lon, lat], [lon0, lat0], False)\n mask = dist > (np.pi / 2)\n lon[mask], lat[mask] = antipode(lon[mask], lat[mask])\n change = np.diff(mask.astype(int))\n ind = np.flatnonzero(change) + 1\n lat = np.hstack(np.split(lat, ind)[::-1])\n lon = np.hstack(np.split(lon, ind)[::-1])\n\n lons[:,i] = lon\n lats[:,i] = lat\n\n return lons, lats"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef pole(strike, dip):\n strike, dip = np.atleast_1d(strike, dip)\n mask = dip > 90\n dip[mask] = 180 - dip[mask]\n strike[mask] += 180\n # Plot the approriate point for a strike of 0 and rotate it\n lon, lat = -dip, 0.0\n lon, lat = _rotate(lon, lat, strike)\n return lon, lat", "response": "Calculates the longitude and latitude of the pole to the plane."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef rake(strike, dip, rake_angle):\n strike, dip, rake_angle = np.atleast_1d(strike, dip, rake_angle)\n # Plot the approriate point for a strike of 0 and rotate it\n dip = 90 - dip\n lon = dip\n\n rake_angle = rake_angle.copy()\n rake_angle[rake_angle < 0] += 180\n lat = 90 - rake_angle\n\n lon, lat = _rotate(lon, lat, strike)\n return lon, lat", "response": "Calculates the longitude and latitude of the linear feature specified by strike dip and rake_angle."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef line(plunge, bearing):\n plunge, bearing = np.atleast_1d(plunge, bearing)\n # Plot the approriate point for a bearing of 0 and rotate it\n lat = 90 - plunge\n lon = 0\n lon, lat = _rotate(lon, lat, bearing)\n return lon, lat", "response": "Calculates the longitude and latitude of the linear feature specified by the plunge and bearing."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef cone(plunge, bearing, angle, segments=100):\n plunges, bearings, angles = np.atleast_1d(plunge, bearing, angle)\n lons, lats = [], []\n for plunge, bearing, angle in zip(plunges, bearings, angles):\n lat = (90 - angle) * np.ones(segments, dtype=float)\n lon = np.linspace(-180, 180, segments)\n lon, lat = _rotate(lon, lat, -plunge, axis='y')\n lon, lat = _rotate(np.degrees(lon), np.degrees(lat), bearing, axis='x')\n lons.append(lon)\n lats.append(lat)\n return np.vstack(lons), np.vstack(lats)", "response": "Calculates the longitude and latitude of a small circle."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nconverting the given plunge and bearing in degrees to a strike and dip tuple.", "response": "def plunge_bearing2pole(plunge, bearing):\n \"\"\"\n Converts the given `plunge` and `bearing` in degrees to a strike and dip\n of the plane whose pole would be parallel to the line specified. (i.e. The\n pole to the plane returned would plot at the same point as the specified\n plunge and bearing.)\n\n Parameters\n ----------\n plunge : number or sequence of numbers\n The plunge of the line(s) in degrees. The plunge is measured in degrees\n downward from the end of the feature specified by the bearing.\n bearing : number or sequence of numbers\n The bearing (azimuth) of the line(s) in degrees.\n\n Returns\n -------\n strike, dip : arrays\n Arrays of strikes and dips in degrees following the right-hand-rule.\n \"\"\"\n plunge, bearing = np.atleast_1d(plunge, bearing)\n strike = bearing + 90\n dip = 90 - plunge\n strike[strike >= 360] -= 360\n return strike, dip"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef pole2plunge_bearing(strike, dip):\n strike, dip = np.atleast_1d(strike, dip)\n bearing = strike - 90\n plunge = 90 - dip\n bearing[bearing < 0] += 360\n return plunge, bearing", "response": "Converts the given strike and dip in dgrees of a plane to a plunge\n and bearing of its pole."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef mean_vector(lons, lats):\n xyz = sph2cart(lons, lats)\n xyz = np.vstack(xyz).T\n mean_vec = xyz.mean(axis=0)\n r_value = np.linalg.norm(mean_vec)\n mean_vec = cart2sph(*mean_vec)\n return mean_vec, r_value", "response": "Returns the resultant vector from a series of longitudes and latitudes."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef fisher_stats(lons, lats, conf=95):\n xyz = sph2cart(lons, lats)\n xyz = np.vstack(xyz).T\n mean_vec = xyz.mean(axis=0)\n r_value = np.linalg.norm(mean_vec)\n num = xyz.shape[0]\n mean_vec = cart2sph(*mean_vec)\n\n if num > 1:\n p = (100.0 - conf) / 100.0\n vector_sum = xyz.sum(axis=0)\n result_vect = np.sqrt(np.sum(np.square(vector_sum)))\n fract1 = (num - result_vect) / result_vect\n fract3 = 1.0 / (num - 1.0)\n angle = np.arccos(1 - fract1 * ((1 / p) ** fract3 - 1))\n angle = np.degrees(angle)\n kappa = (num - 1.0) / (num - result_vect)\n return mean_vec, (r_value, angle, kappa)\n else:\n return None, None", "response": "Function that calculates the Fisher statistics for a set of longitudes and latitudes."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef geographic2pole(lon, lat):\n plunge, bearing = geographic2plunge_bearing(lon, lat)\n strike = bearing + 90\n strike[strike >= 360] -= 360\n dip = 90 - plunge\n return strike, dip", "response": "Converts a longitude and latitude from a stereonet into a strike and dip tuple."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nconverting longitude and latitude in stereonet coordinates into plunge and bearing arrays.", "response": "def geographic2plunge_bearing(lon, lat):\n \"\"\"\n Converts longitude and latitude in stereonet coordinates into a\n plunge/bearing.\n\n Parameters\n ----------\n lon, lat : numbers or sequences of numbers\n Longitudes and latitudes in radians as measured from a\n lower-hemisphere stereonet\n\n Returns\n -------\n plunge : array\n The plunge of the vector in degrees downward from horizontal.\n bearing : array\n The bearing of the vector in degrees clockwise from north.\n \"\"\"\n lon, lat = np.atleast_1d(lon, lat)\n x, y, z = sph2cart(lon, lat)\n\n # Bearing will be in the y-z plane...\n bearing = np.arctan2(z, y)\n\n # Plunge is the angle between the line and the y-z plane\n r = np.sqrt(x*x + y*y + z*z)\n r[r == 0] = 1e-15\n plunge = np.arcsin(x / r)\n\n # Convert back to azimuths in degrees..\n plunge, bearing = np.degrees(plunge), np.degrees(bearing)\n bearing = 90 - bearing\n bearing[bearing < 0] += 360\n\n # If the plunge angle is upwards, get the opposite end of the line\n upwards = plunge < 0\n plunge[upwards] *= -1\n bearing[upwards] -= 180\n bearing[upwards & (bearing < 0)] += 360\n\n return plunge, bearing"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nfinding the intersection of two strikes and dips.", "response": "def plane_intersection(strike1, dip1, strike2, dip2):\n \"\"\"\n Finds the intersection of two planes. Returns a plunge/bearing of the linear\n intersection of the two planes.\n\n Also accepts sequences of strike1s, dip1s, strike2s, dip2s.\n\n Parameters\n ----------\n strike1, dip1 : numbers or sequences of numbers\n The strike and dip (in degrees, following the right-hand-rule) of the\n first plane(s).\n strike2, dip2 : numbers or sequences of numbers\n The strike and dip (in degrees, following the right-hand-rule) of the\n second plane(s).\n\n Returns\n -------\n plunge, bearing : arrays\n The plunge and bearing(s) (in degrees) of the line representing the\n intersection of the two planes.\n \"\"\"\n norm1 = sph2cart(*pole(strike1, dip1))\n norm2 = sph2cart(*pole(strike2, dip2))\n norm1, norm2 = np.array(norm1), np.array(norm2)\n lon, lat = cart2sph(*np.cross(norm1, norm2, axis=0))\n return geographic2plunge_bearing(lon, lat)"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nprojects a linear feature onto the surface of a plane.", "response": "def project_onto_plane(strike, dip, plunge, bearing):\n \"\"\"\n Projects a linear feature(s) onto the surface of a plane. Returns a rake\n angle(s) along the plane.\n\n This is also useful for finding the rake angle of a feature that already\n intersects the plane in question.\n\n Parameters\n ----------\n strike, dip : numbers or sequences of numbers\n The strike and dip (in degrees, following the right-hand-rule) of the\n plane(s).\n plunge, bearing : numbers or sequences of numbers\n The plunge and bearing (in degrees) or of the linear feature(s) to be\n projected onto the plane.\n\n Returns\n -------\n rake : array\n A sequence of rake angles measured downwards from horizontal in\n degrees. Zero degrees corresponds to the \"right- hand\" direction\n indicated by the strike, while a negative angle corresponds to the\n opposite direction. Rakes returned by this function will always be\n between -90 and 90 (inclusive).\n \"\"\"\n # Project the line onto the plane\n norm = sph2cart(*pole(strike, dip))\n feature = sph2cart(*line(plunge, bearing))\n norm, feature = np.array(norm), np.array(feature)\n perp = np.cross(norm, feature, axis=0)\n on_plane = np.cross(perp, norm, axis=0)\n on_plane /= np.sqrt(np.sum(on_plane**2, axis=0))\n\n # Calculate the angle between the projected feature and horizontal\n # This is just a dot product, but we need to work with multiple measurements\n # at once, so einsum is quicker than apply_along_axis.\n strike_vec = sph2cart(*line(0, strike))\n dot = np.einsum('ij,ij->j', on_plane, strike_vec)\n rake = np.degrees(np.arccos(dot))\n\n # Convert rakes over 90 to negative rakes...\n rake[rake > 90] -= 180\n rake[rake < -90] += 180\n return rake"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nproject an azimuth of a linear feature onto a plane as a rake angle. Parameters ---------- strike, dip : numbers The strike and dip of the plane in degrees following the right-hand-rule. azimuth : numbers The azimuth of the linear feature in degrees clockwise from north (i.e. a 0-360 azimuth). Returns ------- rake : number A rake angle in degrees measured downwards from horizontal. Negative values correspond to the opposite end of the strike.", "response": "def azimuth2rake(strike, dip, azimuth):\n \"\"\"\n Projects an azimuth of a linear feature onto a plane as a rake angle.\n\n Parameters\n ----------\n strike, dip : numbers\n The strike and dip of the plane in degrees following the\n right-hand-rule.\n azimuth : numbers\n The azimuth of the linear feature in degrees clockwise from north (i.e.\n a 0-360 azimuth).\n\n Returns\n -------\n rake : number\n A rake angle in degrees measured downwards from horizontal. Negative\n values correspond to the opposite end of the strike.\n \"\"\"\n plunge, bearing = plane_intersection(strike, dip, azimuth, 90)\n rake = project_onto_plane(strike, dip, plunge, bearing)\n return rake"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nconverting x y z in _world_ cartesian coordinates into lower - helisphere stereonet coordinates.", "response": "def xyz2stereonet(x, y, z):\n \"\"\"\n Converts x, y, z in _world_ cartesian coordinates into lower-hemisphere\n stereonet coordinates.\n\n Parameters\n ----------\n x, y, z : array-likes\n Sequences of world coordinates\n\n Returns\n -------\n lon, lat : arrays\n Sequences of longitudes and latitudes (in radians)\n \"\"\"\n x, y, z = np.atleast_1d(x, y, z)\n return cart2sph(-z, x, y)"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef stereonet2xyz(lon, lat):\n lon, lat = np.atleast_1d(lon, lat)\n x, y, z = sph2cart(lon, lat)\n return y, z, -x", "response": "Converts a sequence of longitudes and latitudes from a lower - helisphere stereonet into _world_ x y z coordinates."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\ncalculating the angular distance between two sets of linear features or elementwise linear features or elementwise linear features.", "response": "def angular_distance(first, second, bidirectional=True):\n \"\"\"\n Calculate the angular distance between two linear features or elementwise\n angular distance between two sets of linear features. (Note: a linear\n feature in this context is a point on a stereonet represented\n by a single latitude and longitude.)\n\n Parameters\n ----------\n first : (lon, lat) 2xN array-like or sequence of two numbers\n The longitudes and latitudes of the first measurements in radians.\n second : (lon, lat) 2xN array-like or sequence of two numbers\n The longitudes and latitudes of the second measurements in radians.\n bidirectional : boolean\n If True, only \"inner\" angles will be returned. In other words, all\n angles returned by this function will be in the range [0, pi/2]\n (0 to 90 in degrees). Otherwise, ``first`` and ``second``\n will be treated as vectors going from the origin outwards\n instead of bidirectional infinite lines. Therefore, with\n ``bidirectional=False``, angles returned by this function\n will be in the range [0, pi] (zero to 180 degrees).\n\n Returns\n -------\n dist : array\n The elementwise angular distance between each pair of measurements in\n (lon1, lat1) and (lon2, lat2).\n\n Examples\n --------\n\n Calculate the angle between two lines specified as a plunge/bearing\n\n >>> angle = angular_distance(line(30, 270), line(40, 90))\n >>> np.degrees(angle)\n array([ 70.])\n\n Let's do the same, but change the \"bidirectional\" argument:\n\n >>> first, second = line(30, 270), line(40, 90)\n >>> angle = angular_distance(first, second, bidirectional=False)\n >>> np.degrees(angle)\n array([ 110.])\n\n Calculate the angle between two planes.\n\n >>> angle = angular_distance(pole(0, 10), pole(180, 10))\n >>> np.degrees(angle)\n array([ 20.])\n \"\"\"\n lon1, lat1 = first\n lon2, lat2 = second\n lon1, lat1, lon2, lat2 = np.atleast_1d(lon1, lat1, lon2, lat2)\n xyz1 = sph2cart(lon1, lat1)\n xyz2 = sph2cart(lon2, lat2)\n # This is just a dot product, but we need to work with multiple measurements\n # at once, so einsum is quicker than apply_along_axis.\n dot = np.einsum('ij,ij->j', xyz1, xyz2)\n angle = np.arccos(dot)\n\n # There are numerical sensitivity issues around 180 and 0 degrees...\n # Sometimes a result will have an absolute value slighly over 1.\n if np.any(np.isnan(angle)):\n rtol = 1e-4\n angle[np.isclose(dot, -1, rtol)] = np.pi\n angle[np.isclose(dot, 1, rtol)] = 0\n\n if bidirectional:\n mask = angle > np.pi / 2\n angle[mask] = np.pi - angle[mask]\n\n return angle"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _repole(lon, lat, center):\n vec3 = sph2cart(*center)\n vec3 = np.squeeze(vec3)\n if not np.allclose(vec3, [0, 0, 1]):\n vec1 = np.cross(vec3, [0, 0, 1])\n else:\n vec1 = np.cross(vec3, [1, 0, 0])\n vec2 = np.cross(vec3, vec1)\n vecs = [item / np.linalg.norm(item) for item in [vec1, vec2, vec3]]\n basis = np.column_stack(vecs)\n\n xyz = sph2cart(lon, lat)\n xyz = np.column_stack(xyz)\n prime = xyz.dot(np.linalg.inv(basis))\n lon, lat = cart2sph(*prime.T)\n return lon[:,None], lat[:,None]", "response": "Reproject data such that center is the north pole. Returns lon lat in the new rotated reference frame."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nunify fit_pole and fit_girdle.", "response": "def _sd_of_eigenvector(data, vec, measurement='poles', bidirectional=True):\n \"\"\"Unifies ``fit_pole`` and ``fit_girdle``.\"\"\"\n lon, lat = _convert_measurements(data, measurement)\n vals, vecs = cov_eig(lon, lat, bidirectional)\n x, y, z = vecs[:, vec]\n s, d = stereonet_math.geographic2pole(*stereonet_math.cart2sph(x, y, z))\n return s[0], d[0]"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef eigenvectors(*args, **kwargs):\n lon, lat = _convert_measurements(args, kwargs.get('measurement', 'poles'))\n vals, vecs = cov_eig(lon, lat, kwargs.get('bidirectional', True))\n lon, lat = stereonet_math.cart2sph(*vecs)\n plunges, bearings = stereonet_math.geographic2plunge_bearing(lon, lat)\n # Largest eigenvalue first...\n return plunges[::-1], bearings[::-1], vals[::-1]", "response": "Returns the 3D eigenvectors of the 3D covariance matrix of the avec of the specified set of measurements."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef find_mean_vector(*args, **kwargs):\n lon, lat = _convert_measurements(args, kwargs.get('measurement', 'lines'))\n vector, r_value = stereonet_math.mean_vector(lon, lat)\n plunge, bearing = stereonet_math.geographic2plunge_bearing(*vector)\n return (plunge[0], bearing[0]), r_value", "response": "Returns the mean vector for a set of measurments."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nreturning the mean vector and summary statistics for a set of measurements.", "response": "def find_fisher_stats(*args, **kwargs):\n \"\"\"\n Returns the mean vector and summary statistics for a set of measurements.\n By default, this expects the input to be plunges and bearings, but the type\n of input can be controlled through the ``measurement`` kwarg.\n\n Parameters\n ----------\n *args : 2 or 3 sequences of measurements\n By default, this will be expected to be ``plunge`` & ``bearing``, both\n array-like sequences representing linear features. (Rake measurements\n require three parameters, thus the variable number of arguments.) The\n *measurement* kwarg controls how these arguments are interpreted.\n\n conf : number\n The confidence level (0-100). Defaults to 95%, similar to 2 sigma.\n\n measurement : string, optional\n Controls how the input arguments are interpreted. Defaults to\n ``\"lines\"``. May be one of the following:\n\n ``\"poles\"`` : strikes, dips\n Arguments are assumed to be sequences of strikes and dips of\n planes. Poles to these planes are used for analysis.\n ``\"lines\"`` : plunges, bearings\n Arguments are assumed to be sequences of plunges and bearings\n of linear features.\n ``\"rakes\"`` : strikes, dips, rakes\n Arguments are assumed to be sequences of strikes, dips, and\n rakes along the plane.\n ``\"radians\"`` : lon, lat\n Arguments are assumed to be \"raw\" longitudes and latitudes in\n the stereonet's underlying coordinate system.\n\n Returns\n -------\n\n mean_vector: tuple of two floats\n A set consisting of the plunge and bearing of the mean vector (in\n degrees).\n\n stats : tuple of three floats\n ``(r_value, confidence, kappa)``\n The ``r_value`` is the magnitude of the mean vector as a number between\n 0 and 1.\n The ``confidence`` radius is the opening angle of a small circle that\n corresponds to the confidence in the calculated direction, and is\n dependent on the input ``conf``.\n The ``kappa`` value is the dispersion factor that quantifies the amount\n of dispersion of the given vectors, analgous to a variance/stddev.\n \"\"\"\n # How the heck did this wind up as a separate function?\n lon, lat = _convert_measurements(args, kwargs.get('measurement', 'lines'))\n conf = kwargs.get('conf', 95)\n center, stats = stereonet_math.fisher_stats(lon, lat, conf)\n plunge, bearing = stereonet_math.geographic2plunge_bearing(*center)\n mean_vector = (plunge[0], bearing[0])\n return mean_vector, stats"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nreturns a new Nx2 array - like structure that can be used to create a multi - modal cluster for a set of measurements.", "response": "def kmeans(*args, **kwargs):\n \"\"\"\n Find centers of multi-modal clusters of data using a kmeans approach\n modified for spherical measurements.\n\n Parameters\n ----------\n\n *args : 2 or 3 sequences of measurements\n By default, this will be expected to be ``strike`` & ``dip``, both\n array-like sequences representing poles to planes. (Rake measurements\n require three parameters, thus the variable number of arguments.) The\n ``measurement`` kwarg controls how these arguments are interpreted.\n\n num : int\n The number of clusters to find. Defaults to 2.\n\n bidirectional : bool\n Whether or not the measurements are bi-directional linear/planar\n features or directed vectors. Defaults to True.\n\n tolerance : float\n Iteration will continue until the centers have not changed by more\n than this amount. Defaults to 1e-5.\n\n measurement : string, optional\n Controls how the input arguments are interpreted. Defaults to\n ``\"poles\"``. May be one of the following:\n\n ``\"poles\"`` : strikes, dips\n Arguments are assumed to be sequences of strikes and dips of\n planes. Poles to these planes are used for analysis.\n ``\"lines\"`` : plunges, bearings\n Arguments are assumed to be sequences of plunges and bearings\n of linear features.\n ``\"rakes\"`` : strikes, dips, rakes\n Arguments are assumed to be sequences of strikes, dips, and\n rakes along the plane.\n ``\"radians\"`` : lon, lat\n Arguments are assumed to be \"raw\" longitudes and latitudes in\n the stereonet's underlying coordinate system.\n\n Returns\n -------\n\n centers : An Nx2 array-like\n Longitude and latitude in radians of the centers of each cluster.\n \"\"\"\n lon, lat = _convert_measurements(args, kwargs.get('measurement', 'poles'))\n num = kwargs.get('num', 2)\n bidirectional = kwargs.get('bidirectional', True)\n tolerance = kwargs.get('tolerance', 1e-5)\n\n points = lon, lat\n dist = lambda x: stereonet_math.angular_distance(x, points, bidirectional)\n\n center_lon = np.random.choice(lon, num)\n center_lat = np.random.choice(lat, num)\n centers = np.column_stack([center_lon, center_lat])\n\n while True:\n dists = np.array([dist(item) for item in centers]).T\n closest = dists.argmin(axis=1)\n\n new_centers = []\n for i in range(num):\n mask = mask = closest == i\n _, vecs = cov_eig(lon[mask], lat[mask], bidirectional)\n new_centers.append(stereonet_math.cart2sph(*vecs[:,-1]))\n\n if np.allclose(centers, new_centers, atol=tolerance):\n break\n else:\n centers = new_centers\n\n return centers"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\ncheck if a connection is closed or not.", "response": "def is_disconnect(e, connection, cursor):\n \"\"\"\n Connection state check from SQLAlchemy:\n https://bitbucket.org/sqlalchemy/sqlalchemy/src/tip/lib/sqlalchemy/dialects/postgresql/psycopg2.py\n \"\"\"\n if isinstance(e, OperationalError):\n # these error messages from libpq: interfaces/libpq/fe-misc.c.\n # TODO: these are sent through gettext in libpq and we can't\n # check within other locales - consider using connection.closed\n return 'terminating connection' in str(e) or \\\n 'closed the connection' in str(e) or \\\n 'connection not open' in str(e) or \\\n 'could not receive data from server' in str(e)\n elif isinstance(e, InterfaceError):\n # psycopg2 client errors, psycopg2/conenction.h, psycopg2/cursor.h\n return 'connection already closed' in str(e) or \\\n 'cursor already closed' in str(e)\n elif isinstance(e, ProgrammingError):\n # not sure where this path is originally from, it may\n # be obsolete. It really says \"losed\", not \"closed\".\n return \"closed the connection unexpectedly\" in str(e)\n else:\n return False"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _dispose(self):\n self.close()\n # _DBProxy.dispose doesn't actually call dispose on the pool\n conn_params = self.get_connection_params()\n key = db_pool._serialize(**conn_params)\n try:\n pool = db_pool.pools[key]\n except KeyError:\n pass\n else:\n pool.dispose()\n del db_pool.pools[key]", "response": "Dispose of the pool for this instance closing all connections and deleting the pool."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef calc_fc_size(img_height, img_width):\n '''Calculates shape of data after encoding.\n\n Parameters\n ----------\n img_height : int\n Height of input image.\n img_width : int\n Width of input image.\n\n Returns\n -------\n encoded_shape : tuple(int)\n Gives back 3-tuple with new dims.\n '''\n height, width = img_height, img_width\n for _ in range(5):\n height, width = _get_conv_outsize(\n (height, width),\n 4, 2, 1)\n\n conv_out_layers = 512\n return conv_out_layers, height, width", "response": "Calculates the shape of data after encoding."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef calc_im_size(img_height, img_width):\n '''Calculates shape of data after decoding.\n\n Parameters\n ----------\n img_height : int\n Height of encoded data.\n img_width : int\n Width of encoded data.\n\n Returns\n -------\n encoded_shape : tuple(int)\n Gives back 2-tuple with decoded image dimensions.\n '''\n height, width = img_height, img_width\n for _ in range(5):\n height, width = _get_deconv_outsize((height, width),\n 4, 2, 1)\n\n return height, width", "response": "Calculates the shape of data after decoding."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\ngets all the paths of non - hidden files in a directory and returns a list of those paths.", "response": "def get_paths(directory):\n '''Gets all the paths of non-hidden files in a directory\n and returns a list of those paths.\n\n Parameters\n ----------\n directory : str\n The directory whose contents you wish to grab.\n\n Returns\n -------\n paths : List[str]\n '''\n fnames = [os.path.join(directory, f)\n for f in os.listdir(directory)\n if os.path.isfile(os.path.join(directory, f))\n and not f.startswith('.')]\n return fnames"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef image_resize(file_paths, new_dir, width, height):\n '''Resizes all images with given paths to new dimensions.\n Uses up/downscaling with antialiasing.\n\n Parameters\n ----------\n file_paths : List[str]\n List of path strings for image to resize.\n new_dir : str\n Directory to place resized images.\n width : int\n Target width of new resized images.\n height : int\n Target height of new resized images.\n '''\n if new_dir[-1] != '/':\n new_dir += '/'\n\n if not os.path.exists(os.path.dirname(new_dir)):\n os.makedirs(os.path.dirname(new_dir))\n\n for f in tqdm.tqdm(file_paths):\n img = Image.open(f).resize((width, height), Image.ANTIALIAS).convert('RGB')\n new = os.path.join(new_dir, os.path.basename(f))\n img.save(new)", "response": "Resizes all images with given paths to new dimensions."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef inverse_transform(self, data, test=False):\n '''Transform latent vectors into images.\n\n Parameters\n ----------\n data : array-like shape (n_images, latent_width)\n Input numpy array of images.\n test [optional] : bool\n Controls the test boolean for batch normalization.\n\n Returns\n -------\n images : array-like shape (n_images, image_width, image_height,\n n_colors)\n '''\n if not type(data) == Variable:\n if len(data.shape) < 2:\n data = data[np.newaxis]\n if len(data.shape) != 2:\n raise TypeError(\"Invalid dimensions for latent data. Dim = %s.\\\n Must be a 2d array.\" % str(data.shape))\n data = Variable(data)\n\n else:\n if len(data.data.shape) < 2:\n data.data = data.data[np.newaxis]\n if len(data.data.shape) != 2:\n raise TypeError(\"Invalid dimensions for latent data. Dim = %s.\\\n Must be a 2d array.\" % str(data.data.shape))\n assert data.data.shape[-1] == self.latent_width,\\\n \"Latent shape %d != %d\" % (data.data.shape[-1], self.latent_width)\n\n if self.flag_gpu:\n data.to_gpu()\n out = self.model.decode(data, test=test)\n\n out.to_cpu()\n\n if self.mode == 'linear':\n final = out.data\n else:\n final = out.data.transpose(0, 2, 3, 1)\n\n return final", "response": "Transform latent vectors into images."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nload in image files from list of filepaths.", "response": "def load_images(self, filepaths):\n '''Load in image files from list of paths.\n\n Parameters\n ----------\n\n filepaths : List[str]\n List of file paths of images to be loaded.\n\n Returns\n -------\n images : array-like shape (n_images, n_colors, image_width, image_height)\n Images normalized to have pixel data range [0,1].\n\n '''\n def read(fname):\n im = Image.open(fname)\n im = np.float32(im)\n return im/255.\n x_all = np.array([read(fname) for fname in tqdm.tqdm(filepaths)])\n x_all = x_all.astype('float32')\n if self.mode == 'convolution':\n x_all = x_all.transpose(0, 3, 1, 2)\n print(\"Image Files Loaded!\")\n return x_all"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef fit(\n self,\n img_data,\n save_freq=-1,\n pic_freq=-1,\n\n n_epochs=100,\n batch_size=50,\n weight_decay=True,\n model_path='./VAE_training_model/',\n img_path='./VAE_training_images/',\n img_out_width=10\n ):\n\n '''Fit the VAE model to the image data.\n\n Parameters\n ----------\n\n img_data : array-like shape (n_images, n_colors, image_width, image_height)\n Images used to fit VAE model.\n save_freq [optional] : int\n Sets the number of epochs to wait before saving the model and optimizer states.\n Also saves image files of randomly generated images using those states in a\n separate directory. Does not save if negative valued.\n pic_freq [optional] : int\n Sets the number of batches to wait before displaying a picture or randomly\n generated images using the current model state.\n Does not display if negative valued.\n n_epochs [optional] : int\n Gives the number of training epochs to run through for the fitting\n process.\n batch_size [optional] : int\n The size of the batch to use when training. Note: generally larger\n batch sizes will result in fater epoch iteration, but at the const\n of lower granulatity when updating the layer weights.\n weight_decay [optional] : bool\n Flag that controls adding weight decay hooks to the optimizer.\n model_path [optional] : str\n Directory where the model and optimizer state files will be saved.\n img_path [optional] : str\n Directory where the end of epoch training image files will be saved.\n img_out_width : int\n Controls the number of randomly genreated images per row in the output\n saved imags.\n '''\n width = img_out_width\n self.opt.setup(self.model)\n\n if weight_decay:\n self.opt.add_hook(chainer.optimizer.WeightDecay(0.00001))\n\n n_data = img_data.shape[0]\n\n batch_iter = list(range(0, n_data, batch_size))\n n_batches = len(batch_iter)\n save_counter = 0\n for epoch in range(1, n_epochs + 1):\n print('epoch: %i' % epoch)\n t1 = time.time()\n indexes = np.random.permutation(n_data)\n last_loss_kl = 0.\n last_loss_rec = 0.\n count = 0\n for i in tqdm.tqdm(batch_iter):\n\n x_batch = Variable(img_data[indexes[i: i + batch_size]])\n\n if self.flag_gpu:\n x_batch.to_gpu()\n\n out, kl_loss, rec_loss = self.model.forward(x_batch)\n total_loss = rec_loss + kl_loss*self.kl_ratio\n\n self.opt.zero_grads()\n total_loss.backward()\n self.opt.update()\n\n last_loss_kl += kl_loss.data\n last_loss_rec += rec_loss.data\n plot_pics = Variable(img_data[indexes[:width]])\n count += 1\n if pic_freq > 0:\n assert type(pic_freq) == int, \"pic_freq must be an integer.\"\n if count % pic_freq == 0:\n fig = self._plot_img(\n plot_pics,\n img_path=img_path,\n epoch=epoch\n )\n display(fig)\n\n if save_freq > 0:\n save_counter += 1\n assert type(save_freq) == int, \"save_freq must be an integer.\"\n if epoch % save_freq == 0:\n name = \"vae_epoch%s\" % str(epoch)\n if save_counter == 1:\n save_meta = True\n else:\n save_meta = False\n self.save(model_path, name, save_meta=save_meta)\n fig = self._plot_img(\n plot_pics,\n img_path=img_path,\n epoch=epoch,\n batch=n_batches,\n save=True\n )\n\n msg = \"rec_loss = {0} , kl_loss = {1}\"\n print(msg.format(last_loss_rec/n_batches, last_loss_kl/n_batches))\n t_diff = time.time()-t1\n print(\"time: %f\\n\\n\" % t_diff)", "response": "Fit the VAE model to the image data."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nsave the current state of the class to a sequence of files in the format : . h5 .", "response": "def save(self, path, name, save_meta=True):\n '''Saves model as a sequence of files in the format:\n {path}/{name}_{'model', 'opt', 'meta'}.h5\n\n Parameters\n ----------\n path : str\n The directory of the file you wish to save the model to.\n name : str\n The name prefix of the model and optimizer files you wish\n to save.\n save_meta [optional] : bool\n Flag that controls whether to save the class metadata along with\n the encoder, decoder, and respective optimizer states.\n '''\n _save_model(self.model, str(path), \"%s_model\" % str(name))\n _save_model(self.opt, str(path), \"%s_opt\" % str(name))\n if save_meta:\n self._save_meta(os.path.join(path, \"%s_meta\" % str(name)))"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nloads in model as a class instance with the specified model and optimizer states.", "response": "def load(cls, model, opt, meta, flag_gpu=None):\n '''Loads in model as a class instance with with the specified\n model and optimizer states.\n\n Parameters\n ----------\n model : str\n Path to the model state file.\n opt : str\n Path to the optimizer state file.\n meta : str\n Path to the class metadata state file.\n flag_gpu : bool\n Specifies whether to load the model to use gpu capabilities.\n\n Returns\n -------\n\n class instance of self.\n '''\n mess = \"Model file {0} does not exist. Please check the file path.\"\n assert os.path.exists(model), mess.format(model)\n assert os.path.exists(opt), mess.format(opt)\n assert os.path.exists(meta), mess.format(meta)\n with open(meta, 'r') as f:\n meta = json.load(f)\n if flag_gpu is not None:\n meta['flag_gpu'] = flag_gpu\n\n loaded_class = cls(**meta)\n\n serializers.load_hdf5(model, loaded_class.model)\n loaded_class.opt.setup(loaded_class.model)\n serializers.load_hdf5(opt, loaded_class.opt)\n\n if meta['flag_gpu']:\n loaded_class.model = loaded_class.model.to_gpu()\n\n return loaded_class"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nfit the GAN model to the image data.", "response": "def fit(\n self,\n img_data,\n save_freq=-1,\n pic_freq=-1,\n n_epochs=100,\n batch_size=50,\n weight_decay=True,\n model_path='./GAN_training_model/',\n img_path='./GAN_training_images/',\n img_out_width=10,\n mirroring=False\n ):\n '''Fit the GAN model to the image data.\n\n Parameters\n ----------\n\n img_data : array-like shape (n_images, n_colors, image_width, image_height)\n Images used to fit VAE model.\n save_freq [optional] : int\n Sets the number of epochs to wait before saving the model and optimizer states.\n Also saves image files of randomly generated images using those states in a\n separate directory. Does not save if negative valued.\n pic_freq [optional] : int\n Sets the number of batches to wait before displaying a picture or randomly\n generated images using the current model state.\n Does not display if negative valued.\n n_epochs [optional] : int\n Gives the number of training epochs to run through for the fitting\n process.\n batch_size [optional] : int\n The size of the batch to use when training. Note: generally larger\n batch sizes will result in fater epoch iteration, but at the const\n of lower granulatity when updating the layer weights.\n weight_decay [optional] : bool\n Flag that controls adding weight decay hooks to the optimizer.\n model_path [optional] : str\n Directory where the model and optimizer state files will be saved.\n img_path [optional] : str\n Directory where the end of epoch training image files will be saved.\n img_out_width : int\n Controls the number of randomly genreated images per row in the output\n saved imags.\n mirroring [optional] : bool\n Controls whether images are randomly mirrored along the verical axis with\n a .5 probability. Artificially increases images variance for training set.\n '''\n width = img_out_width\n self.dec_opt.setup(self.dec)\n self.disc_opt.setup(self.disc)\n\n if weight_decay:\n self.dec_opt.add_hook(chainer.optimizer.WeightDecay(0.00001))\n self.disc_opt.add_hook(chainer.optimizer.WeightDecay(0.00001))\n\n n_data = img_data.shape[0]\n\n batch_iter = list(range(0, n_data, batch_size))\n n_batches = len(batch_iter)\n\n c_samples = np.random.standard_normal((width, self.latent_width)).astype(np.float32)\n save_counter = 0\n\n for epoch in range(1, n_epochs + 1):\n print('epoch: %i' % epoch)\n t1 = time.time()\n indexes = np.random.permutation(n_data)\n last_loss_dec = 0.\n last_loss_disc = 0.\n count = 0\n for i in tqdm.tqdm(batch_iter):\n x = img_data[indexes[i: i + batch_size]]\n size = x.shape[0]\n if mirroring:\n for j in range(size):\n if np.random.randint(2):\n x[j, :, :, :] = x[j, :, :, ::-1]\n x_batch = Variable(x)\n zeros = Variable(np.zeros(size, dtype=np.int32))\n ones = Variable(np.ones(size, dtype=np.int32))\n\n if self.flag_gpu:\n x_batch.to_gpu()\n zeros.to_gpu()\n ones.to_gpu()\n\n disc_samp, disc_batch = self._forward(x_batch)\n\n L_dec = F.softmax_cross_entropy(disc_samp, ones)\n\n L_disc = F.softmax_cross_entropy(disc_samp, zeros)\n L_disc += F.softmax_cross_entropy(disc_batch, ones)\n L_disc /= 2.\n\n self.dec_opt.zero_grads()\n L_dec.backward()\n self.dec_opt.update()\n\n self.disc_opt.zero_grads()\n L_disc.backward()\n self.disc_opt.update()\n\n last_loss_dec += L_dec.data\n last_loss_disc += L_disc.data\n count += 1\n if pic_freq > 0:\n assert type(pic_freq) == int, \"pic_freq must be an integer.\"\n if count % pic_freq == 0:\n fig = self._plot_img(\n c_samples,\n img_path=img_path,\n epoch=epoch\n )\n display(fig)\n\n if save_freq > 0:\n save_counter += 1\n assert type(save_freq) == int, \"save_freq must be an integer.\"\n if epoch % save_freq == 0:\n name = \"gan_epoch%s\" % str(epoch)\n if save_counter == 1:\n save_meta = True\n else:\n save_meta = False\n self.save(model_path, name, save_meta=save_meta)\n fig = self._plot_img(\n c_samples,\n img_path=img_path,\n epoch=epoch,\n batch=n_batches,\n save_pic=True\n )\n\n msg = \"dec_loss = {0} , disc_loss = {1}\"\n print(msg.format(last_loss_dec/n_batches, last_loss_disc/n_batches))\n t_diff = time.time()-t1\n print(\"time: %f\\n\\n\" % t_diff)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef save(self, path, name, save_meta=True):\n '''Saves model as a sequence of files in the format:\n {path}/{name}_{'dec', 'disc', 'dec_opt',\n 'disc_opt', 'meta'}.h5\n\n Parameters\n ----------\n path : str\n The directory of the file you wish to save the model to.\n name : str\n The name prefix of the model and optimizer files you wish\n to save.\n save_meta [optional] : bool\n Flag that controls whether to save the class metadata along with\n the generator, discriminator, and respective optimizer states.\n '''\n _save_model(self.dec, str(path), \"%s_dec\" % str(name))\n _save_model(self.disc, str(path), \"%s_disc\" % str(name))\n _save_model(self.dec_opt, str(path), \"%s_dec_opt\" % str(name))\n _save_model(self.disc_opt, str(path), \"%s_disc_opt\" % str(name))\n if save_meta:\n self._save_meta(os.path.join(path, \"%s_meta\" % str(name)))", "response": "Saves the current state of the class to a sequence of files in the format dec_opt meta. h5\n "} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef transform(self, data, test=False):\n '''Transform image data to latent space.\n\n Parameters\n ----------\n data : array-like shape (n_images, image_width, image_height,\n n_colors)\n Input numpy array of images.\n test [optional] : bool\n Controls the test boolean for batch normalization.\n\n Returns\n -------\n latent_vec : array-like shape (n_images, latent_width)\n '''\n #make sure that data has the right shape.\n if not type(data) == Variable:\n if len(data.shape) < 4:\n data = data[np.newaxis]\n if len(data.shape) != 4:\n raise TypeError(\"Invalid dimensions for image data. Dim = %s.\\\n Must be 4d array.\" % str(data.shape))\n if data.shape[1] != self.color_channels:\n if data.shape[-1] == self.color_channels:\n data = data.transpose(0, 3, 1, 2)\n else:\n raise TypeError(\"Invalid dimensions for image data. Dim = %s\"\n % str(data.shape))\n data = Variable(data)\n else:\n if len(data.data.shape) < 4:\n data.data = data.data[np.newaxis]\n if len(data.data.shape) != 4:\n raise TypeError(\"Invalid dimensions for image data. Dim = %s.\\\n Must be 4d array.\" % str(data.data.shape))\n if data.data.shape[1] != self.color_channels:\n if data.data.shape[-1] == self.color_channels:\n data.data = data.data.transpose(0, 3, 1, 2)\n else:\n raise TypeError(\"Invalid dimensions for image data. Dim = %s\"\n % str(data.shape))\n\n # Actual transformation.\n if self.flag_gpu:\n data.to_gpu()\n z = self._encode(data, test=test)[0]\n\n z.to_cpu()\n\n return z.data", "response": "Transform image data to latent space."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nfit the VAE model to the image data.", "response": "def fit(\n self,\n img_data,\n gamma=1.0,\n save_freq=-1,\n pic_freq=-1,\n n_epochs=100,\n batch_size=50,\n weight_decay=True,\n model_path='./VAEGAN_training_model/',\n img_path='./VAEGAN_training_images/',\n img_out_width=10,\n mirroring=False\n ):\n '''Fit the VAE/GAN model to the image data.\n\n Parameters\n ----------\n\n img_data : array-like shape (n_images, n_colors, image_width, image_height)\n Images used to fit VAE model.\n gamma [optional] : float\n Sets the multiplicative factor that weights the relative importance of\n reconstruction loss vs. ability to fool the discriminator. Higher weight\n means greater focus on faithful reconstruction.\n save_freq [optional] : int\n Sets the number of epochs to wait before saving the model and optimizer states.\n Also saves image files of randomly generated images using those states in a\n separate directory. Does not save if negative valued.\n pic_freq [optional] : int\n Sets the number of batches to wait before displaying a picture or randomly\n generated images using the current model state.\n Does not display if negative valued.\n n_epochs [optional] : int\n Gives the number of training epochs to run through for the fitting\n process.\n batch_size [optional] : int\n The size of the batch to use when training. Note: generally larger\n batch sizes will result in fater epoch iteration, but at the const\n of lower granulatity when updating the layer weights.\n weight_decay [optional] : bool\n Flag that controls adding weight decay hooks to the optimizer.\n model_path [optional] : str\n Directory where the model and optimizer state files will be saved.\n img_path [optional] : str\n Directory where the end of epoch training image files will be saved.\n img_out_width : int\n Controls the number of randomly genreated images per row in the output\n saved imags.\n mirroring [optional] : bool\n Controls whether images are randomly mirrored along the verical axis with\n a .5 probability. Artificially increases images variance for training set.\n '''\n width = img_out_width\n self.enc_opt.setup(self.enc)\n self.dec_opt.setup(self.dec)\n self.disc_opt.setup(self.disc)\n\n if weight_decay:\n self.enc_opt.add_hook(chainer.optimizer.WeightDecay(0.00001))\n self.dec_opt.add_hook(chainer.optimizer.WeightDecay(0.00001))\n self.disc_opt.add_hook(chainer.optimizer.WeightDecay(0.00001))\n\n n_data = img_data.shape[0]\n\n batch_iter = list(range(0, n_data, batch_size))\n n_batches = len(batch_iter)\n\n c_samples = np.random.standard_normal((width, self.latent_width)).astype(np.float32)\n save_counter = 0\n\n for epoch in range(1, n_epochs + 1):\n print('epoch: %i' % epoch)\n t1 = time.time()\n indexes = np.random.permutation(n_data)\n sum_l_enc = 0.\n sum_l_dec = 0.\n sum_l_disc = 0.\n\n sum_l_gan = 0.\n sum_l_like = 0.\n sum_l_prior = 0.\n count = 0\n for i in tqdm.tqdm(batch_iter):\n x = img_data[indexes[i: i + batch_size]]\n size = x.shape[0]\n if mirroring:\n for j in range(size):\n if np.random.randint(2):\n x[j, :, :, :] = x[j, :, :, ::-1]\n x_batch = Variable(x)\n zeros = Variable(np.zeros(size, dtype=np.int32))\n ones = Variable(np.ones(size, dtype=np.int32))\n\n if self.flag_gpu:\n x_batch.to_gpu()\n zeros.to_gpu()\n ones.to_gpu()\n\n kl_loss, dif_l, disc_rec, disc_batch, disc_samp = self._forward(x_batch)\n\n L_batch_GAN = F.softmax_cross_entropy(disc_batch, ones)\n L_rec_GAN = F.softmax_cross_entropy(disc_rec, zeros)\n L_samp_GAN = F.softmax_cross_entropy(disc_samp, zeros)\n\n l_gan = (L_batch_GAN + L_rec_GAN + L_samp_GAN)/3.\n l_like = dif_l\n l_prior = kl_loss\n\n enc_loss = self.kl_ratio*l_prior + l_like\n dec_loss = gamma*l_like - l_gan\n disc_loss = l_gan\n\n self.enc_opt.zero_grads()\n enc_loss.backward()\n self.enc_opt.update()\n\n self.dec_opt.zero_grads()\n dec_loss.backward()\n self.dec_opt.update()\n\n self.disc_opt.zero_grads()\n disc_loss.backward()\n self.disc_opt.update()\n\n sum_l_enc += enc_loss.data\n sum_l_dec += dec_loss.data\n sum_l_disc += disc_loss.data\n\n sum_l_gan += l_gan.data\n sum_l_like += l_like.data\n sum_l_prior += l_prior.data\n count += 1\n\n plot_data = img_data[indexes[:width]]\n if pic_freq > 0:\n assert type(pic_freq) == int, \"pic_freq must be an integer.\"\n if count % pic_freq == 0:\n fig = self._plot_img(\n plot_data,\n c_samples,\n img_path=img_path,\n epoch=epoch\n )\n display(fig)\n\n if save_freq > 0:\n save_counter += 1\n assert type(save_freq) == int, \"save_freq must be an integer.\"\n if epoch % save_freq == 0:\n name = \"vaegan_epoch%s\" % str(epoch)\n if save_counter == 1:\n save_meta = True\n else:\n save_meta = False\n self.save(model_path, name, save_meta=save_meta)\n fig = self._plot_img(\n plot_data,\n c_samples,\n img_path=img_path,\n epoch=epoch,\n batch=n_batches,\n save_pic=True\n )\n sum_l_enc /= n_batches\n sum_l_dec /= n_batches\n sum_l_disc /= n_batches\n sum_l_gan /= n_batches\n sum_l_like /= n_batches\n sum_l_prior /= n_batches\n msg = \"enc_loss = {0}, dec_loss = {1} , disc_loss = {2}\"\n msg2 = \"gan_loss = {0}, sim_loss = {1}, kl_loss = {2}\"\n print(msg.format(sum_l_enc, sum_l_dec, sum_l_disc))\n print(msg2.format(sum_l_gan, sum_l_like, sum_l_prior))\n t_diff = time.time()-t1\n print(\"time: %f\\n\\n\" % t_diff)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef load(cls, enc, dec, disc, enc_opt, dec_opt, disc_opt, meta, flag_gpu=None):\n '''Loads in model as a class instance with with the specified\n model and optimizer states.\n\n Parameters\n ----------\n enc : str\n Path to the encoder state file.\n dec : str\n Path to the decoder/generator state file.\n disc : str\n Path to the discriminator state file.\n enc_opt : str\n Path to the encoder optimizer state file.\n dec_opt : str\n Path to the decoder/generator optimizer state file.\n disc_opt : str\n Path to the discriminator optimizer state file.\n meta : str\n Path to the class metadata state file.\n flag_gpu : bool\n Specifies whether to load the model to use gpu capabilities.\n\n Returns\n -------\n\n class instance of self.\n '''\n mess = \"Model file {0} does not exist. Please check the file path.\"\n assert os.path.exists(enc), mess.format(enc)\n assert os.path.exists(dec), mess.format(dec)\n assert os.path.exists(disc), mess.format(disc)\n assert os.path.exists(dec_opt), mess.format(dec_opt)\n assert os.path.exists(disc_opt), mess.format(disc_opt)\n assert os.path.exists(meta), mess.format(meta)\n with open(meta, 'r') as f:\n meta = json.load(f)\n if flag_gpu is not None:\n meta['flag_gpu'] = flag_gpu\n\n loaded_class = cls(**meta)\n\n serializers.load_hdf5(enc, loaded_class.enc)\n serializers.load_hdf5(dec, loaded_class.dec)\n serializers.load_hdf5(disc, loaded_class.disc)\n loaded_class.enc_opt.setup(loaded_class.enc)\n loaded_class.dec_opt.setup(loaded_class.dec)\n loaded_class.disc_opt.setup(loaded_class.disc)\n serializers.load_hdf5(enc_opt, loaded_class.enc_opt)\n serializers.load_hdf5(dec_opt, loaded_class.dec_opt)\n serializers.load_hdf5(disc_opt, loaded_class.disc_opt)\n\n if meta['flag_gpu']:\n loaded_class.enc.to_gpu()\n loaded_class.dec.to_gpu()\n loaded_class.disc.to_gpu()\n\n return loaded_class", "response": "Loads in model as a class instance with the specified\n model and optimizer states."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef quadratic_jacobian_polynomial(nodes):\n # First evaluate the Jacobian at each of the 6 nodes.\n jac_parts = _helpers.matrix_product(nodes, _QUADRATIC_JACOBIAN_HELPER)\n jac_at_nodes = np.empty((1, 6), order=\"F\")\n jac_at_nodes[0, 0] = two_by_two_det(jac_parts[:, :2])\n jac_at_nodes[0, 1] = two_by_two_det(jac_parts[:, 2:4])\n jac_at_nodes[0, 2] = two_by_two_det(jac_parts[:, 4:6])\n jac_at_nodes[0, 3] = two_by_two_det(jac_parts[:, 6:8])\n jac_at_nodes[0, 4] = two_by_two_det(jac_parts[:, 8:10])\n jac_at_nodes[0, 5] = two_by_two_det(jac_parts[:, 10:])\n # Convert the nodal values to the Bernstein basis...\n bernstein = _helpers.matrix_product(jac_at_nodes, _QUADRATIC_TO_BERNSTEIN)\n return bernstein", "response": "r Compute the Jacobian determinant of a quadratic surface."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef cubic_jacobian_polynomial(nodes):\n # First evaluate the Jacobian at each of the 15 nodes\n # in the quartic triangle.\n jac_parts = _helpers.matrix_product(nodes, _CUBIC_JACOBIAN_HELPER)\n jac_at_nodes = np.empty((1, 15), order=\"F\")\n jac_at_nodes[0, 0] = two_by_two_det(jac_parts[:, :2])\n jac_at_nodes[0, 1] = two_by_two_det(jac_parts[:, 2:4])\n jac_at_nodes[0, 2] = two_by_two_det(jac_parts[:, 4:6])\n jac_at_nodes[0, 3] = two_by_two_det(jac_parts[:, 6:8])\n jac_at_nodes[0, 4] = two_by_two_det(jac_parts[:, 8:10])\n jac_at_nodes[0, 5] = two_by_two_det(jac_parts[:, 10:12])\n jac_at_nodes[0, 6] = two_by_two_det(jac_parts[:, 12:14])\n jac_at_nodes[0, 7] = two_by_two_det(jac_parts[:, 14:16])\n jac_at_nodes[0, 8] = two_by_two_det(jac_parts[:, 16:18])\n jac_at_nodes[0, 9] = two_by_two_det(jac_parts[:, 18:20])\n jac_at_nodes[0, 10] = two_by_two_det(jac_parts[:, 20:22])\n jac_at_nodes[0, 11] = two_by_two_det(jac_parts[:, 22:24])\n jac_at_nodes[0, 12] = two_by_two_det(jac_parts[:, 24:26])\n jac_at_nodes[0, 13] = two_by_two_det(jac_parts[:, 26:28])\n jac_at_nodes[0, 14] = two_by_two_det(jac_parts[:, 28:])\n # Convert the nodal values to the Bernstein basis...\n bernstein = _helpers.matrix_product(jac_at_nodes, _QUARTIC_TO_BERNSTEIN)\n bernstein /= _QUARTIC_BERNSTEIN_FACTOR\n return bernstein", "response": "r Compute the Jacobian determinant of a cubic surface."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _de_casteljau_one_round(nodes, degree, lambda1, lambda2, lambda3):\n dimension, num_nodes = nodes.shape\n num_new_nodes = num_nodes - degree - 1\n new_nodes = np.empty((dimension, num_new_nodes), order=\"F\")\n index = 0\n # parent_i1 = index + k\n # parent_i2 = index + k + 1\n # parent_i3 = index + degree + 1\n parent_i1 = 0\n parent_i2 = 1\n parent_i3 = degree + 1\n for k in six.moves.xrange(degree):\n for unused_j in six.moves.xrange(degree - k):\n # NOTE: i = (degree - 1) - j - k\n new_nodes[:, index] = (\n lambda1 * nodes[:, parent_i1]\n + lambda2 * nodes[:, parent_i2]\n + lambda3 * nodes[:, parent_i3]\n )\n # Update all the indices.\n parent_i1 += 1\n parent_i2 += 1\n parent_i3 += 1\n index += 1\n # Update the indices that depend on k.\n parent_i1 += 1\n parent_i2 += 1\n return new_nodes", "response": "r Performs one round of the de Casteljau algorithm for surfaces."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef make_transform(degree, weights_a, weights_b, weights_c):\n num_nodes = ((degree + 1) * (degree + 2)) // 2\n id_mat = np.eye(num_nodes, order=\"F\")\n # Pre-compute the matrices that do the reduction so we don't\n # have to **actually** perform the de Casteljau algorithm\n # every time.\n transform = {\n 0: de_casteljau_one_round(id_mat, degree, *weights_a),\n 1: de_casteljau_one_round(id_mat, degree, *weights_b),\n 2: de_casteljau_one_round(id_mat, degree, *weights_c),\n }\n return transform", "response": "Compute the matrix that computes the de Casteljau algorithm for the given degree."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef reduced_to_matrix(shape, degree, vals_by_weight):\n result = np.empty(shape, order=\"F\")\n index = 0\n for k in six.moves.xrange(degree + 1):\n for j in six.moves.xrange(degree + 1 - k):\n i = degree - j - k\n key = (0,) * i + (1,) * j + (2,) * k\n result[:, index] = vals_by_weight[key][:, 0]\n index += 1\n return result", "response": "r Converts a reduced values dictionary into a matrix."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _specialize_surface(nodes, degree, weights_a, weights_b, weights_c):\n # Uses A-->0, B-->1, C-->2 to represent the specialization used.\n partial_vals = {\n (0,): de_casteljau_one_round(nodes, degree, *weights_a),\n (1,): de_casteljau_one_round(nodes, degree, *weights_b),\n (2,): de_casteljau_one_round(nodes, degree, *weights_c),\n }\n for reduced_deg in six.moves.xrange(degree - 1, 0, -1):\n new_partial = {}\n transform = make_transform(\n reduced_deg, weights_a, weights_b, weights_c\n )\n for key, sub_nodes in six.iteritems(partial_vals):\n # Our keys are ascending so we increment from the last value.\n for next_id in six.moves.xrange(key[-1], 2 + 1):\n new_key = key + (next_id,)\n new_partial[new_key] = _helpers.matrix_product(\n sub_nodes, transform[next_id]\n )\n partial_vals = new_partial\n return reduced_to_matrix(nodes.shape, degree, partial_vals)", "response": "Specialize a surface to a specialization."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef _subdivide_nodes(nodes, degree):\n if degree == 1:\n nodes_a = _helpers.matrix_product(nodes, LINEAR_SUBDIVIDE_A)\n nodes_b = _helpers.matrix_product(nodes, LINEAR_SUBDIVIDE_B)\n nodes_c = _helpers.matrix_product(nodes, LINEAR_SUBDIVIDE_C)\n nodes_d = _helpers.matrix_product(nodes, LINEAR_SUBDIVIDE_D)\n elif degree == 2:\n nodes_a = _helpers.matrix_product(nodes, QUADRATIC_SUBDIVIDE_A)\n nodes_b = _helpers.matrix_product(nodes, QUADRATIC_SUBDIVIDE_B)\n nodes_c = _helpers.matrix_product(nodes, QUADRATIC_SUBDIVIDE_C)\n nodes_d = _helpers.matrix_product(nodes, QUADRATIC_SUBDIVIDE_D)\n elif degree == 3:\n nodes_a = _helpers.matrix_product(nodes, CUBIC_SUBDIVIDE_A)\n nodes_b = _helpers.matrix_product(nodes, CUBIC_SUBDIVIDE_B)\n nodes_c = _helpers.matrix_product(nodes, CUBIC_SUBDIVIDE_C)\n nodes_d = _helpers.matrix_product(nodes, CUBIC_SUBDIVIDE_D)\n elif degree == 4:\n nodes_a = _helpers.matrix_product(nodes, QUARTIC_SUBDIVIDE_A)\n nodes_b = _helpers.matrix_product(nodes, QUARTIC_SUBDIVIDE_B)\n nodes_c = _helpers.matrix_product(nodes, QUARTIC_SUBDIVIDE_C)\n nodes_d = _helpers.matrix_product(nodes, QUARTIC_SUBDIVIDE_D)\n else:\n nodes_a = specialize_surface(\n nodes,\n degree,\n _WEIGHTS_SUBDIVIDE0,\n _WEIGHTS_SUBDIVIDE1,\n _WEIGHTS_SUBDIVIDE2,\n )\n nodes_b = specialize_surface(\n nodes,\n degree,\n _WEIGHTS_SUBDIVIDE3,\n _WEIGHTS_SUBDIVIDE2,\n _WEIGHTS_SUBDIVIDE1,\n )\n nodes_c = specialize_surface(\n nodes,\n degree,\n _WEIGHTS_SUBDIVIDE1,\n _WEIGHTS_SUBDIVIDE4,\n _WEIGHTS_SUBDIVIDE3,\n )\n nodes_d = specialize_surface(\n nodes,\n degree,\n _WEIGHTS_SUBDIVIDE2,\n _WEIGHTS_SUBDIVIDE3,\n _WEIGHTS_SUBDIVIDE5,\n )\n return nodes_a, nodes_b, nodes_c, nodes_d", "response": "Subdivides a surface into four sub - surfaces."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _jacobian_both(nodes, degree, dimension):\n _, num_nodes = nodes.shape\n result = np.empty((2 * dimension, num_nodes - degree - 1), order=\"F\")\n result[:dimension, :] = jacobian_s(nodes, degree, dimension)\n result[dimension:, :] = jacobian_t(nodes, degree, dimension)\n return result", "response": "r Compute s and t partial of a Jacobian surfaces in a resource set."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef classify_tangent_intersection(\n intersection, nodes1, tangent1, nodes2, tangent2\n):\n \"\"\"Helper for func:`classify_intersection` at tangencies.\n\n .. note::\n\n This is a helper used only by :func:`classify_intersection`.\n\n Args:\n intersection (.Intersection): An intersection object.\n nodes1 (numpy.ndarray): Control points for the first curve at\n the intersection.\n tangent1 (numpy.ndarray): The tangent vector to the first curve\n at the intersection (``2 x 1`` array).\n nodes2 (numpy.ndarray): Control points for the second curve at\n the intersection.\n tangent2 (numpy.ndarray): The tangent vector to the second curve\n at the intersection (``2 x 1`` array).\n\n Returns:\n IntersectionClassification: The \"inside\" curve type, based on\n the classification enum. Will either be ``opposed`` or one\n of the ``tangent`` values.\n\n Raises:\n NotImplementedError: If the curves are tangent at the intersection\n and have the same curvature.\n \"\"\"\n # Each array is 2 x 1 (i.e. a column vector), we want the vector\n # dot product.\n dot_prod = np.vdot(tangent1[:, 0], tangent2[:, 0])\n # NOTE: When computing curvatures we assume that we don't have lines\n # here, because lines that are tangent at an intersection are\n # parallel and we don't handle that case.\n curvature1 = _curve_helpers.get_curvature(nodes1, tangent1, intersection.s)\n curvature2 = _curve_helpers.get_curvature(nodes2, tangent2, intersection.t)\n if dot_prod < 0:\n # If the tangent vectors are pointing in the opposite direction,\n # then the curves are facing opposite directions.\n sign1, sign2 = _SIGN([curvature1, curvature2])\n if sign1 == sign2:\n # If both curvatures are positive, since the curves are\n # moving in opposite directions, the tangency isn't part of\n # the surface intersection.\n if sign1 == 1.0:\n return CLASSIFICATION_T.OPPOSED\n\n else:\n return CLASSIFICATION_T.TANGENT_BOTH\n\n else:\n delta_c = abs(curvature1) - abs(curvature2)\n if delta_c == 0.0:\n raise NotImplementedError(_SAME_CURVATURE)\n\n elif sign1 == _SIGN(delta_c):\n return CLASSIFICATION_T.OPPOSED\n\n else:\n return CLASSIFICATION_T.TANGENT_BOTH\n\n else:\n if curvature1 > curvature2:\n return CLASSIFICATION_T.TANGENT_FIRST\n\n elif curvature1 < curvature2:\n return CLASSIFICATION_T.TANGENT_SECOND\n\n else:\n raise NotImplementedError(_SAME_CURVATURE)", "response": "This function returns the intersection class for the tangent curves at the given tangent vectors."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef ignored_edge_corner(edge_tangent, corner_tangent, corner_previous_edge):\n cross_prod = _helpers.cross_product(\n edge_tangent.ravel(order=\"F\"), corner_tangent.ravel(order=\"F\")\n )\n # A negative cross product indicates that ``edge_tangent`` is\n # \"inside\" / \"to the left\" of ``corner_tangent`` (due to right-hand rule).\n if cross_prod > 0.0:\n return False\n\n # Do the same for the **other** tangent at the corner.\n alt_corner_tangent = _curve_helpers.evaluate_hodograph(\n 1.0, corner_previous_edge\n )\n # Change the direction of the \"in\" tangent so that it points \"out\".\n alt_corner_tangent *= -1.0\n cross_prod = _helpers.cross_product(\n edge_tangent.ravel(order=\"F\"), alt_corner_tangent.ravel(order=\"F\")\n )\n return cross_prod <= 0.0", "response": "Check if an edge lies inside another edge."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\ncheck if an intersection is an ignored double corner.", "response": "def ignored_double_corner(\n intersection, tangent_s, tangent_t, edge_nodes1, edge_nodes2\n):\n \"\"\"Check if an intersection is an \"ignored\" double corner.\n\n .. note::\n\n This is a helper used only by :func:`ignored_corner`, which in turn is\n only used by :func:`classify_intersection`.\n\n Helper for :func:`ignored_corner` where both ``s`` and\n ``t`` are ``0``.\n\n Does so by checking if either edge through the ``t`` corner goes\n through the interior of the other surface. An interior check\n is done by checking that a few cross products are positive.\n\n Args:\n intersection (.Intersection): An intersection to \"diagnose\".\n tangent_s (numpy.ndarray): The tangent vector (``2 x 1`` array) to\n the first curve at the intersection.\n tangent_t (numpy.ndarray): The tangent vector (``2 x 1`` array) to\n the second curve at the intersection.\n edge_nodes1 (Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray]): The\n nodes of the three edges of the first surface being intersected.\n edge_nodes2 (Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray]): The\n nodes of the three edges of the second surface being intersected.\n\n Returns:\n bool: Indicates if the corner is to be ignored.\n \"\"\"\n # Compute the other edge for the ``s`` surface.\n prev_index = (intersection.index_first - 1) % 3\n prev_edge = edge_nodes1[prev_index]\n alt_tangent_s = _curve_helpers.evaluate_hodograph(1.0, prev_edge)\n # First check if ``tangent_t`` is interior to the ``s`` surface.\n cross_prod1 = _helpers.cross_product(\n tangent_s.ravel(order=\"F\"), tangent_t.ravel(order=\"F\")\n )\n # A positive cross product indicates that ``tangent_t`` is\n # interior to ``tangent_s``. Similar for ``alt_tangent_s``.\n # If ``tangent_t`` is interior to both, then the surfaces\n # do more than just \"kiss\" at the corner, so the corner should\n # not be ignored.\n if cross_prod1 >= 0.0:\n # Only compute ``cross_prod2`` if we need to.\n cross_prod2 = _helpers.cross_product(\n alt_tangent_s.ravel(order=\"F\"), tangent_t.ravel(order=\"F\")\n )\n if cross_prod2 >= 0.0:\n return False\n\n # If ``tangent_t`` is not interior, we check the other ``t``\n # edge that ends at the corner.\n prev_index = (intersection.index_second - 1) % 3\n prev_edge = edge_nodes2[prev_index]\n alt_tangent_t = _curve_helpers.evaluate_hodograph(1.0, prev_edge)\n # Change the direction of the \"in\" tangent so that it points \"out\".\n alt_tangent_t *= -1.0\n cross_prod3 = _helpers.cross_product(\n tangent_s.ravel(order=\"F\"), alt_tangent_t.ravel(order=\"F\")\n )\n if cross_prod3 >= 0.0:\n # Only compute ``cross_prod4`` if we need to.\n cross_prod4 = _helpers.cross_product(\n alt_tangent_s.ravel(order=\"F\"), alt_tangent_t.ravel(order=\"F\")\n )\n if cross_prod4 >= 0.0:\n return False\n\n # If neither of ``tangent_t`` or ``alt_tangent_t`` are interior\n # to the ``s`` surface, one of two things is true. Either\n # the two surfaces have no interior intersection (1) or the\n # ``s`` surface is bounded by both edges of the ``t`` surface\n # at the corner intersection (2). To detect (2), we only need\n # check if ``tangent_s`` is interior to both ``tangent_t``\n # and ``alt_tangent_t``. ``cross_prod1`` contains\n # (tangent_s) x (tangent_t), so it's negative will tell if\n # ``tangent_s`` is interior. Similarly, ``cross_prod3``\n # contains (tangent_s) x (alt_tangent_t), but we also reversed\n # the sign on ``alt_tangent_t`` so switching the sign back\n # and reversing the arguments in the cross product cancel out.\n return cross_prod1 > 0.0 or cross_prod3 < 0.0"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef ignored_corner(\n intersection, tangent_s, tangent_t, edge_nodes1, edge_nodes2\n):\n \"\"\"Check if an intersection is an \"ignored\" corner.\n\n .. note::\n\n This is a helper used only by :func:`classify_intersection`.\n\n An \"ignored\" corner is one where the surfaces just \"kiss\" at\n the point of intersection but their interiors do not meet.\n\n We can determine this by comparing the tangent lines from\n the point of intersection.\n\n .. note::\n\n This assumes the ``intersection`` has been shifted to the\n beginning of a curve so only checks if ``s == 0.0`` or ``t == 0.0``\n (rather than also checking for ``1.0``).\n\n Args:\n intersection (.Intersection): An intersection to \"diagnose\".\n tangent_s (numpy.ndarray): The tangent vector (``2 x 1`` array) to\n the first curve at the intersection.\n tangent_t (numpy.ndarray): The tangent vector (``2 x 1`` array) to\n the second curve at the intersection.\n edge_nodes1 (Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray]): The\n nodes of the three edges of the first surface being intersected.\n edge_nodes2 (Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray]): The\n nodes of the three edges of the second surface being intersected.\n\n Returns:\n bool: Indicates if the corner is to be ignored.\n \"\"\"\n if intersection.s == 0.0:\n if intersection.t == 0.0:\n # Double corner.\n return ignored_double_corner(\n intersection, tangent_s, tangent_t, edge_nodes1, edge_nodes2\n )\n\n else:\n # s-only corner.\n prev_index = (intersection.index_first - 1) % 3\n prev_edge = edge_nodes1[prev_index]\n return ignored_edge_corner(tangent_t, tangent_s, prev_edge)\n\n elif intersection.t == 0.0:\n # t-only corner.\n prev_index = (intersection.index_second - 1) % 3\n prev_edge = edge_nodes2[prev_index]\n return ignored_edge_corner(tangent_s, tangent_t, prev_edge)\n\n else:\n # Not a corner.\n return False", "response": "Check if an intersection is an ignored corner."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef handle_ends(index1, s, index2, t):\n edge_end = False\n if s == 1.0:\n s = 0.0\n index1 = (index1 + 1) % 3\n edge_end = True\n # NOTE: This is not a typo, the values can be updated twice if both ``s``\n # and ``t`` are ``1.0``\n if t == 1.0:\n t = 0.0\n index2 = (index2 + 1) % 3\n edge_end = True\n is_corner = s == 0.0 or t == 0.0\n return edge_end, is_corner, (index1, s, index2, t)", "response": "This function handles the end of an intersection between two nodes."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef to_front(intersection, intersections, unused):\n if intersection.s == 1.0:\n next_index = (intersection.index_first + 1) % 3\n # Make sure we haven't accidentally ignored an existing intersection.\n for other_int in intersections:\n if other_int.s == 0.0 and other_int.index_first == next_index:\n if other_int in unused:\n unused.remove(other_int)\n return other_int\n\n # If we haven't already returned, create **another** artificial\n # intersection.\n return _intersection_helpers.Intersection(\n next_index, 0.0, None, None, interior_curve=CLASSIFICATION_T.FIRST\n )\n\n elif intersection.t == 1.0:\n # NOTE: We assume, but do not check, that ``s == 1.0`` and ``t == 1.0``\n # are mutually exclusive.\n next_index = (intersection.index_second + 1) % 3\n # Make sure we haven't accidentally ignored an existing intersection.\n for other_int in intersections:\n if other_int.t == 0.0 and other_int.index_second == next_index:\n if other_int in unused:\n unused.remove(other_int)\n return other_int\n\n # If we haven't already returned, create **another** artificial\n # intersection.\n return _intersection_helpers.Intersection(\n None, None, next_index, 0.0, interior_curve=CLASSIFICATION_T.SECOND\n )\n\n else:\n return intersection", "response": "Rotates a node to the front."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef get_next_first(intersection, intersections, to_end=True):\n along_edge = None\n index_first = intersection.index_first\n s = intersection.s\n for other_int in intersections:\n other_s = other_int.s\n if other_int.index_first == index_first and other_s > s:\n if along_edge is None or other_s < along_edge.s:\n along_edge = other_int\n if along_edge is None:\n if to_end:\n # If there is no other intersection on the edge, just return\n # the segment end.\n return _intersection_helpers.Intersection(\n index_first,\n 1.0,\n None,\n None,\n interior_curve=CLASSIFICATION_T.FIRST,\n )\n\n else:\n return None\n\n else:\n return along_edge", "response": "Gets the next node along the current edge."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nget the next node along the current edge.", "response": "def get_next_second(intersection, intersections, to_end=True):\n \"\"\"Gets the next node along the current (second) edge.\n\n .. note::\n\n This is a helper used only by :func:`get_next`, which in\n turn is only used by :func:`basic_interior_combine`, which itself\n is only used by :func:`combine_intersections`.\n\n Along with :func:`get_next_first`, this function does the majority of the\n heavy lifting in :func:`get_next`. **Very** similar to\n :func:`get_next_first`, but this works with the second curve while the\n other function works with the first.\n\n Args:\n intersection (.Intersection): The current intersection.\n intersections (List[.Intersection]): List of all detected\n intersections, provided as a reference for potential\n points to arrive at.\n to_end (Optional[bool]): Indicates if the next node should just be\n the end of the first edge or :data:`None`.\n\n Returns:\n Optional[.Intersection]: The \"next\" point along a surface of\n intersection. This will produce the next intersection along the\n current (second) edge or the end of the same edge. If ``to_end`` is\n :data:`False` and there are no other intersections along the current\n edge, will return :data:`None` (rather than the end of the same edge).\n \"\"\"\n along_edge = None\n index_second = intersection.index_second\n t = intersection.t\n for other_int in intersections:\n other_t = other_int.t\n if other_int.index_second == index_second and other_t > t:\n if along_edge is None or other_t < along_edge.t:\n along_edge = other_int\n if along_edge is None:\n if to_end:\n # If there is no other intersection on the edge, just return\n # the segment end.\n return _intersection_helpers.Intersection(\n None,\n None,\n index_second,\n 1.0,\n interior_curve=CLASSIFICATION_T.SECOND,\n )\n\n else:\n return None\n\n else:\n return along_edge"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\ngets the next node along the current edge.", "response": "def get_next_coincident(intersection, intersections):\n \"\"\"Gets the next node along the current (coincident) edge.\n\n .. note::\n\n This is a helper used only by :func:`get_next`, which in\n turn is only used by :func:`basic_interior_combine`, which itself\n is only used by :func:`combine_intersections`.\n\n Along with :func:`get_next_first` and :func:`get_next_second`, this\n function does the majority of the heavy lifting in :func:`get_next`.\n\n This function moves immediately to the \"next\" intersection along the\n \"current\" edge. An intersection labeled as ``COINCIDENT`` can only occur\n at the beginning of a segment. The end will correspond to the ``s`` or\n ``t`` parameter being equal to ``1.0`` and so it will get \"rotated\"\n to the front of the next edge, where it will be classified according\n to a different edge pair.\n\n It's also worth noting that some coincident segments will correspond\n to curves moving in opposite directions. In that case, there is\n no \"interior\" intersection (the curves are facing away) and so that\n segment won't be part of an intersection.\n\n Args:\n intersection (.Intersection): The current intersection.\n intersections (List[.Intersection]): List of all detected\n intersections, provided as a reference for potential\n points to arrive at.\n\n Returns:\n .Intersection: The \"next\" point along a surface of intersection.\n This will produce the next intersection along the current (second)\n edge or the end of the same edge.\n \"\"\"\n # Moving along the first or second edge will produce the same result, but\n # since we \"rotate\" all intersections to the beginning of an edge, the\n # index may not match the first or second (or both).\n along_first = get_next_first(intersection, intersections, to_end=False)\n if along_first is None:\n along_second = get_next_second(\n intersection, intersections, to_end=False\n )\n else:\n return along_first\n\n if along_second is None:\n return _intersection_helpers.Intersection(\n intersection.index_first,\n 1.0,\n intersection.index_second,\n 1.0,\n interior_curve=CLASSIFICATION_T.COINCIDENT,\n )\n\n else:\n return along_second"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef get_next(intersection, intersections, unused):\n result = None\n if is_first(intersection.interior_curve):\n result = get_next_first(intersection, intersections)\n elif is_second(intersection.interior_curve):\n result = get_next_second(intersection, intersections)\n elif intersection.interior_curve == CLASSIFICATION_T.COINCIDENT:\n result = get_next_coincident(intersection, intersections)\n else:\n raise ValueError(\n 'Cannot get next node if not starting from \"FIRST\", '\n '\"TANGENT_FIRST\", \"SECOND\", \"TANGENT_SECOND\" or \"COINCIDENT\".'\n )\n\n if result in unused:\n unused.remove(result)\n return result", "response": "Gets the next node along a given edge."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nconverts a pair of intersection nodes to a base curve segment.", "response": "def ends_to_curve(start_node, end_node):\n \"\"\"Convert a \"pair\" of intersection nodes to a curve segment.\n\n .. note::\n\n This is a helper used only by :func:`basic_interior_combine`, which in\n turn is only used by :func:`combine_intersections`.\n\n .. note::\n\n This function could specialize to the first or second segment\n attached to ``start_node`` and ``end_node``. We determine\n first / second based on the classification of ``start_node``,\n but the callers of this function could provide that information /\n isolate the base curve and the two parameters for us.\n\n .. note::\n\n This only checks the classification of the ``start_node``.\n\n Args:\n start_node (.Intersection): The beginning of a segment.\n end_node (.Intersection): The end of (the same) segment.\n\n Returns:\n Tuple[int, float, float]: The 3-tuple of:\n\n * The edge index along the first surface (if in ``{0, 1, 2}``)\n or the edge index along the second surface shifted to the right by\n 3 (if in ``{3, 4, 5}``)\n * The start parameter along the edge\n * The end parameter along the edge\n\n Raises:\n ValueError: If the ``start_node`` and ``end_node`` disagree on\n the first curve when classified as \"FIRST\".\n ValueError: If the ``start_node`` and ``end_node`` disagree on\n the second curve when classified as \"SECOND\".\n ValueError: If the ``start_node`` and ``end_node`` disagree on\n the both curves when classified as \"COINCIDENT\".\n ValueError: If the ``start_node`` is not classified as\n :attr:`~.IntersectionClassification.FIRST`,\n :attr:`~.IntersectionClassification.TANGENT_FIRST`,\n :attr:`~.IntersectionClassification.SECOND`,\n :attr:`~.IntersectionClassification.TANGENT_SECOND` or\n :attr:`~.IntersectionClassification.COINCIDENT`.\n \"\"\"\n if is_first(start_node.interior_curve):\n if end_node.index_first != start_node.index_first:\n raise ValueError(_WRONG_CURVE)\n\n return start_node.index_first, start_node.s, end_node.s\n\n elif is_second(start_node.interior_curve):\n if end_node.index_second != start_node.index_second:\n raise ValueError(_WRONG_CURVE)\n\n return start_node.index_second + 3, start_node.t, end_node.t\n\n elif start_node.interior_curve == CLASSIFICATION_T.COINCIDENT:\n if end_node.index_first == start_node.index_first:\n return start_node.index_first, start_node.s, end_node.s\n\n elif end_node.index_second == start_node.index_second:\n return start_node.index_second + 3, start_node.t, end_node.t\n\n else:\n raise ValueError(_WRONG_CURVE)\n\n else:\n raise ValueError(\n 'Segment start must be classified as \"FIRST\", \"TANGENT_FIRST\", '\n '\"SECOND\", \"TANGENT_SECOND\" or \"COINCIDENT\".'\n )"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef no_intersections(nodes1, degree1, nodes2, degree2):\n # NOTE: This is a circular import.\n from bezier import _surface_intersection\n\n located = _surface_intersection.locate_point(\n nodes2, degree2, nodes1[0, 0], nodes1[1, 0]\n )\n if located is not None:\n return None, True\n\n located = _surface_intersection.locate_point(\n nodes1, degree1, nodes2[0, 0], nodes2[1, 0]\n )\n if located is not None:\n return None, False\n\n return [], None", "response": "r Determine if one surface is in the other."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef tangent_only_intersections(all_types):\n if len(all_types) != 1:\n raise ValueError(\"Unexpected value, types should all match\", all_types)\n\n point_type = all_types.pop()\n if point_type == CLASSIFICATION_T.OPPOSED:\n return [], None\n\n elif point_type == CLASSIFICATION_T.IGNORED_CORNER:\n return [], None\n\n elif point_type == CLASSIFICATION_T.TANGENT_FIRST:\n return None, True\n\n elif point_type == CLASSIFICATION_T.TANGENT_SECOND:\n return None, False\n\n elif point_type == CLASSIFICATION_T.COINCIDENT_UNUSED:\n return [], None\n\n else:\n raise ValueError(\"Point type not for tangency\", point_type)", "response": "Determine intersection in the case of only - tangent intersections."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef basic_interior_combine(intersections, max_edges=10):\n unused = intersections[:]\n result = []\n while unused:\n start = unused.pop()\n curr_node = start\n next_node = get_next(curr_node, intersections, unused)\n edge_ends = [(curr_node, next_node)]\n while next_node is not start:\n curr_node = to_front(next_node, intersections, unused)\n # NOTE: We also check to break when moving a corner node\n # to the front. This is because ``intersections``\n # de-duplicates corners by selecting the one\n # (of 2 or 4 choices) at the front of segment(s).\n if curr_node is start:\n break\n\n next_node = get_next(curr_node, intersections, unused)\n edge_ends.append((curr_node, next_node))\n if len(edge_ends) > max_edges:\n raise RuntimeError(\n \"Unexpected number of edges\", len(edge_ends)\n )\n\n edge_info = tuple(\n ends_to_curve(start_node, end_node)\n for start_node, end_node in edge_ends\n )\n result.append(edge_info)\n if len(result) == 1:\n if result[0] in FIRST_SURFACE_INFO:\n return None, True\n\n elif result[0] in SECOND_SURFACE_INFO:\n return None, False\n\n return result, None", "response": "Combine intersections that don t involve tangencies."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef combine_intersections(\n intersections, nodes1, degree1, nodes2, degree2, all_types\n):\n r\"\"\"Combine curve-curve intersections into curved polygon(s).\n\n .. note::\n\n This is a helper used only by :meth:`.Surface.intersect`.\n\n Does so assuming each intersection lies on an edge of one of\n two :class:`.Surface`-s.\n\n .. note ::\n\n This assumes that each ``intersection`` has been classified via\n :func:`classify_intersection` and only the intersections classified\n as ``FIRST`` and ``SECOND`` were kept.\n\n Args:\n intersections (List[.Intersection]): Intersections from each of the\n 9 edge-edge pairs from a surface-surface pairing.\n nodes1 (numpy.ndarray): The nodes defining the first surface in\n the intersection (assumed in :math:\\mathbf{R}^2`).\n degree1 (int): The degree of the surface given by ``nodes1``.\n nodes2 (numpy.ndarray): The nodes defining the second surface in\n the intersection (assumed in :math:\\mathbf{R}^2`).\n degree2 (int): The degree of the surface given by ``nodes2``.\n all_types (Set[.IntersectionClassification]): The set of all\n intersection classifications encountered among the intersections\n for the given surface-surface pair.\n\n Returns:\n Tuple[Optional[list], Optional[bool]]: Pair (2-tuple) of\n\n * List of \"edge info\" lists. Each list represents a curved polygon\n and contains 3-tuples of edge index, start and end (see the\n output of :func:`ends_to_curve`).\n * \"Contained\" boolean. If not :data:`None`, indicates\n that one of the surfaces is contained in the other.\n \"\"\"\n if intersections:\n return basic_interior_combine(intersections)\n\n elif all_types:\n return tangent_only_intersections(all_types)\n\n else:\n return no_intersections(nodes1, degree1, nodes2, degree2)", "response": "r Combine two curved polygons into one."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef _evaluate_barycentric(nodes, degree, lambda1, lambda2, lambda3):\n dimension, num_nodes = nodes.shape\n binom_val = 1.0\n result = np.zeros((dimension, 1), order=\"F\")\n index = num_nodes - 1\n result[:, 0] += nodes[:, index]\n # curve evaluate_multi_barycentric() takes arrays.\n lambda1 = np.asfortranarray([lambda1])\n lambda2 = np.asfortranarray([lambda2])\n for k in six.moves.xrange(degree - 1, -1, -1):\n # We want to go from (d C (k + 1)) to (d C k).\n binom_val = (binom_val * (k + 1)) / (degree - k)\n index -= 1 # Step to last element in column.\n # k = d - 1, d - 2, ...\n # d - k = 1, 2, ...\n # We know column k has (d - k + 1) elements.\n new_index = index - degree + k # First element in column.\n col_nodes = nodes[:, new_index : index + 1] # noqa: E203\n col_nodes = np.asfortranarray(col_nodes)\n col_result = _curve_helpers.evaluate_multi_barycentric(\n col_nodes, lambda1, lambda2\n )\n result *= lambda3\n result += binom_val * col_result\n # Update index for next iteration.\n index = new_index\n return result", "response": "r Evaluates a point on a single barycentric surface."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\ncompute the nodes of each edge of a base class.", "response": "def _compute_edge_nodes(nodes, degree):\n \"\"\"Compute the nodes of each edges of a surface.\n\n .. note::\n\n There is also a Fortran implementation of this function, which\n will be used if it can be built.\n\n Args:\n nodes (numpy.ndarray): Control point nodes that define the surface.\n degree (int): The degree of the surface define by ``nodes``.\n\n Returns:\n Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray]: The nodes in\n the edges of the surface.\n \"\"\"\n dimension, _ = np.shape(nodes)\n nodes1 = np.empty((dimension, degree + 1), order=\"F\")\n nodes2 = np.empty((dimension, degree + 1), order=\"F\")\n nodes3 = np.empty((dimension, degree + 1), order=\"F\")\n curr2 = degree\n curr3 = -1\n for i in six.moves.xrange(degree + 1):\n nodes1[:, i] = nodes[:, i]\n nodes2[:, i] = nodes[:, curr2]\n nodes3[:, i] = nodes[:, curr3]\n # Update the indices.\n curr2 += degree - i\n curr3 -= i + 2\n return nodes1, nodes2, nodes3"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef shoelace_for_area(nodes):\n _, num_nodes = nodes.shape\n if num_nodes == 2:\n shoelace = SHOELACE_LINEAR\n scale_factor = 2.0\n elif num_nodes == 3:\n shoelace = SHOELACE_QUADRATIC\n scale_factor = 6.0\n elif num_nodes == 4:\n shoelace = SHOELACE_CUBIC\n scale_factor = 20.0\n elif num_nodes == 5:\n shoelace = SHOELACE_QUARTIC\n scale_factor = 70.0\n else:\n raise _helpers.UnsupportedDegree(num_nodes - 1, supported=(1, 2, 3, 4))\n\n result = 0.0\n for multiplier, index1, index2 in shoelace:\n result += multiplier * (\n nodes[0, index1] * nodes[1, index2]\n - nodes[1, index1] * nodes[0, index2]\n )\n\n return result / scale_factor", "response": "r Compute an auxiliary shoelace sum used to compute area."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nsave an image to the docs images directory.", "response": "def save_image(figure, filename):\n \"\"\"Save an image to the docs images directory.\n\n Args:\n filename (str): The name of the file (not containing\n directory info).\n \"\"\"\n path = os.path.join(IMAGES_DIR, filename)\n figure.savefig(path, bbox_inches=\"tight\")\n plt.close(figure)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nfill out the columns of matrix with a series of points.", "response": "def stack1d(*points):\n \"\"\"Fill out the columns of matrix with a series of points.\n\n This is because ``np.hstack()`` will just make another 1D vector\n out of them and ``np.vstack()`` will put them in the rows.\n\n Args:\n points (Tuple[numpy.ndarray, ...]): Tuple of 1D points (i.e.\n arrays with shape ``(2,)``.\n\n Returns:\n numpy.ndarray: The array with each point in ``points`` as its\n columns.\n \"\"\"\n result = np.empty((2, len(points)), order=\"F\")\n for index, point in enumerate(points):\n result[:, index] = point\n return result"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef linearization_error(nodes):\n if NO_IMAGES:\n return\n\n curve = bezier.Curve.from_nodes(nodes)\n line = bezier.Curve.from_nodes(nodes[:, (0, -1)])\n midpoints = np.hstack([curve.evaluate(0.5), line.evaluate(0.5)])\n ax = curve.plot(256)\n line.plot(256, ax=ax)\n ax.plot(\n midpoints[0, :], midpoints[1, :], color=\"black\", linestyle=\"dashed\"\n )\n ax.axis(\"scaled\")\n save_image(ax.figure, \"linearization_error.png\")", "response": "Image for : func :. linearization_error docstring."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nimage for : func :. newton_refine", "response": "def newton_refine1(s, new_s, curve1, t, new_t, curve2):\n \"\"\"Image for :func:`.newton_refine` docstring.\"\"\"\n if NO_IMAGES:\n return\n\n points = np.hstack([curve1.evaluate(s), curve2.evaluate(t)])\n points_new = np.hstack([curve1.evaluate(new_s), curve2.evaluate(new_t)])\n ax = curve1.plot(256)\n curve2.plot(256, ax=ax)\n ax.plot(\n points[0, :],\n points[1, :],\n color=\"black\",\n linestyle=\"None\",\n marker=\"o\",\n markeredgewidth=1,\n markerfacecolor=\"None\",\n )\n ax.plot(\n points_new[0, :],\n points_new[1, :],\n color=\"black\",\n linestyle=\"None\",\n marker=\"o\",\n )\n ax.axis(\"scaled\")\n save_image(ax.figure, \"newton_refine1.png\")"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef newton_refine2(s_vals, curve1, curve2):\n if NO_IMAGES:\n return\n\n ax = curve1.plot(256)\n ax.lines[-1].zorder = 1\n curve2.plot(256, ax=ax)\n ax.lines[-1].zorder = 1\n points = curve1.evaluate_multi(np.asfortranarray(s_vals))\n colors = seaborn.dark_palette(\"blue\", 5)\n ax.scatter(\n points[0, :], points[1, :], c=colors, s=20, alpha=0.75, zorder=2\n )\n ax.axis(\"scaled\")\n ax.set_xlim(0.0, 1.0)\n ax.set_ylim(0.0, 1.0)\n save_image(ax.figure, \"newton_refine2.png\")", "response": "Image for : func :. newton_refine"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef segment_intersection1(start0, end0, start1, end1, s):\n if NO_IMAGES:\n return\n\n line0 = bezier.Curve.from_nodes(stack1d(start0, end0))\n line1 = bezier.Curve.from_nodes(stack1d(start1, end1))\n ax = line0.plot(2)\n line1.plot(256, ax=ax)\n (x_val,), (y_val,) = line0.evaluate(s)\n ax.plot([x_val], [y_val], color=\"black\", marker=\"o\")\n ax.axis(\"scaled\")\n save_image(ax.figure, \"segment_intersection1.png\")", "response": "Image for : func :. segment_intersection"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef segment_intersection2(start0, end0, start1, end1):\n if NO_IMAGES:\n return\n\n line0 = bezier.Curve.from_nodes(stack1d(start0, end0))\n line1 = bezier.Curve.from_nodes(stack1d(start1, end1))\n ax = line0.plot(2)\n line1.plot(2, ax=ax)\n ax.axis(\"scaled\")\n save_image(ax.figure, \"segment_intersection2.png\")", "response": "Image for : func :. segment_intersection docstring."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef helper_parallel_lines(start0, end0, start1, end1, filename):\n if NO_IMAGES:\n return\n\n figure = plt.figure()\n ax = figure.gca()\n points = stack1d(start0, end0, start1, end1)\n ax.plot(points[0, :2], points[1, :2], marker=\"o\")\n ax.plot(points[0, 2:], points[1, 2:], marker=\"o\")\n ax.axis(\"scaled\")\n _plot_helpers.add_plot_boundary(ax)\n save_image(figure, filename)", "response": "Image for parallel lines parameters docstring."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nimages for : class. Curve docstring.", "response": "def curve_constructor(curve):\n \"\"\"Image for :class`.Curve` docstring.\"\"\"\n if NO_IMAGES:\n return\n\n ax = curve.plot(256)\n line = ax.lines[0]\n nodes = curve._nodes\n ax.plot(\n nodes[0, :], nodes[1, :], color=\"black\", linestyle=\"None\", marker=\"o\"\n )\n add_patch(ax, nodes, line.get_color())\n ax.axis(\"scaled\")\n ax.set_xlim(-0.125, 1.125)\n ax.set_ylim(-0.0625, 0.5625)\n save_image(ax.figure, \"curve_constructor.png\")"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef curve_evaluate(curve):\n if NO_IMAGES:\n return\n\n ax = curve.plot(256)\n points = curve.evaluate_multi(np.asfortranarray([0.75]))\n ax.plot(\n points[0, :], points[1, :], color=\"black\", linestyle=\"None\", marker=\"o\"\n )\n ax.axis(\"scaled\")\n ax.set_xlim(-0.125, 1.125)\n ax.set_ylim(-0.0625, 0.5625)\n save_image(ax.figure, \"curve_evaluate.png\")", "response": "Image for : meth. Curve. evaluate docstring."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef curve_subdivide(curve, left, right):\n if NO_IMAGES:\n return\n\n figure = plt.figure()\n ax = figure.gca()\n add_patch(ax, curve._nodes, \"gray\")\n ax = left.plot(256, ax=ax)\n line = ax.lines[-1]\n add_patch(ax, left._nodes, line.get_color())\n right.plot(256, ax=ax)\n line = ax.lines[-1]\n add_patch(ax, right._nodes, line.get_color())\n ax.axis(\"scaled\")\n ax.set_xlim(-0.125, 2.125)\n ax.set_ylim(-0.125, 3.125)\n save_image(ax.figure, \"curve_subdivide.png\")", "response": "Image for : meth. Curve. subdivide docstring."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nimages for : meth. Curve. intersect docstring.", "response": "def curve_intersect(curve1, curve2, s_vals):\n \"\"\"Image for :meth`.Curve.intersect` docstring.\"\"\"\n if NO_IMAGES:\n return\n\n ax = curve1.plot(256)\n curve2.plot(256, ax=ax)\n intersections = curve1.evaluate_multi(s_vals)\n ax.plot(\n intersections[0, :],\n intersections[1, :],\n color=\"black\",\n linestyle=\"None\",\n marker=\"o\",\n )\n ax.axis(\"scaled\")\n ax.set_xlim(0.0, 0.75)\n ax.set_ylim(0.0, 0.75)\n save_image(ax.figure, \"curve_intersect.png\")"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef surface_constructor(surface):\n if NO_IMAGES:\n return\n\n ax = surface.plot(256, with_nodes=True)\n line = ax.lines[0]\n nodes = surface._nodes\n add_patch(ax, nodes[:, (0, 1, 2, 5)], line.get_color())\n delta = 1.0 / 32.0\n ax.text(\n nodes[0, 0],\n nodes[1, 0],\n r\"$v_0$\",\n fontsize=20,\n verticalalignment=\"top\",\n horizontalalignment=\"right\",\n )\n ax.text(\n nodes[0, 1],\n nodes[1, 1],\n r\"$v_1$\",\n fontsize=20,\n verticalalignment=\"top\",\n horizontalalignment=\"center\",\n )\n ax.text(\n nodes[0, 2],\n nodes[1, 2],\n r\"$v_2$\",\n fontsize=20,\n verticalalignment=\"top\",\n horizontalalignment=\"left\",\n )\n ax.text(\n nodes[0, 3] - delta,\n nodes[1, 3],\n r\"$v_3$\",\n fontsize=20,\n verticalalignment=\"center\",\n horizontalalignment=\"right\",\n )\n ax.text(\n nodes[0, 4] + delta,\n nodes[1, 4],\n r\"$v_4$\",\n fontsize=20,\n verticalalignment=\"center\",\n horizontalalignment=\"left\",\n )\n ax.text(\n nodes[0, 5],\n nodes[1, 5] + delta,\n r\"$v_5$\",\n fontsize=20,\n verticalalignment=\"bottom\",\n horizontalalignment=\"center\",\n )\n ax.axis(\"scaled\")\n ax.set_xlim(-0.125, 1.125)\n ax.set_ylim(-0.125, 1.125)\n save_image(ax.figure, \"surface_constructor.png\")", "response": "Image for : class. Surface docstring."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nimages for : meth. Surface. evaluate_barycentric docstring.", "response": "def surface_evaluate_barycentric(surface, point):\n \"\"\"Image for :meth`.Surface.evaluate_barycentric` docstring.\"\"\"\n if NO_IMAGES:\n return\n\n ax = surface.plot(256)\n ax.plot(\n point[0, :], point[1, :], color=\"black\", linestyle=\"None\", marker=\"o\"\n )\n ax.axis(\"scaled\")\n ax.set_xlim(-0.125, 1.125)\n ax.set_ylim(-0.125, 1.125)\n save_image(ax.figure, \"surface_evaluate_barycentric.png\")"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nimages for : meth. Surface. evaluate_cartesian_multi docstring.", "response": "def surface_evaluate_cartesian_multi(surface, points):\n \"\"\"Image for :meth`.Surface.evaluate_cartesian_multi` docstring.\"\"\"\n if NO_IMAGES:\n return\n\n ax = surface.plot(256)\n ax.plot(\n points[0, :], points[1, :], color=\"black\", linestyle=\"None\", marker=\"o\"\n )\n delta = 1.0 / 32.0\n font_size = 18\n ax.text(\n points[0, 0],\n points[1, 0],\n r\"$w_0$\",\n fontsize=font_size,\n verticalalignment=\"top\",\n horizontalalignment=\"right\",\n )\n ax.text(\n points[0, 1] + 2 * delta,\n points[1, 1],\n r\"$w_1$\",\n fontsize=font_size,\n verticalalignment=\"center\",\n horizontalalignment=\"left\",\n )\n ax.text(\n points[0, 2],\n points[1, 2] + delta,\n r\"$w_2$\",\n fontsize=font_size,\n verticalalignment=\"bottom\",\n horizontalalignment=\"left\",\n )\n ax.axis(\"scaled\")\n ax.set_xlim(-3.125, 2.375)\n ax.set_ylim(-0.25, 2.125)\n save_image(ax.figure, \"surface_evaluate_cartesian_multi.png\")"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nimage for : meth. Surface. is_valid docstring.", "response": "def surface_is_valid1(surface):\n \"\"\"Image for :meth`.Surface.is_valid` docstring.\"\"\"\n if NO_IMAGES:\n return\n\n ax = surface.plot(256)\n ax.axis(\"scaled\")\n ax.set_xlim(-0.125, 2.125)\n ax.set_ylim(-0.125, 2.125)\n save_image(ax.figure, \"surface_is_valid1.png\")"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nimage for : meth. Surface. is_valid", "response": "def surface_is_valid3(surface):\n \"\"\"Image for :meth`.Surface.is_valid` docstring.\"\"\"\n if NO_IMAGES:\n return\n\n edge1, edge2, edge3 = surface.edges\n N = 128\n # Compute points on each edge.\n std_s = np.linspace(0.0, 1.0, N + 1)\n points1 = edge1.evaluate_multi(std_s)\n points2 = edge2.evaluate_multi(std_s)\n points3 = edge3.evaluate_multi(std_s)\n # Compute the actual boundary where the Jacobian is 0.\n s_vals = np.linspace(0.0, 0.2, N)\n t_discrim = np.sqrt((1.0 - s_vals) * (1.0 - 5.0 * s_vals))\n t_top = 0.5 * (1.0 - s_vals + t_discrim)\n t_bottom = 0.5 * (1.0 - s_vals - t_discrim)\n jacobian_zero_params = np.zeros((2 * N - 1, 2), order=\"F\")\n jacobian_zero_params[:N, 0] = s_vals\n jacobian_zero_params[:N, 1] = t_top\n jacobian_zero_params[N:, 0] = s_vals[-2::-1]\n jacobian_zero_params[N:, 1] = t_bottom[-2::-1]\n jac_edge = surface.evaluate_cartesian_multi(jacobian_zero_params)\n # Add the surface to the plot and add a dashed line\n # for each \"true\" edge.\n figure = plt.figure()\n ax = figure.gca()\n line, = ax.plot(jac_edge[0, :], jac_edge[1, :])\n color = line.get_color()\n ax.plot(points1[0, :], points1[1, :], color=\"black\", linestyle=\"dashed\")\n ax.plot(points2[0, :], points2[1, :], color=\"black\", linestyle=\"dashed\")\n ax.plot(points3[0, :], points3[1, :], color=\"black\", linestyle=\"dashed\")\n polygon = np.hstack([points1[:, 1:], points2[:, 1:], jac_edge[:, 1:]])\n add_patch(ax, polygon, color, with_nodes=False)\n ax.axis(\"scaled\")\n ax.set_xlim(-0.0625, 1.0625)\n ax.set_ylim(-0.0625, 1.0625)\n save_image(ax.figure, \"surface_is_valid3.png\")"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nimaging for : meth. Surface. subdivide docstring.", "response": "def surface_subdivide1():\n \"\"\"Image for :meth`.Surface.subdivide` docstring.\"\"\"\n if NO_IMAGES:\n return\n\n surface = bezier.Surface.from_nodes(\n np.asfortranarray([[0.0, 1.0, 0.0], [0.0, 0.0, 1.0]])\n )\n surf_a, surf_b, surf_c, surf_d = surface.subdivide()\n figure, ((ax1, ax2), (ax3, ax4)) = plt.subplots(2, 2)\n for ax in (ax1, ax2, ax3, ax4):\n surface.plot(2, ax=ax)\n surf_a.plot(2, ax=ax1)\n ax1.text(\n 1.0 / 6.0,\n 1.0 / 6.0,\n r\"$A$\",\n fontsize=20,\n verticalalignment=\"center\",\n horizontalalignment=\"center\",\n )\n surf_b.plot(2, ax=ax2)\n ax2.text(\n 1.0 / 3.0,\n 1.0 / 3.0,\n r\"$B$\",\n fontsize=20,\n verticalalignment=\"center\",\n horizontalalignment=\"center\",\n )\n surf_c.plot(2, ax=ax3)\n ax3.text(\n 2.0 / 3.0,\n 1.0 / 6.0,\n r\"$C$\",\n fontsize=20,\n verticalalignment=\"center\",\n horizontalalignment=\"center\",\n )\n surf_d.plot(2, ax=ax4)\n ax4.text(\n 1.0 / 6.0,\n 2.0 / 3.0,\n r\"$D$\",\n fontsize=20,\n verticalalignment=\"center\",\n horizontalalignment=\"center\",\n )\n for ax in (ax1, ax2, ax3, ax4):\n ax.axis(\"scaled\")\n save_image(figure, \"surface_subdivide1\")"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef surface_subdivide2(surface, sub_surface_b):\n if NO_IMAGES:\n return\n\n # Plot set-up.\n figure = plt.figure()\n ax = figure.gca()\n colors = seaborn.husl_palette(6)\n N = 128\n s_vals = np.linspace(0.0, 1.0, N + 1)\n # Add edges from surface.\n add_edges(ax, surface, s_vals, colors[4])\n # Now do the same for surface B.\n add_edges(ax, sub_surface_b, s_vals, colors[0])\n # Add the control points polygon for the original surface.\n nodes = surface._nodes[:, (0, 2, 4, 5, 0)]\n add_patch(ax, nodes, colors[2], with_nodes=False)\n # Add the control points polygon for the sub-surface.\n nodes = sub_surface_b._nodes[:, (0, 1, 2, 5, 3, 0)]\n add_patch(ax, nodes, colors[1], with_nodes=False)\n # Plot **all** the nodes.\n sub_nodes = sub_surface_b._nodes\n ax.plot(\n sub_nodes[0, :],\n sub_nodes[1, :],\n color=\"black\",\n linestyle=\"None\",\n marker=\"o\",\n )\n # Take those same points and add the boundary.\n ax.plot(nodes[0, :], nodes[1, :], color=\"black\", linestyle=\"dashed\")\n ax.axis(\"scaled\")\n ax.set_xlim(-1.125, 2.125)\n ax.set_ylim(-0.125, 4.125)\n save_image(ax.figure, \"surface_subdivide2\")", "response": "Image for : meth. Surface. subdivide docstring."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef curved_polygon_constructor1(curved_poly):\n if NO_IMAGES:\n return\n\n ax = curved_poly.plot(256)\n ax.axis(\"scaled\")\n ax.set_xlim(-0.125, 2.125)\n ax.set_ylim(-0.625, 1.625)\n save_image(ax.figure, \"curved_polygon_constructor1.png\")", "response": "Image for : class. CurvedPolygon docstring."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef curve_specialize(curve, new_curve):\n if NO_IMAGES:\n return\n\n ax = curve.plot(256)\n interval = r\"$\\left[0, 1\\right]$\"\n line = ax.lines[-1]\n line.set_label(interval)\n color1 = line.get_color()\n new_curve.plot(256, ax=ax)\n interval = r\"$\\left[-\\frac{1}{4}, \\frac{3}{4}\\right]$\"\n line = ax.lines[-1]\n line.set_label(interval)\n ax.plot(\n curve._nodes[0, (0, -1)],\n curve._nodes[1, (0, -1)],\n color=color1,\n linestyle=\"None\",\n marker=\"o\",\n )\n ax.plot(\n new_curve._nodes[0, (0, -1)],\n new_curve._nodes[1, (0, -1)],\n color=line.get_color(),\n linestyle=\"None\",\n marker=\"o\",\n )\n ax.legend(loc=\"lower right\", fontsize=12)\n ax.axis(\"scaled\")\n ax.set_xlim(-0.375, 1.125)\n ax.set_ylim(-0.75, 0.625)\n save_image(ax.figure, \"curve_specialize.png\")", "response": "Image for : meth. Curve. specialize docstring."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nimages for the Newton - Refine surface.", "response": "def newton_refine_surface(surface, x_val, y_val, s, t, new_s, new_t):\n \"\"\"Image for :func:`._surface_helpers.newton_refine` docstring.\"\"\"\n if NO_IMAGES:\n return\n\n figure, (ax1, ax2) = plt.subplots(1, 2)\n # Plot features of the parameter space in ax1.\n tri_surf = bezier.Surface.from_nodes(\n np.asfortranarray([[0.0, 1.0, 0.0], [0.0, 0.0, 1.0]])\n )\n tri_surf.plot(2, ax=ax1)\n ax1.plot([0.25], [0.5], marker=\"H\")\n ax1.plot([s], [t], color=\"black\", linestyle=\"None\", marker=\"o\")\n ax1.plot(\n [new_s],\n [new_t],\n color=\"black\",\n linestyle=\"None\",\n marker=\"o\",\n markeredgewidth=1,\n markerfacecolor=\"None\",\n )\n # Plot the equivalent output in ax2.\n surface.plot(256, ax=ax2)\n points = surface.evaluate_cartesian_multi(\n np.asfortranarray([[s, t], [new_s, new_t]])\n )\n ax2.plot([x_val], [y_val], marker=\"H\")\n ax2.plot(\n points[0, [0]],\n points[1, [0]],\n color=\"black\",\n linestyle=\"None\",\n marker=\"o\",\n )\n ax2.plot(\n points[0, [1]],\n points[1, [1]],\n color=\"black\",\n linestyle=\"None\",\n marker=\"o\",\n markeredgewidth=1,\n markerfacecolor=\"None\",\n )\n # Set the axis bounds / scaling.\n ax1.axis(\"scaled\")\n ax1.set_xlim(-0.0625, 1.0625)\n ax1.set_ylim(-0.0625, 1.0625)\n ax2.axis(\"scaled\")\n ax2.set_xlim(-0.125, 2.125)\n ax2.set_ylim(-0.125, 2.125)\n save_image(figure, \"newton_refine_surface.png\")"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nimaging for :func:`._surface_helpers.classify_intersection` docstring.", "response": "def classify_intersection1(s, curve1, tangent1, curve2, tangent2):\n \"\"\"Image for :func:`._surface_helpers.classify_intersection` docstring.\"\"\"\n if NO_IMAGES:\n return\n\n surface1 = bezier.Surface.from_nodes(\n np.asfortranarray(\n [[1.0, 1.75, 2.0, 1.0, 1.5, 1.0], [0.0, 0.25, 1.0, 1.0, 1.5, 2.0]]\n )\n )\n surface2 = bezier.Surface.from_nodes(\n np.asfortranarray(\n [\n [0.0, 1.6875, 2.0, 0.25, 1.25, 0.5],\n [0.0, 0.0625, 0.5, 1.0, 1.25, 2.0],\n ]\n )\n )\n ax = classify_help(s, curve1, surface1, curve2, surface2, 0)\n (int_x,), (int_y,) = curve1.evaluate(s)\n # Remove the alpha from the color\n color1 = ax.patches[0].get_facecolor()[:3]\n color2 = ax.patches[1].get_facecolor()[:3]\n ax.plot(\n [int_x, int_x + tangent1[0, 0]],\n [int_y, int_y + tangent1[1, 0]],\n color=color1,\n linestyle=\"dashed\",\n )\n ax.plot(\n [int_x, int_x + tangent2[0, 0]],\n [int_y, int_y + tangent2[1, 0]],\n color=color2,\n linestyle=\"dashed\",\n )\n ax.plot([int_x], [int_y], color=color1, linestyle=\"None\", marker=\"o\")\n ax.axis(\"scaled\")\n ax.set_xlim(-0.125, 2.125)\n ax.set_ylim(-0.125, 1.125)\n save_image(ax.figure, \"classify_intersection1.png\")"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef classify_intersection2(s, curve1, curve2):\n if NO_IMAGES:\n return\n\n surface1 = bezier.Surface.from_nodes(\n np.asfortranarray(\n [[1.0, 1.5, 2.0, 1.25, 1.75, 1.5], [0.0, 1.0, 0.0, 1.0, 1.0, 2.0]]\n )\n )\n surface2 = bezier.Surface.from_nodes(\n np.asfortranarray(\n [[0.0, 1.5, 3.0, 0.75, 2.25, 1.5], [0.0, 1.0, 0.0, 2.0, 2.0, 4.0]]\n )\n )\n ax = classify_help(s, curve1, surface1, curve2, surface2, 1)\n ax.set_xlim(-0.0625, 3.0625)\n ax.set_ylim(-0.0625, 0.5625)\n save_image(ax.figure, \"classify_intersection2.png\")", "response": "Image for _surface_helpers. classify_intersection docstring."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nimaging for _surface_helpers. classify_intersection docstring.", "response": "def classify_intersection5(s, curve1, curve2):\n \"\"\"Image for :func:`._surface_helpers.classify_intersection` docstring.\"\"\"\n if NO_IMAGES:\n return\n\n surface1 = bezier.Surface.from_nodes(\n np.asfortranarray(\n [\n [1.0, 1.5, 2.0, 1.25, 1.75, 1.5],\n [0.0, 1.0, 0.0, 0.9375, 0.9375, 1.875],\n ]\n )\n )\n surface2 = bezier.Surface.from_nodes(\n np.asfortranarray(\n [\n [3.0, 1.5, 0.0, 2.25, 0.75, 1.5],\n [0.0, 1.0, 0.0, -2.0, -2.0, -4.0],\n ]\n )\n )\n figure, (ax1, ax2) = plt.subplots(2, 1)\n classify_help(s, curve1, surface1, curve2, surface2, 0, ax=ax1)\n classify_help(s, curve1, surface1, curve2, surface2, 1, ax=ax2)\n # Remove the alpha from the color\n color1 = ax1.patches[0].get_facecolor()[:3]\n color2 = ax1.patches[1].get_facecolor()[:3]\n # Now add the \"degenerate\" intersection polygons. The first\n # comes from specializing to\n # left1(0.5, 1.0)-left2(0.0, 0.25)-right1(0.375, 0.5)\n surface3 = bezier.Surface.from_nodes(\n np.asfortranarray(\n [\n [1.5, 1.75, 2.0, 1.6875, 1.9375, 1.875],\n [0.5, 0.5, 0.0, 0.5, 0.234375, 0.46875],\n ]\n )\n )\n # NOTE: We don't require the intersection polygon be valid.\n surface3.plot(256, ax=ax1)\n # The second comes from specializing to\n # left1(0.0, 0.5)-right1(0.5, 0.625)-left3(0.75, 1.0)\n surface4 = bezier.Surface.from_nodes(\n np.asfortranarray(\n [\n [1.0, 1.25, 1.5, 1.0625, 1.3125, 1.125],\n [0.0, 0.5, 0.5, 0.234375, 0.5, 0.46875],\n ]\n )\n )\n # NOTE: We don't require the intersection polygon be valid.\n surface4.plot(256, ax=ax2)\n (int_x,), (int_y,) = curve1.evaluate(s)\n ax1.plot([int_x], [int_y], color=color1, linestyle=\"None\", marker=\"o\")\n ax2.plot([int_x], [int_y], color=color2, linestyle=\"None\", marker=\"o\")\n for ax in (ax1, ax2):\n ax.axis(\"scaled\")\n ax.set_xlim(-0.0625, 3.0625)\n ax.set_ylim(-0.0625, 0.5625)\n plt.setp(ax1.get_xticklabels(), visible=False)\n figure.tight_layout(h_pad=-7.0)\n save_image(figure, \"classify_intersection5.png\")"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef classify_intersection7(s, curve1a, curve1b, curve2):\n if NO_IMAGES:\n return\n\n surface1 = bezier.Surface.from_nodes(\n np.asfortranarray(\n [\n [0.0, 4.5, 9.0, 0.0, 4.5, 0.0],\n [0.0, 0.0, 2.25, 1.25, 2.375, 2.5],\n ]\n )\n )\n surface2 = bezier.Surface.from_nodes(\n np.asfortranarray(\n [\n [11.25, 9.0, 2.75, 8.125, 3.875, 5.0],\n [0.0, 4.5, 1.0, -0.75, -0.25, -1.5],\n ]\n )\n )\n figure, (ax1, ax2) = plt.subplots(2, 1)\n classify_help(s, curve1a, surface1, curve2, surface2, None, ax=ax1)\n surface1._nodes = np.asfortranarray(surface1._nodes[:, (2, 4, 5, 1, 3, 0)])\n surface1._edges = None\n classify_help(0.0, curve1b, surface1, curve2, surface2, 0, ax=ax2)\n for ax in (ax1, ax2):\n ax.set_xlim(-0.125, 11.5)\n ax.set_ylim(-0.125, 2.625)\n plt.setp(ax1.get_xticklabels(), visible=False)\n figure.tight_layout(h_pad=-5.0)\n save_image(figure, \"classify_intersection7.png\")", "response": "Image for _surface_helpers. classify_intersection7 docstring."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef get_curvature(nodes, s, tangent_vec, curvature):\n if NO_IMAGES:\n return\n\n curve = bezier.Curve.from_nodes(nodes)\n # Find the center of the circle along the direction\n # perpendicular to the tangent vector (90 degree left turn).\n radius_dir = np.asfortranarray([[-tangent_vec[1, 0]], [tangent_vec[0, 0]]])\n radius_dir /= np.linalg.norm(radius_dir, ord=2)\n point = curve.evaluate(s)\n circle_center = point + radius_dir / curvature\n # Add the curve.\n ax = curve.plot(256)\n # Add the circle.\n circle_center = circle_center.ravel(order=\"F\")\n circle = plt.Circle(circle_center, 1.0 / abs(curvature), alpha=0.25)\n ax.add_artist(circle)\n # Add the point.\n ax.plot(\n point[0, :], point[1, :], color=\"black\", marker=\"o\", linestyle=\"None\"\n )\n ax.axis(\"scaled\")\n ax.set_xlim(-0.0625, 1.0625)\n ax.set_ylim(-0.0625, 0.625)\n save_image(ax.figure, \"get_curvature.png\")", "response": "Image for getting the curvature of a node."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nimaging for : meth. Curve. locate docstring.", "response": "def curve_locate(curve, point1, point2, point3):\n \"\"\"Image for :meth`.Curve.locate` docstring.\"\"\"\n if NO_IMAGES:\n return\n\n ax = curve.plot(256)\n points = np.hstack([point1, point2, point3])\n ax.plot(\n points[0, :], points[1, :], color=\"black\", linestyle=\"None\", marker=\"o\"\n )\n ax.axis(\"scaled\")\n ax.set_xlim(-0.8125, 0.0625)\n ax.set_ylim(0.75, 2.0625)\n save_image(ax.figure, \"curve_locate.png\")"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef newton_refine_curve(curve, point, s, new_s):\n if NO_IMAGES:\n return\n\n ax = curve.plot(256)\n ax.plot(point[0, :], point[1, :], marker=\"H\")\n wrong_points = curve.evaluate_multi(np.asfortranarray([s, new_s]))\n ax.plot(\n wrong_points[0, [0]],\n wrong_points[1, [0]],\n color=\"black\",\n linestyle=\"None\",\n marker=\"o\",\n )\n ax.plot(\n wrong_points[0, [1]],\n wrong_points[1, [1]],\n color=\"black\",\n linestyle=\"None\",\n marker=\"o\",\n markeredgewidth=1,\n markerfacecolor=\"None\",\n )\n # Set the axis bounds / scaling.\n ax.axis(\"scaled\")\n ax.set_xlim(-0.125, 3.125)\n ax.set_ylim(-0.125, 1.375)\n save_image(ax.figure, \"newton_refine_curve.png\")", "response": "Image for the Newton - Refine curve."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef newton_refine_curve_cusp(curve, s_vals):\n if NO_IMAGES:\n return\n\n ax = curve.plot(256)\n ax.lines[-1].zorder = 1\n points = curve.evaluate_multi(np.asfortranarray(s_vals))\n colors = seaborn.dark_palette(\"blue\", 6)\n ax.scatter(\n points[0, :], points[1, :], c=colors, s=20, alpha=0.75, zorder=2\n )\n # Set the axis bounds / scaling.\n ax.axis(\"scaled\")\n ax.set_xlim(-0.125, 6.125)\n ax.set_ylim(-3.125, 3.125)\n save_image(ax.figure, \"newton_refine_curve_cusp.png\")", "response": "Image for the Newton Reine curve."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef classify_intersection8(s, curve1, surface1, curve2, surface2):\n if NO_IMAGES:\n return\n\n ax = classify_help(s, curve1, surface1, curve2, surface2, None)\n ax.set_xlim(-1.125, 1.125)\n ax.set_ylim(-0.125, 1.125)\n save_image(ax.figure, \"classify_intersection8.png\")", "response": "Image for classify_intersection8 docstring."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef _edges_classify_intersection9():\n edges1 = (\n bezier.Curve.from_nodes(\n np.asfortranarray([[32.0, 30.0], [20.0, 25.0]])\n ),\n bezier.Curve.from_nodes(\n np.asfortranarray([[30.0, 25.0, 20.0], [25.0, 20.0, 20.0]])\n ),\n bezier.Curve.from_nodes(\n np.asfortranarray([[20.0, 25.0, 30.0], [20.0, 20.0, 15.0]])\n ),\n bezier.Curve.from_nodes(\n np.asfortranarray([[30.0, 32.0], [15.0, 20.0]])\n ),\n )\n edges2 = (\n bezier.Curve.from_nodes(\n np.asfortranarray([[8.0, 10.0], [20.0, 15.0]])\n ),\n bezier.Curve.from_nodes(\n np.asfortranarray([[10.0, 15.0, 20.0], [15.0, 20.0, 20.0]])\n ),\n bezier.Curve.from_nodes(\n np.asfortranarray([[20.0, 15.0, 10.0], [20.0, 20.0, 25.0]])\n ),\n bezier.Curve.from_nodes(\n np.asfortranarray([[10.0, 8.0], [25.0, 20.0]])\n ),\n )\n return edges1, edges2", "response": "The edges for the curved polygon intersection used below."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nimages for _surface_helpers. classify_intersection docstring.", "response": "def classify_intersection9(s, curve1, curve2):\n \"\"\"Image for :func:`._surface_helpers.classify_intersection` docstring.\"\"\"\n if NO_IMAGES:\n return\n\n surface1 = bezier.Surface.from_nodes(\n np.asfortranarray(\n [\n [0.0, 20.0, 40.0, 10.0, 30.0, 20.0],\n [0.0, 40.0, 0.0, 25.0, 25.0, 50.0],\n ]\n )\n )\n surface2 = bezier.Surface.from_nodes(\n np.asfortranarray(\n [\n [40.0, 20.0, 0.0, 30.0, 10.0, 20.0],\n [40.0, 0.0, 40.0, 15.0, 15.0, -10.0],\n ]\n )\n )\n figure, (ax1, ax2) = plt.subplots(1, 2)\n classify_help(s, curve1, surface1, curve2, surface2, 0, ax=ax1)\n classify_help(s, curve1, surface1, curve2, surface2, 1, ax=ax2)\n # Remove the alpha from the color\n color1 = ax1.patches[0].get_facecolor()[:3]\n color2 = ax1.patches[1].get_facecolor()[:3]\n # Now add the \"degenerate\" intersection polygons.\n cp_edges1, cp_edges2 = _edges_classify_intersection9()\n curved_polygon1 = bezier.CurvedPolygon(*cp_edges1)\n curved_polygon1.plot(256, ax=ax1)\n curved_polygon2 = bezier.CurvedPolygon(*cp_edges2)\n curved_polygon2.plot(256, ax=ax2)\n (int_x,), (int_y,) = curve1.evaluate(s)\n ax1.plot([int_x], [int_y], color=color1, linestyle=\"None\", marker=\"o\")\n ax2.plot([int_x], [int_y], color=color2, linestyle=\"None\", marker=\"o\")\n for ax in (ax1, ax2):\n ax.axis(\"scaled\")\n ax.set_xlim(-2.0, 42.0)\n ax.set_ylim(-12.0, 52.0)\n plt.setp(ax2.get_yticklabels(), visible=False)\n figure.tight_layout(w_pad=1.0)\n save_image(figure, \"classify_intersection9.png\")"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef curve_elevate(curve, elevated):\n if NO_IMAGES:\n return\n\n figure, (ax1, ax2) = plt.subplots(1, 2)\n curve.plot(256, ax=ax1)\n color = ax1.lines[-1].get_color()\n add_patch(ax1, curve._nodes, color)\n elevated.plot(256, ax=ax2)\n color = ax2.lines[-1].get_color()\n add_patch(ax2, elevated._nodes, color)\n ax1.axis(\"scaled\")\n ax2.axis(\"scaled\")\n _plot_helpers.add_plot_boundary(ax1)\n ax2.set_xlim(*ax1.get_xlim())\n ax2.set_ylim(*ax1.get_ylim())\n save_image(figure, \"curve_elevate.png\")", "response": "Image for : meth :. curve. Curve. elevate docstring."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef unit_triangle():\n if NO_IMAGES:\n return\n\n nodes = np.asfortranarray([[0.0, 1.0, 0.0], [0.0, 0.0, 1.0]])\n surface = bezier.Surface(nodes, degree=1)\n ax = surface.plot(256)\n ax.axis(\"scaled\")\n _plot_helpers.add_plot_boundary(ax)\n save_image(ax.figure, \"unit_triangle.png\")", "response": "Image for : class :. surface. Surface docstring."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef curve_reduce(curve, reduced):\n if NO_IMAGES:\n return\n\n figure, (ax1, ax2) = plt.subplots(1, 2, sharex=True, sharey=True)\n curve.plot(256, ax=ax1)\n color = ax1.lines[-1].get_color()\n add_patch(ax1, curve._nodes, color)\n reduced.plot(256, ax=ax2)\n color = ax2.lines[-1].get_color()\n add_patch(ax2, reduced._nodes, color)\n ax1.axis(\"scaled\")\n ax2.axis(\"scaled\")\n _plot_helpers.add_plot_boundary(ax2)\n save_image(figure, \"curve_reduce.png\")", "response": "Image for : meth :. curve. Curve. reduce docstring."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nimaging for : meth :. curve. Curve. reduce docstring.", "response": "def curve_reduce_approx(curve, reduced):\n \"\"\"Image for :meth:`.curve.Curve.reduce` docstring.\"\"\"\n if NO_IMAGES:\n return\n\n ax = curve.plot(256)\n color = ax.lines[-1].get_color()\n add_patch(ax, curve._nodes, color, alpha=0.25, node_color=color)\n reduced.plot(256, ax=ax)\n color = ax.lines[-1].get_color()\n add_patch(ax, reduced._nodes, color, alpha=0.25, node_color=color)\n ax.axis(\"scaled\")\n _plot_helpers.add_plot_boundary(ax)\n save_image(ax.figure, \"curve_reduce_approx.png\")"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef update_locate_candidates(candidate, next_candidates, x_val, y_val, degree):\n centroid_x, centroid_y, width, candidate_nodes = candidate\n point = np.asfortranarray([x_val, y_val])\n if not _helpers.contains_nd(candidate_nodes, point):\n return\n\n nodes_a, nodes_b, nodes_c, nodes_d = _surface_helpers.subdivide_nodes(\n candidate_nodes, degree\n )\n half_width = 0.5 * width\n next_candidates.extend(\n (\n (\n centroid_x - half_width,\n centroid_y - half_width,\n half_width,\n nodes_a,\n ),\n (centroid_x, centroid_y, -half_width, nodes_b),\n (centroid_x + width, centroid_y - half_width, half_width, nodes_c),\n (centroid_x - half_width, centroid_y + width, half_width, nodes_d),\n )\n )", "response": "This function updates the list of candidates during geometric search for a point."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef mean_centroid(candidates):\n sum_x = 0.0\n sum_y = 0.0\n for centroid_x, centroid_y, _, _ in candidates:\n sum_x += centroid_x\n sum_y += centroid_y\n denom = 3.0 * len(candidates)\n return sum_x / denom, sum_y / denom", "response": "Take the mean of all centroids in set of reference triangles."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _locate_point(nodes, degree, x_val, y_val):\n # We track the centroid rather than base_x/base_y/width (by storing triple\n # the centroid -- to avoid division by three until needed). We also need\n # to track the width (or rather, just the sign of the width).\n candidates = [(1.0, 1.0, 1.0, nodes)]\n for _ in six.moves.xrange(MAX_LOCATE_SUBDIVISIONS + 1):\n next_candidates = []\n for candidate in candidates:\n update_locate_candidates(\n candidate, next_candidates, x_val, y_val, degree\n )\n candidates = next_candidates\n if not candidates:\n return None\n\n # We take the average of all centroids from the candidates\n # that may contain the point.\n s_approx, t_approx = mean_centroid(candidates)\n s, t = newton_refine(nodes, degree, x_val, y_val, s_approx, t_approx)\n actual = _surface_helpers.evaluate_barycentric(\n nodes, degree, 1.0 - s - t, s, t\n )\n expected = np.asfortranarray([x_val, y_val])\n if not _helpers.vector_close(\n actual.ravel(order=\"F\"), expected, eps=LOCATE_EPS\n ):\n s, t = newton_refine(nodes, degree, x_val, y_val, s, t)\n return s, t", "response": "r Finds a point on a base - image of a given degree on a given set of nodes."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef same_intersection(intersection1, intersection2, wiggle=0.5 ** 40):\n if intersection1.index_first != intersection2.index_first:\n return False\n\n if intersection1.index_second != intersection2.index_second:\n return False\n\n return np.allclose(\n [intersection1.s, intersection1.t],\n [intersection2.s, intersection2.t],\n atol=0.0,\n rtol=wiggle,\n )", "response": "Check if two intersections are close to machine precision."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nverifies that a set of intersections had expected duplicates.", "response": "def verify_duplicates(duplicates, uniques):\n \"\"\"Verify that a set of intersections had expected duplicates.\n\n .. note::\n\n This is a helper used only by :func:`generic_intersect`.\n\n Args:\n duplicates (List[.Intersection]): List of intersections\n corresponding to duplicates that were filtered out.\n uniques (List[.Intersection]): List of \"final\" intersections\n with duplicates filtered out.\n\n Raises:\n ValueError: If the ``uniques`` are not actually all unique.\n ValueError: If one of the ``duplicates`` does not correspond to\n an intersection in ``uniques``.\n ValueError: If a duplicate occurs only once but does not have\n exactly one of ``s`` and ``t`` equal to ``0.0``.\n ValueError: If a duplicate occurs three times but does not have\n exactly both ``s == t == 0.0``.\n ValueError: If a duplicate occurs a number other than one or three\n times.\n \"\"\"\n for uniq1, uniq2 in itertools.combinations(uniques, 2):\n if same_intersection(uniq1, uniq2):\n raise ValueError(\"Non-unique intersection\")\n\n counter = collections.Counter()\n for dupe in duplicates:\n matches = []\n for index, uniq in enumerate(uniques):\n if same_intersection(dupe, uniq):\n matches.append(index)\n if len(matches) != 1:\n raise ValueError(\"Duplicate not among uniques\", dupe)\n\n matched = matches[0]\n counter[matched] += 1\n for index, count in six.iteritems(counter):\n uniq = uniques[index]\n if count == 1:\n if (uniq.s, uniq.t).count(0.0) != 1:\n raise ValueError(\"Count == 1 should be a single corner\", uniq)\n\n elif count == 3:\n if (uniq.s, uniq.t) != (0.0, 0.0):\n raise ValueError(\"Count == 3 should be a double corner\", uniq)\n\n else:\n raise ValueError(\"Unexpected duplicate count\", count)"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef verify_edge_segments(edge_infos):\n if edge_infos is None:\n return\n\n for edge_info in edge_infos:\n num_segments = len(edge_info)\n for index in six.moves.xrange(-1, num_segments - 1):\n index1, start1, end1 = edge_info[index]\n # First, verify the start and end parameters for the current\n # segment.\n if not 0.0 <= start1 < end1 <= 1.0:\n raise ValueError(BAD_SEGMENT_PARAMS, edge_info[index])\n\n # Then, verify that the indices are not the same.\n index2, _, _ = edge_info[index + 1]\n if index1 == index2:\n raise ValueError(\n SEGMENTS_SAME_EDGE, edge_info[index], edge_info[index + 1]\n )", "response": "Verify that the edge segments in an intersection are valid."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nadding an intersection that is COINCIDENT_UNUSED but on an edge end.", "response": "def add_edge_end_unused(intersection, duplicates, intersections):\n \"\"\"Add intersection that is ``COINCIDENT_UNUSED`` but on an edge end.\n\n This is a helper for :func:`~._surface_intersection.add_intersection`.\n It assumes that\n\n * ``intersection`` will have at least one of ``s == 0.0`` or ``t == 0.0``\n * A \"misclassified\" intersection in ``intersections`` that matches\n ``intersection`` will be the \"same\" if it matches both ``index_first``\n and ``index_second`` and if it matches the start index exactly\n\n Args:\n intersection (.Intersection): An intersection to be added.\n duplicates (List[.Intersection]): List of duplicate intersections.\n intersections (List[.Intersection]): List of \"accepted\" (i.e.\n non-duplicate) intersections.\n \"\"\"\n found = None\n for other in intersections:\n if (\n intersection.index_first == other.index_first\n and intersection.index_second == other.index_second\n ):\n if intersection.s == 0.0 and other.s == 0.0:\n found = other\n break\n\n if intersection.t == 0.0 and other.t == 0.0:\n found = other\n break\n\n if found is not None:\n intersections.remove(found)\n duplicates.append(found)\n intersections.append(intersection)"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef check_unused(intersection, duplicates, intersections):\n for other in intersections:\n if (\n other.interior_curve == UNUSED_T\n and intersection.index_first == other.index_first\n and intersection.index_second == other.index_second\n ):\n if intersection.s == 0.0 and other.s == 0.0:\n duplicates.append(intersection)\n return True\n\n if intersection.t == 0.0 and other.t == 0.0:\n duplicates.append(intersection)\n return True\n\n return False", "response": "Checks if a valid intersection is already in intersections."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nadds an intersection to the current node list.", "response": "def add_intersection( # pylint: disable=too-many-arguments\n index1,\n s,\n index2,\n t,\n interior_curve,\n edge_nodes1,\n edge_nodes2,\n duplicates,\n intersections,\n):\n \"\"\"Create an :class:`Intersection` and append.\n\n The intersection will be classified as either a duplicate or a valid\n intersection and appended to one of ``duplicates`` or ``intersections``\n depending on that classification.\n\n Args:\n index1 (int): The index (among 0, 1, 2) of the first edge in the\n intersection.\n s (float): The parameter along the first curve of the intersection.\n index2 (int): The index (among 0, 1, 2) of the second edge in the\n intersection.\n t (float): The parameter along the second curve of the intersection.\n interior_curve (Optional[.IntersectionClassification]): The\n classification of the intersection, if known. If :data:`None`,\n the classification will be computed below.\n edge_nodes1 (Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray]): The\n nodes of the three edges of the first surface being intersected.\n edge_nodes2 (Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray]): The\n nodes of the three edges of the second surface being intersected.\n duplicates (List[.Intersection]): List of duplicate intersections.\n intersections (List[.Intersection]): List of \"accepted\" (i.e.\n non-duplicate) intersections.\n \"\"\"\n # NOTE: There is no corresponding \"enable\", but the disable only applies\n # in this lexical scope.\n # pylint: disable=too-many-locals\n edge_end, is_corner, intersection_args = _surface_helpers.handle_ends(\n index1, s, index2, t\n )\n if edge_end:\n intersection = _intersection_helpers.Intersection(*intersection_args)\n intersection.interior_curve = interior_curve\n if interior_curve == UNUSED_T:\n add_edge_end_unused(intersection, duplicates, intersections)\n else:\n duplicates.append(intersection)\n else:\n intersection = _intersection_helpers.Intersection(index1, s, index2, t)\n if is_corner:\n is_duplicate = check_unused(\n intersection, duplicates, intersections\n )\n if is_duplicate:\n return\n\n # Classify the intersection.\n if interior_curve is None:\n interior_curve = _surface_helpers.classify_intersection(\n intersection, edge_nodes1, edge_nodes2\n )\n intersection.interior_curve = interior_curve\n intersections.append(intersection)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef classify_coincident(st_vals, coincident):\n if not coincident:\n return None\n\n if st_vals[0, 0] >= st_vals[0, 1] or st_vals[1, 0] >= st_vals[1, 1]:\n return UNUSED_T\n\n else:\n return CLASSIFICATION_T.COINCIDENT", "response": "r Determine if the intersection parameters of two curves are used."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nchecks if an intersection can be used as part of a curved polygon.", "response": "def should_use(intersection):\n \"\"\"Check if an intersection can be used as part of a curved polygon.\n\n Will return :data:`True` if the intersection is classified as\n :attr:`~.IntersectionClassification.FIRST`,\n :attr:`~.IntersectionClassification.SECOND` or\n :attr:`~.IntersectionClassification.COINCIDENT` or if the intersection\n is classified is a corner / edge end which is classified as\n :attr:`~.IntersectionClassification.TANGENT_FIRST` or\n :attr:`~.IntersectionClassification.TANGENT_SECOND`.\n\n Args:\n intersection (.Intersection): An intersection to be added.\n\n Returns:\n bool: Indicating if the intersection will be used.\n \"\"\"\n if intersection.interior_curve in ACCEPTABLE_CLASSIFICATIONS:\n return True\n\n if intersection.interior_curve in TANGENT_CLASSIFICATIONS:\n return intersection.s == 0.0 or intersection.t == 0.0\n\n return False"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef surface_intersections(edge_nodes1, edge_nodes2, all_intersections):\n # NOTE: There is no corresponding \"enable\", but the disable only applies\n # in this lexical scope.\n # pylint: disable=too-many-locals\n intersections = []\n duplicates = []\n for index1, nodes1 in enumerate(edge_nodes1):\n for index2, nodes2 in enumerate(edge_nodes2):\n st_vals, coincident = all_intersections(nodes1, nodes2)\n interior_curve = classify_coincident(st_vals, coincident)\n for s, t in st_vals.T:\n add_intersection(\n index1,\n s,\n index2,\n t,\n interior_curve,\n edge_nodes1,\n edge_nodes2,\n duplicates,\n intersections,\n )\n all_types = set()\n to_keep = []\n unused = []\n for intersection in intersections:\n all_types.add(intersection.interior_curve)\n # Only keep the intersections which are \"acceptable\".\n if should_use(intersection):\n to_keep.append(intersection)\n else:\n unused.append(intersection)\n return to_keep, duplicates, unused, all_types", "response": "Find all intersections among edges of two surfaces."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef generic_intersect(\n nodes1, degree1, nodes2, degree2, verify, all_intersections\n):\n r\"\"\"Find all intersections among edges of two surfaces.\n\n This treats intersections which have ``s == 1.0`` or ``t == 1.0``\n as duplicates. The duplicates will be checked by :func:`verify_duplicates`\n if ``verify`` is :data:`True`.\n\n Args:\n nodes1 (numpy.ndarray): The nodes defining the first surface in\n the intersection (assumed in :math:\\mathbf{R}^2`).\n degree1 (int): The degree of the surface given by ``nodes1``.\n nodes2 (numpy.ndarray): The nodes defining the second surface in\n the intersection (assumed in :math:\\mathbf{R}^2`).\n degree2 (int): The degree of the surface given by ``nodes2``.\n verify (Optional[bool]): Indicates if duplicate intersections\n should be checked.\n all_intersections (Callable): A helper that intersects B |eacute| zier\n curves. Takes the nodes of each curve as input and returns an\n array (``2 x N``) of intersections.\n\n Returns:\n Tuple[Optional[list], Optional[bool], tuple]: 3-tuple of\n\n * List of \"edge info\" lists. Each list represents a curved polygon\n and contains 3-tuples of edge index, start and end (see the\n output of :func:`ends_to_curve`).\n * \"Contained\" boolean. If not :data:`None`, indicates\n that one of the surfaces is contained in the other.\n * The nodes of three edges of the first surface being intersected\n followed by the nodes of the three edges of the second.\n \"\"\"\n bbox_int = _geometric_intersection.bbox_intersect(nodes1, nodes2)\n if bbox_int != INTERSECTION_T:\n return [], None, ()\n\n # We need **all** edges (as nodes).\n edge_nodes1 = _surface_helpers.compute_edge_nodes(nodes1, degree1)\n edge_nodes2 = _surface_helpers.compute_edge_nodes(nodes2, degree2)\n # Run through **all** pairs of edges.\n intersections, duplicates, unused, all_types = surface_intersections(\n edge_nodes1, edge_nodes2, all_intersections\n )\n edge_infos, contained = _surface_helpers.combine_intersections(\n intersections, nodes1, degree1, nodes2, degree2, all_types\n )\n # Verify the duplicates and intersection if need be.\n if verify:\n verify_duplicates(duplicates, intersections + unused)\n verify_edge_segments(edge_infos)\n if edge_infos is None or edge_infos == []:\n return edge_infos, contained, ()\n\n return edge_infos, contained, edge_nodes1 + edge_nodes2", "response": "r Returns a list of lists of edges that intersect the two surfaces."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef _geometric_intersect(nodes1, degree1, nodes2, degree2, verify):\n all_intersections = _geometric_intersection.all_intersections\n return generic_intersect(\n nodes1, degree1, nodes2, degree2, verify, all_intersections\n )", "response": "r Returns a tuple of lists containing intersections between two surfaces."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef algebraic_intersect(nodes1, degree1, nodes2, degree2, verify):\n all_intersections = _algebraic_intersection.all_intersections\n return generic_intersect(\n nodes1, degree1, nodes2, degree2, verify, all_intersections\n )", "response": "r Returns a tuple of lists containing all intersections among the two surfaces."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nmodify the module search path.", "response": "def modify_path():\n \"\"\"Modify the module search path.\"\"\"\n # Only modify path on Windows.\n if os.name != \"nt\":\n return\n\n path = os.environ.get(\"PATH\")\n if path is None:\n return\n\n try:\n extra_dll_dir = pkg_resources.resource_filename(\"bezier\", \"extra-dll\")\n if os.path.isdir(extra_dll_dir):\n os.environ[\"PATH\"] = path + os.pathsep + extra_dll_dir\n except ImportError:\n pass"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef handle_import_error(caught_exc, name):\n for template in TEMPLATES:\n expected_msg = template.format(name)\n if caught_exc.args == (expected_msg,):\n return\n\n raise caught_exc", "response": "Allow or re - raise an import error."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nchecking if the current build is gfortran on macOS.", "response": "def is_macos_gfortran(f90_compiler):\n \"\"\"Checks if the current build is ``gfortran`` on macOS.\n\n Args:\n f90_compiler (numpy.distutils.fcompiler.FCompiler): A Fortran compiler\n instance.\n\n Returns:\n bool: Only :data:`True` if\n\n * Current OS is macOS (checked via ``sys.platform``).\n * ``f90_compiler`` corresponds to ``gfortran``.\n \"\"\"\n # NOTE: NumPy may not be installed, but we don't want **this** module to\n # cause an import failure.\n from numpy.distutils.fcompiler import gnu\n\n # Only macOS.\n if sys.platform != MAC_OS:\n return False\n\n # Only ``gfortran``.\n if not isinstance(f90_compiler, gnu.Gnu95FCompiler):\n return False\n\n return True"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\npatch up the Fortran Gnu95FCompiler s library_dirs.", "response": "def patch_f90_compiler(f90_compiler):\n \"\"\"Patch up ``f90_compiler.library_dirs``.\n\n On macOS, a Homebrew installed ``gfortran`` needs some help. The\n ``numpy.distutils`` \"default\" constructor for ``Gnu95FCompiler`` only has\n a single library search path, but there are many library paths included in\n the full ``gcc`` install.\n\n Args:\n f90_compiler (numpy.distutils.fcompiler.FCompiler): A Fortran compiler\n instance.\n \"\"\"\n if not is_macos_gfortran(f90_compiler):\n return\n\n library_dirs = f90_compiler.library_dirs\n # ``library_dirs`` is a list (i.e. mutable), so we can update in place.\n library_dirs[:] = setup_helpers.gfortran_search_path(library_dirs)"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nconverting a description of edges into a curved polygon.", "response": "def _make_intersection(edge_info, all_edge_nodes):\n \"\"\"Convert a description of edges into a curved polygon.\n\n .. note::\n\n This is a helper used only by :meth:`.Surface.intersect`.\n\n Args:\n edge_info (Tuple[Tuple[int, float, float], ...]): Information\n describing each edge in the curved polygon by indicating which\n surface / edge on the surface and then start and end parameters\n along that edge. (See :func:`.ends_to_curve`.)\n all_edge_nodes (Tuple[numpy.ndarray, ...]): The nodes of three edges\n of the first surface being intersected followed by the nodes of\n the three edges of the second.\n\n Returns:\n .CurvedPolygon: The intersection corresponding to ``edge_info``.\n \"\"\"\n edges = []\n for index, start, end in edge_info:\n nodes = all_edge_nodes[index]\n new_nodes = _curve_helpers.specialize_curve(nodes, start, end)\n degree = new_nodes.shape[1] - 1\n edge = _curve_mod.Curve(new_nodes, degree, _copy=False)\n edges.append(edge)\n return curved_polygon.CurvedPolygon(\n *edges, metadata=edge_info, _verify=False\n )"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef from_nodes(cls, nodes, _copy=True):\n _, num_nodes = nodes.shape\n degree = cls._get_degree(num_nodes)\n return cls(nodes, degree, _copy=_copy)", "response": "Create a : class :. Surface from a numpy array of nodes."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _get_degree(num_nodes):\n # 8 * num_nodes = 4(d + 1)(d + 2)\n # = 4d^2 + 12d + 8\n # = (2d + 3)^2 - 1\n d_float = 0.5 * (np.sqrt(8.0 * num_nodes + 1.0) - 3.0)\n d_int = int(np.round(d_float))\n if (d_int + 1) * (d_int + 2) == 2 * num_nodes:\n return d_int\n\n else:\n raise ValueError(num_nodes, \"not a triangular number\")", "response": "Get the degree of the current surface."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef area(self):\n if self._dimension != 2:\n raise NotImplementedError(\n \"2D is the only supported dimension\",\n \"Current dimension\",\n self._dimension,\n )\n\n edge1, edge2, edge3 = self._get_edges()\n return _surface_helpers.compute_area(\n (edge1._nodes, edge2._nodes, edge3._nodes)\n )", "response": "Calculates the area of the current surface."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef _compute_edges(self):\n nodes1, nodes2, nodes3 = _surface_helpers.compute_edge_nodes(\n self._nodes, self._degree\n )\n edge1 = _curve_mod.Curve(nodes1, self._degree, _copy=False)\n edge2 = _curve_mod.Curve(nodes2, self._degree, _copy=False)\n edge3 = _curve_mod.Curve(nodes3, self._degree, _copy=False)\n return edge1, edge2, edge3", "response": "Compute the edges of the current node and edge of the current node."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nget the edges for the current surface.", "response": "def _get_edges(self):\n \"\"\"Get the edges for the current surface.\n\n If they haven't been computed yet, first compute and store them.\n\n This is provided as a means for internal calls to get the edges\n without copying (since :attr:`.edges` copies before giving to\n a user to keep the stored data immutable).\n\n Returns:\n Tuple[~bezier.curve.Curve, ~bezier.curve.Curve, \\\n ~bezier.curve.Curve]: The edges of\n the surface.\n \"\"\"\n if self._edges is None:\n self._edges = self._compute_edges()\n return self._edges"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _verify_barycentric(lambda1, lambda2, lambda3):\n weights_total = lambda1 + lambda2 + lambda3\n if not np.allclose(weights_total, 1.0, atol=0.0):\n raise ValueError(\n \"Weights do not sum to 1\", lambda1, lambda2, lambda3\n )\n\n if lambda1 < 0.0 or lambda2 < 0.0 or lambda3 < 0.0:\n raise ValueError(\n \"Weights must be positive\", lambda1, lambda2, lambda3\n )", "response": "Verifies that the weights are barycentric and on the reference triangle."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef evaluate_barycentric_multi(self, param_vals, _verify=True):\n if _verify:\n if param_vals.ndim != 2:\n raise ValueError(\"Parameter values must be 2D array\")\n\n for lambda1, lambda2, lambda3 in param_vals:\n self._verify_barycentric(lambda1, lambda2, lambda3)\n return _surface_helpers.evaluate_barycentric_multi(\n self._nodes, self._degree, param_vals, self._dimension\n )", "response": "r This method evaluates multiple points on the surface."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _verify_cartesian(s, t):\n if s < 0.0 or t < 0.0 or s + t > 1.0:\n raise ValueError(\"Point lies outside reference triangle\", s, t)", "response": "Verifies that a point is in the reference triangle."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef evaluate_cartesian(self, s, t, _verify=True):\n if _verify:\n self._verify_cartesian(s, t)\n return _surface_helpers.evaluate_barycentric(\n self._nodes, self._degree, 1.0 - s - t, s, t\n )", "response": "r Returns a point on the surface."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef plot(self, pts_per_edge, color=None, ax=None, with_nodes=False):\n if self._dimension != 2:\n raise NotImplementedError(\n \"2D is the only supported dimension\",\n \"Current dimension\",\n self._dimension,\n )\n\n if ax is None:\n ax = _plot_helpers.new_axis()\n _plot_helpers.add_patch(ax, color, pts_per_edge, *self._get_edges())\n if with_nodes:\n ax.plot(\n self._nodes[0, :],\n self._nodes[1, :],\n color=\"black\",\n marker=\"o\",\n linestyle=\"None\",\n )\n return ax", "response": "Plots the current control points of the current object."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef subdivide(self):\n nodes_a, nodes_b, nodes_c, nodes_d = _surface_helpers.subdivide_nodes(\n self._nodes, self._degree\n )\n return (\n Surface(nodes_a, self._degree, _copy=False),\n Surface(nodes_b, self._degree, _copy=False),\n Surface(nodes_c, self._degree, _copy=False),\n Surface(nodes_d, self._degree, _copy=False),\n )", "response": "Split the surface into four sub - surfaces."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef locate(self, point, _verify=True):\n if _verify:\n if self._dimension != 2:\n raise NotImplementedError(\"Only 2D surfaces supported.\")\n\n if point.shape != (self._dimension, 1):\n point_dimensions = \" x \".join(\n str(dimension) for dimension in point.shape\n )\n msg = _LOCATE_ERROR_TEMPLATE.format(\n self._dimension, self._dimension, point, point_dimensions\n )\n raise ValueError(msg)\n\n return _surface_intersection.locate_point(\n self._nodes, self._degree, point[0, 0], point[1, 0]\n )", "response": "r Returns the s t and t values corresponding to the point on the current surface."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef intersect(self, other, strategy=_STRATEGY.GEOMETRIC, _verify=True):\n if _verify:\n if not isinstance(other, Surface):\n raise TypeError(\n \"Can only intersect with another surface\",\n \"Received\",\n other,\n )\n\n if self._dimension != 2 or other._dimension != 2:\n raise NotImplementedError(\n \"Intersection only implemented in 2D\"\n )\n\n if strategy == _STRATEGY.GEOMETRIC:\n do_intersect = _surface_intersection.geometric_intersect\n elif strategy == _STRATEGY.ALGEBRAIC:\n do_intersect = _surface_intersection.algebraic_intersect\n else:\n raise ValueError(\"Unexpected strategy.\", strategy)\n\n edge_infos, contained, all_edge_nodes = do_intersect(\n self._nodes, self._degree, other._nodes, other._degree, _verify\n )\n if edge_infos is None:\n if contained:\n return [self]\n\n else:\n return [other]\n\n else:\n return [\n _make_intersection(edge_info, all_edge_nodes)\n for edge_info in edge_infos\n ]", "response": "Find the common intersection with another surface."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef elevate(self):\n _, num_nodes = self._nodes.shape\n # (d + 1)(d + 2)/2 --> (d + 2)(d + 3)/2\n num_new = num_nodes + self._degree + 2\n new_nodes = np.zeros((self._dimension, num_new), order=\"F\")\n # NOTE: We start from the index triples (i, j, k) for the current\n # nodes and map them onto (i + 1, j, k), etc. This index\n # tracking is also done in :func:`.de_casteljau_one_round`.\n index = 0\n # parent_i1 = index + k\n # parent_i2 = index + k + 1\n # parent_i3 = index + degree + 2\n parent_i1 = 0\n parent_i2 = 1\n parent_i3 = self._degree + 2\n for k in six.moves.xrange(self._degree + 1):\n for j in six.moves.xrange(self._degree + 1 - k):\n i = self._degree - j - k\n new_nodes[:, parent_i1] += (i + 1) * self._nodes[:, index]\n new_nodes[:, parent_i2] += (j + 1) * self._nodes[:, index]\n new_nodes[:, parent_i3] += (k + 1) * self._nodes[:, index]\n # Update all the indices.\n parent_i1 += 1\n parent_i2 += 1\n parent_i3 += 1\n index += 1\n # Update the indices that depend on k.\n parent_i1 += 1\n parent_i2 += 1\n # Hold off on division until the end, to (attempt to) avoid round-off.\n denominator = self._degree + 1.0\n new_nodes /= denominator\n return Surface(new_nodes, self._degree + 1, _copy=False)", "response": "r Return a degree - elevated version of the current surface."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef _newton_refine(s, nodes1, t, nodes2):\n # NOTE: We form -F(s, t) since we want to solve -DF^{-1} F(s, t).\n func_val = _curve_helpers.evaluate_multi(\n nodes2, np.asfortranarray([t])\n ) - _curve_helpers.evaluate_multi(nodes1, np.asfortranarray([s]))\n if np.all(func_val == 0.0):\n # No refinement is needed.\n return s, t\n\n # NOTE: This assumes the curves are 2D.\n jac_mat = np.empty((2, 2), order=\"F\")\n jac_mat[:, :1] = _curve_helpers.evaluate_hodograph(s, nodes1)\n jac_mat[:, 1:] = -_curve_helpers.evaluate_hodograph(t, nodes2)\n # Solve the system.\n singular, delta_s, delta_t = _helpers.solve2x2(jac_mat, func_val[:, 0])\n if singular:\n raise ValueError(\"Jacobian is singular.\")\n\n else:\n return s + delta_s, t + delta_t", "response": "r Apply one step of 2D Newton s method."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\ngetting the library directory paths for ``gfortran``. Looks for ``libraries: =`` in the output of ``gfortran -print-search-dirs`` and then parses the paths. If this fails for any reason, this method will print an error and return ``library_dirs``. Args: library_dirs (List[str]): Existing library directories. Returns: List[str]: The library directories for ``gfortran``.", "response": "def gfortran_search_path(library_dirs):\n \"\"\"Get the library directory paths for ``gfortran``.\n\n Looks for ``libraries: =`` in the output of ``gfortran -print-search-dirs``\n and then parses the paths. If this fails for any reason, this method will\n print an error and return ``library_dirs``.\n\n Args:\n library_dirs (List[str]): Existing library directories.\n\n Returns:\n List[str]: The library directories for ``gfortran``.\n \"\"\"\n cmd = (\"gfortran\", \"-print-search-dirs\")\n process = subprocess.Popen(cmd, stdout=subprocess.PIPE)\n return_code = process.wait()\n # Bail out if the command failed.\n if return_code != 0:\n return library_dirs\n\n cmd_output = process.stdout.read().decode(\"utf-8\")\n # Find single line starting with ``libraries: ``.\n search_lines = cmd_output.strip().split(\"\\n\")\n library_lines = [\n line[len(FORTRAN_LIBRARY_PREFIX) :]\n for line in search_lines\n if line.startswith(FORTRAN_LIBRARY_PREFIX)\n ]\n if len(library_lines) != 1:\n msg = GFORTRAN_MISSING_LIBS.format(cmd_output)\n print(msg, file=sys.stderr)\n return library_dirs\n\n # Go through each library in the ``libraries: = ...`` line.\n library_line = library_lines[0]\n accepted = set(library_dirs)\n for part in library_line.split(os.pathsep):\n full_path = os.path.abspath(part.strip())\n if os.path.isdir(full_path):\n accepted.add(full_path)\n else:\n # Ignore anything that isn't a directory.\n msg = GFORTRAN_BAD_PATH.format(full_path)\n print(msg, file=sys.stderr)\n return sorted(accepted)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nupdate a given set of compiler flags.", "response": "def _update_flags(compiler_flags, remove_flags=()):\n \"\"\"Update a given set of compiler flags.\n\n Args:\n compiler_flags (List[str]): Existing flags associated with a compiler.\n remove_flags (Optional[Container[str]]): A container of flags to remove\n that will override any of the defaults.\n\n Returns:\n List[str]: The modified list (i.e. some flags added and some removed).\n \"\"\"\n for flag in GFORTRAN_SHARED_FLAGS:\n if flag not in compiler_flags:\n compiler_flags.append(flag)\n if DEBUG_ENV in os.environ:\n to_add = GFORTRAN_DEBUG_FLAGS\n to_remove = GFORTRAN_OPTIMIZE_FLAGS\n else:\n to_add = GFORTRAN_OPTIMIZE_FLAGS\n if os.environ.get(WHEEL_ENV) is None:\n to_add += (GFORTRAN_NATIVE_FLAG,)\n to_remove = GFORTRAN_DEBUG_FLAGS\n for flag in to_add:\n if flag not in compiler_flags:\n compiler_flags.append(flag)\n return [\n flag\n for flag in compiler_flags\n if not (flag in to_remove or flag in remove_flags)\n ]"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\npatches up f90_compiler. gfortran. f90. gfortran. f90. f90.", "response": "def patch_f90_compiler(f90_compiler):\n \"\"\"Patch up ``f90_compiler``.\n\n For now, only updates the flags for ``gfortran``. In this case, it add\n any of ``GFORTRAN_SHARED_FLAGS`` that are missing. In debug mode, it also\n adds any flags in ``GFORTRAN_DEBUG_FLAGS`` and makes sure none of the flags\n in ``GFORTRAN_OPTIMIZE_FLAGS`` are present. In standard mode (\"OPTIMIZE\"),\n makes sure flags in ``GFORTRAN_OPTIMIZE_FLAGS`` are present and flags in\n ``GFORTRAN_DEBUG_FLAGS`` are not.\n\n Args:\n f90_compiler (numpy.distutils.fcompiler.FCompiler): A Fortran compiler\n instance.\n \"\"\"\n # NOTE: NumPy may not be installed, but we don't want **this** module to\n # cause an import failure in ``setup.py``.\n from numpy.distutils.fcompiler import gnu\n\n # Only ``gfortran``.\n if not isinstance(f90_compiler, gnu.Gnu95FCompiler):\n return False\n\n f90_compiler.compiler_f77[:] = _update_flags(\n f90_compiler.compiler_f77, remove_flags=(\"-Werror\",)\n )\n f90_compiler.compiler_f90[:] = _update_flags(f90_compiler.compiler_f90)"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nstarts the journaling of the current entry point.", "response": "def start_journaling(self):\n \"\"\"Capture calls to the system by compilers.\n\n See: https://github.com/numpy/numpy/blob/v1.15.2/\\\n numpy/distutils/ccompiler.py#L154\n\n Intercepts all calls to ``CCompiler.spawn`` and keeps the\n arguments around to be stored in the local ``commands``\n instance attribute.\n \"\"\"\n import numpy.distutils.ccompiler\n\n if self.journal_file is None:\n return\n\n def journaled_spawn(patched_self, cmd, display=None):\n self.commands.append(cmd)\n return numpy.distutils.ccompiler.CCompiler_spawn(\n patched_self, cmd, display=None\n )\n\n numpy.distutils.ccompiler.replace_method(\n distutils.ccompiler.CCompiler, \"spawn\", journaled_spawn\n )"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef save_journal(self):\n if self.journal_file is None:\n return\n\n try:\n as_text = self._commands_to_text()\n with open(self.journal_file, \"w\") as file_obj:\n file_obj.write(as_text)\n except Exception as exc:\n msg = BAD_JOURNAL.format(exc)\n print(msg, file=sys.stderr)", "response": "Save the commands to a file."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\ncreates a static library for the current locale.", "response": "def _default_static_lib(self, obj_files):\n \"\"\"Create a static library (i.e. a ``.a`` / ``.lib`` file).\n\n Args:\n obj_files (List[str]): List of paths of compiled object files.\n \"\"\"\n c_compiler = self.F90_COMPILER.c_compiler\n static_lib_dir = os.path.join(self.build_lib, \"bezier\", \"lib\")\n if not os.path.exists(static_lib_dir):\n os.makedirs(static_lib_dir)\n c_compiler.create_static_lib(\n obj_files, \"bezier\", output_dir=static_lib_dir\n )\n # NOTE: We must \"modify\" the paths for the ``extra_objects`` in\n # each extension since they were compiled with\n # ``output_dir=self.build_temp``.\n for extension in self.extensions:\n extension.extra_objects[:] = [\n os.path.join(self.build_temp, rel_path)\n for rel_path in extension.extra_objects\n ]"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _default_cleanup(self):\n if not self.inplace:\n return\n\n shutil.move(\n os.path.join(self.build_lib, \"bezier\", \"lib\"),\n os.path.join(\"src\", \"bezier\"),\n )", "response": "Default cleanup after run."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nverify that two consecutive edge endpoints share an endpoint.", "response": "def _verify_pair(prev, curr):\n \"\"\"Verify a pair of sides share an endpoint.\n\n .. note::\n\n This currently checks that edge endpoints match **exactly**\n but allowing some roundoff may be desired.\n\n Args:\n prev (.Curve): \"Previous\" curve at piecewise junction.\n curr (.Curve): \"Next\" curve at piecewise junction.\n\n Raises:\n ValueError: If the previous side is not in 2D.\n ValueError: If consecutive sides don't share an endpoint.\n \"\"\"\n if prev._dimension != 2:\n raise ValueError(\"Curve not in R^2\", prev)\n\n end = prev._nodes[:, -1]\n start = curr._nodes[:, 0]\n if not _helpers.vector_close(end, start):\n raise ValueError(\n \"Not sufficiently close\",\n \"Consecutive sides do not have common endpoint\",\n prev,\n curr,\n )"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nverifies that the edges define a curved polygon.", "response": "def _verify(self):\n \"\"\"Verify that the edges define a curved polygon.\n\n This may not be entirely comprehensive, e.g. won't check\n self-intersection of the defined polygon.\n\n .. note::\n\n This currently checks that edge endpoints match **exactly**\n but allowing some roundoff may be desired.\n\n Raises:\n ValueError: If there are fewer than two sides.\n ValueError: If one of the sides is not in 2D.\n ValueError: If consecutive sides don't share an endpoint.\n \"\"\"\n if self._num_sides < 2:\n raise ValueError(\"At least two sides required.\")\n\n for prev, curr in six.moves.zip(self._edges, self._edges[1:]):\n self._verify_pair(prev, curr)\n # Now we check that the final edge wraps around.\n prev = self._edges[-1]\n curr = self._edges[0]\n self._verify_pair(prev, curr)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef area(self):\n edges = tuple(edge._nodes for edge in self._edges)\n return _surface_helpers.compute_area(edges)", "response": "r A method to compute the area of the current curved polygon."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef plot(self, pts_per_edge, color=None, ax=None):\n if ax is None:\n ax = _plot_helpers.new_axis()\n _plot_helpers.add_patch(ax, color, pts_per_edge, *self._edges)\n return ax", "response": "Plot the current curved polygon."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef compute_implicit_line(nodes):\n delta = nodes[:, -1] - nodes[:, 0]\n length = np.linalg.norm(delta, ord=2)\n # Normalize and rotate 90 degrees to the \"left\".\n coeff_a = -delta[1] / length\n coeff_b = delta[0] / length\n # c = - ax - by = (delta[1] x - delta[0] y) / L\n # NOTE: We divide by ``length`` at the end to \"put off\" rounding.\n coeff_c = (delta[1] * nodes[0, 0] - delta[0] * nodes[1, 0]) / length\n return coeff_a, coeff_b, coeff_c", "response": "Compute the implicit form of the line connecting curve endpoints."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef compute_fat_line(nodes):\n coeff_a, coeff_b, coeff_c = compute_implicit_line(nodes)\n # NOTE: This assumes, but does not check, that there are two rows.\n _, num_nodes = nodes.shape\n d_min = 0.0\n d_max = 0.0\n for index in six.moves.xrange(1, num_nodes - 1): # Only interior nodes.\n curr_dist = (\n coeff_a * nodes[0, index] + coeff_b * nodes[1, index] + coeff_c\n )\n if curr_dist < d_min:\n d_min = curr_dist\n elif curr_dist > d_max:\n d_max = curr_dist\n return coeff_a, coeff_b, coeff_c, d_min, d_max", "response": "Compute the fat line around a B |eacute| zier curve."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nupdate the start and end of the clipped parameter range.", "response": "def _update_parameters(s_min, s_max, start0, end0, start1, end1):\n \"\"\"Update clipped parameter range.\n\n .. note::\n\n This is a helper for :func:`clip_range`.\n\n Does so by intersecting one of the two fat lines with an edge\n of the convex hull of the distance polynomial of the curve being\n clipped.\n\n If both of ``s_min`` and ``s_max`` are \"unset\", then any :math:`s`\n value that is valid for ``s_min`` would also be valid for ``s_max``.\n Rather than adding a special case to handle this scenario, **only**\n ``s_min`` will be updated.\n\n In cases where a given parameter :math:`s` would be a valid update\n for both ``s_min`` and ``s_max``\n This function **only** updates ``s_min``\n\n Args:\n s_min (float): Current start of clipped interval. If \"unset\", this\n value will be ``DEFAULT_S_MIN``.\n s_max (float): Current end of clipped interval. If \"unset\", this\n value will be ``DEFAULT_S_MAX``.\n start0 (numpy.ndarray): A 1D NumPy ``2``-array that is the start\n vector of one of the two fat lines.\n end0 (numpy.ndarray): A 1D NumPy ``2``-array that is the end\n vector of one of the two fat lines.\n start1 (numpy.ndarray): A 1D NumPy ``2``-array that is the start\n vector of an edge of the convex hull of the distance\n polynomial :math:`d(t)` as an explicit B |eacute| zier curve.\n end1 (numpy.ndarray): A 1D NumPy ``2``-array that is the end\n vector of an edge of the convex hull of the distance\n polynomial :math:`d(t)` as an explicit B |eacute| zier curve.\n\n Returns:\n Tuple[float, float]: The (possibly updated) start and end\n of the clipped parameter range.\n\n Raises:\n NotImplementedError: If the two line segments are parallel. (This\n case will be supported at some point, just not now.)\n \"\"\"\n s, t, success = _geometric_intersection.segment_intersection(\n start0, end0, start1, end1\n )\n if not success:\n raise NotImplementedError(NO_PARALLEL)\n\n if _helpers.in_interval(t, 0.0, 1.0):\n if _helpers.in_interval(s, 0.0, s_min):\n return s, s_max\n\n elif _helpers.in_interval(s, s_max, 1.0):\n return s_min, s\n\n return s_min, s_max"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef _check_parameter_range(s_min, s_max):\n if s_min == DEFAULT_S_MIN:\n # Based on the way ``_update_parameters`` works, we know\n # both parameters must be unset if ``s_min``.\n return 0.0, 1.0\n\n if s_max == DEFAULT_S_MAX:\n return s_min, s_min\n\n return s_min, s_max", "response": "r This function checks the range of the current time range."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef clip_range(nodes1, nodes2):\n # NOTE: There is no corresponding \"enable\", but the disable only applies\n # in this lexical scope.\n # pylint: disable=too-many-locals\n coeff_a, coeff_b, coeff_c, d_min, d_max = compute_fat_line(nodes1)\n # NOTE: This assumes, but does not check, that there are two rows.\n _, num_nodes2 = nodes2.shape\n polynomial = np.empty((2, num_nodes2), order=\"F\")\n denominator = float(num_nodes2 - 1)\n for index in six.moves.xrange(num_nodes2):\n polynomial[0, index] = index / denominator\n polynomial[1, index] = (\n coeff_a * nodes2[0, index] + coeff_b * nodes2[1, index] + coeff_c\n )\n # Define segments for the top and the bottom of the region\n # bounded by the fat line.\n start_bottom = np.asfortranarray([0.0, d_min])\n end_bottom = np.asfortranarray([1.0, d_min])\n start_top = np.asfortranarray([0.0, d_max])\n end_top = np.asfortranarray([1.0, d_max])\n s_min = DEFAULT_S_MIN\n s_max = DEFAULT_S_MAX\n # NOTE: We avoid computing the convex hull and just compute where\n # all segments connecting two control points intersect the\n # fat lines.\n for start_index in six.moves.xrange(num_nodes2 - 1):\n for end_index in six.moves.xrange(start_index + 1, num_nodes2):\n s_min, s_max = _update_parameters(\n s_min,\n s_max,\n start_bottom,\n end_bottom,\n polynomial[:, start_index],\n polynomial[:, end_index],\n )\n s_min, s_max = _update_parameters(\n s_min,\n s_max,\n start_top,\n end_top,\n polynomial[:, start_index],\n polynomial[:, end_index],\n )\n return _check_parameter_range(s_min, s_max)", "response": "r Reduce the parameter range where two curves can intersect."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nposting - process a generated journal file on Travis macOS.", "response": "def post_process_travis_macos(journal_filename):\n \"\"\"Post-process a generated journal file on Travis macOS.\n\n Args:\n journal_filename (str): The name of the journal file.\n \"\"\"\n travis_build_dir = os.environ.get(\"TRAVIS_BUILD_DIR\", \"\")\n with open(journal_filename, \"r\") as file_obj:\n content = file_obj.read()\n processed = content.replace(travis_build_dir, \"${TRAVIS_BUILD_DIR}\")\n with open(journal_filename, \"w\") as file_obj:\n file_obj.write(processed)"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef mod_replace(match, sphinx_modules):\n sphinx_modules.append(match.group(\"module\"))\n return \"`{}`_\".format(match.group(\"value\"))", "response": "Convert Sphinx mod name to plain reST link name."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nconverts Sphinx doc to plain reST link.", "response": "def doc_replace(match, sphinx_docs):\n \"\"\"Convert Sphinx ``:doc:`` to plain reST link.\n\n Args:\n match (_sre.SRE_Match): A match (from ``re``) to be used\n in substitution.\n sphinx_docs (list): List to be track the documents that have been\n encountered.\n\n Returns:\n str: The ``match`` converted to a link.\n \"\"\"\n sphinx_docs.append(match.group(\"path\"))\n return \"`{}`_\".format(match.group(\"value\"))"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef get_diff(value1, value2, name1, name2):\n lines1 = [line + \"\\n\" for line in value1.splitlines()]\n lines2 = [line + \"\\n\" for line in value2.splitlines()]\n diff_lines = difflib.context_diff(\n lines1, lines2, fromfile=name1, tofile=name2\n )\n return \"\".join(diff_lines)", "response": "Get a diff between two strings."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef populate_readme(revision, rtd_version, **extra_kwargs):\n with open(TEMPLATE_FILE, \"r\") as file_obj:\n template = file_obj.read()\n img_prefix = IMG_PREFIX.format(revision=revision)\n extra_links = EXTRA_LINKS.format(\n rtd_version=rtd_version, revision=revision\n )\n docs_img = DOCS_IMG.format(rtd_version=rtd_version)\n bernstein_basis = BERNSTEIN_BASIS_PLAIN.format(img_prefix=img_prefix)\n bezier_defn = BEZIER_DEFN_PLAIN.format(img_prefix=img_prefix)\n sum_to_unity = SUM_TO_UNITY_PLAIN.format(img_prefix=img_prefix)\n template_kwargs = {\n \"code_block1\": PLAIN_CODE_BLOCK,\n \"code_block2\": PLAIN_CODE_BLOCK,\n \"code_block3\": PLAIN_CODE_BLOCK,\n \"testcleanup\": \"\",\n \"toctree\": \"\",\n \"bernstein_basis\": bernstein_basis,\n \"bezier_defn\": bezier_defn,\n \"sum_to_unity\": sum_to_unity,\n \"img_prefix\": img_prefix,\n \"extra_links\": extra_links,\n \"docs\": \"|docs| \",\n \"docs_img\": docs_img,\n \"pypi\": \"\\n\\n|pypi| \",\n \"pypi_img\": PYPI_IMG,\n \"versions\": \"|versions|\\n\\n\",\n \"versions_img\": VERSIONS_IMG,\n \"rtd_version\": rtd_version,\n \"revision\": revision,\n \"circleci_badge\": CIRCLECI_BADGE,\n \"circleci_path\": \"\",\n \"travis_badge\": TRAVIS_BADGE,\n \"travis_path\": \"\",\n \"appveyor_badge\": APPVEYOR_BADGE,\n \"appveyor_path\": \"\",\n \"coveralls_badge\": COVERALLS_BADGE,\n \"coveralls_path\": COVERALLS_PATH,\n \"zenodo\": \"|zenodo|\",\n \"zenodo_img\": ZENODO_IMG,\n \"joss\": \" |JOSS|\",\n \"joss_img\": JOSS_IMG,\n }\n template_kwargs.update(**extra_kwargs)\n readme_contents = template.format(**template_kwargs)\n # Apply regular expressions to convert Sphinx \"roles\" to plain reST.\n readme_contents = INLINE_MATH_EXPR.sub(inline_math, readme_contents)\n sphinx_modules = []\n to_replace = functools.partial(mod_replace, sphinx_modules=sphinx_modules)\n readme_contents = MOD_EXPR.sub(to_replace, readme_contents)\n if sphinx_modules != [\"bezier.curve\", \"bezier.surface\"]:\n raise ValueError(\"Unexpected sphinx_modules\", sphinx_modules)\n\n sphinx_docs = []\n to_replace = functools.partial(doc_replace, sphinx_docs=sphinx_docs)\n readme_contents = DOC_EXPR.sub(to_replace, readme_contents)\n if sphinx_docs != [\"python/reference/bezier\", \"development\"]:\n raise ValueError(\"Unexpected sphinx_docs\", sphinx_docs)\n\n return readme_contents", "response": "Populate the README template with values."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\npopulate the template and compare to the expected README.", "response": "def readme_verify():\n \"\"\"Populate the template and compare to ``README``.\n\n Raises:\n ValueError: If the current README doesn't agree with the expected\n value computed from the template.\n \"\"\"\n expected = populate_readme(REVISION, RTD_VERSION)\n # Actually get the stored contents.\n with open(README_FILE, \"r\") as file_obj:\n contents = file_obj.read()\n if contents != expected:\n err_msg = \"\\n\" + get_diff(\n contents, expected, \"README.rst.actual\", \"README.rst.expected\"\n )\n raise ValueError(err_msg)\n\n else:\n print(\"README contents are as expected.\")"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef release_readme_verify():\n version = \"{version}\"\n expected = populate_readme(\n version,\n version,\n pypi=\"\",\n pypi_img=\"\",\n versions=\"\\n\\n\",\n versions_img=\"\",\n circleci_badge=CIRCLECI_BADGE_RELEASE,\n circleci_path=\"/{circleci_build}\",\n travis_badge=TRAVIS_BADGE_RELEASE,\n travis_path=\"/builds/{travis_build}\",\n appveyor_badge=APPVEYOR_BADGE_RELEASE,\n appveyor_path=\"/build/{appveyor_build}\",\n coveralls_badge=COVERALLS_BADGE_RELEASE,\n coveralls_path=\"builds/{coveralls_build}\",\n )\n with open(RELEASE_README_FILE, \"r\") as file_obj:\n contents = file_obj.read()\n if contents != expected:\n err_msg = \"\\n\" + get_diff(\n contents,\n expected,\n \"README.rst.release.actual\",\n \"README.rst.release.expected\",\n )\n raise ValueError(err_msg)\n\n else:\n print(\"README.rst.release.template contents are as expected.\")", "response": "Specialize the template to a PyPI release template."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _index_verify(index_file, **extra_kwargs):\n side_effect = extra_kwargs.pop(\"side_effect\", None)\n with open(TEMPLATE_FILE, \"r\") as file_obj:\n template = file_obj.read()\n template_kwargs = {\n \"code_block1\": SPHINX_CODE_BLOCK1,\n \"code_block2\": SPHINX_CODE_BLOCK2,\n \"code_block3\": SPHINX_CODE_BLOCK3,\n \"testcleanup\": TEST_CLEANUP,\n \"toctree\": TOCTREE,\n \"bernstein_basis\": BERNSTEIN_BASIS_SPHINX,\n \"bezier_defn\": BEZIER_DEFN_SPHINX,\n \"sum_to_unity\": SUM_TO_UNITY_SPHINX,\n \"img_prefix\": \"\",\n \"extra_links\": \"\",\n \"docs\": \"\",\n \"docs_img\": \"\",\n \"pypi\": \"\\n\\n|pypi| \",\n \"pypi_img\": PYPI_IMG,\n \"versions\": \"|versions|\\n\\n\",\n \"versions_img\": VERSIONS_IMG,\n \"rtd_version\": RTD_VERSION,\n \"revision\": REVISION,\n \"circleci_badge\": CIRCLECI_BADGE,\n \"circleci_path\": \"\",\n \"travis_badge\": TRAVIS_BADGE,\n \"travis_path\": \"\",\n \"appveyor_badge\": APPVEYOR_BADGE,\n \"appveyor_path\": \"\",\n \"coveralls_badge\": COVERALLS_BADGE,\n \"coveralls_path\": COVERALLS_PATH,\n \"zenodo\": \"|zenodo|\",\n \"zenodo_img\": ZENODO_IMG,\n \"joss\": \" |JOSS|\",\n \"joss_img\": JOSS_IMG,\n }\n template_kwargs.update(**extra_kwargs)\n expected = template.format(**template_kwargs)\n if side_effect is not None:\n expected = side_effect(expected)\n with open(index_file, \"r\") as file_obj:\n contents = file_obj.read()\n if contents != expected:\n err_msg = \"\\n\" + get_diff(\n contents,\n expected,\n index_file + \".actual\",\n index_file + \".expected\",\n )\n raise ValueError(err_msg)\n\n else:\n rel_name = os.path.relpath(index_file, _ROOT_DIR)\n msg = \"{} contents are as expected.\".format(rel_name)\n print(msg)", "response": "Populate the template and compare to the index file."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef release_docs_side_effect(content):\n # First replace **all** curly braces.\n result = content.replace(\"{\", \"{{\").replace(\"}\", \"}}\")\n # Then reset the actual template arguments.\n result = result.replace(\"{{version}}\", \"{version}\")\n result = result.replace(\"{{circleci_build}}\", \"{circleci_build}\")\n result = result.replace(\"{{travis_build}}\", \"{travis_build}\")\n result = result.replace(\"{{appveyor_build}}\", \"{appveyor_build}\")\n result = result.replace(\"{{coveralls_build}}\", \"{coveralls_build}\")\n return result", "response": "Updates the template so that curly braces are escaped correctly."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\npopulates template and compare to index. rst. release. template.", "response": "def release_docs_index_verify():\n \"\"\"Populate template and compare to ``docs/index.rst.release.template``.\n\n Raises:\n ValueError: If the current ``index.rst.release.template`` doesn't\n agree with the expected value computed from the template.\n \"\"\"\n version = \"{version}\"\n _index_verify(\n RELEASE_INDEX_FILE,\n side_effect=release_docs_side_effect,\n pypi=\"\",\n pypi_img=\"\",\n versions=\"\\n\\n\",\n versions_img=\"\",\n rtd_version=version,\n revision=version,\n circleci_badge=CIRCLECI_BADGE_RELEASE,\n circleci_path=\"/{circleci_build}\",\n travis_badge=TRAVIS_BADGE_RELEASE,\n travis_path=\"/builds/{travis_build}\",\n appveyor_badge=APPVEYOR_BADGE_RELEASE,\n appveyor_path=\"/build/{appveyor_build}\",\n coveralls_badge=COVERALLS_BADGE_RELEASE,\n coveralls_path=\"builds/{coveralls_build}\",\n )"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef development_verify():\n with open(DEVELOPMENT_TEMPLATE, \"r\") as file_obj:\n template = file_obj.read()\n expected = template.format(revision=REVISION, rtd_version=RTD_VERSION)\n with open(DEVELOPMENT_FILE, \"r\") as file_obj:\n contents = file_obj.read()\n if contents != expected:\n err_msg = \"\\n\" + get_diff(\n contents,\n expected,\n \"DEVELOPMENT.rst.actual\",\n \"DEVELOPMENT.rst.expected\",\n )\n raise ValueError(err_msg)\n\n else:\n print(\"DEVELOPMENT.rst contents are as expected.\")", "response": "Populate template and compare to the current development file."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef native_libraries_verify():\n with open(BINARY_EXT_TEMPLATE, \"r\") as file_obj:\n template = file_obj.read()\n expected = template.format(revision=REVISION)\n with open(BINARY_EXT_FILE, \"r\") as file_obj:\n contents = file_obj.read()\n if contents != expected:\n err_msg = \"\\n\" + get_diff(\n contents,\n expected,\n \"docs/python/binary-extension.rst.actual\",\n \"docs/python/binary-extension.rst.expected\",\n )\n raise ValueError(err_msg)\n\n else:\n print(\"docs/python/binary-extension.rst contents are as expected.\")", "response": "Populate the template and compare to the binary - extension. rst file."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _evaluate3(nodes, x_val, y_val):\n # NOTE: This may be (a) slower and (b) less precise than\n # hard-coding the determinant.\n sylvester_mat = np.zeros((6, 6), order=\"F\")\n delta = nodes - np.asfortranarray([[x_val], [y_val]])\n delta[:, 1:3] *= 3.0\n # Swap rows/columns so that x-y are right next to each other.\n # This will only change the determinant up to a sign.\n sylvester_mat[:2, :4] = delta\n sylvester_mat[2:4, 1:5] = delta\n sylvester_mat[4:, 2:] = delta\n return np.linalg.det(sylvester_mat)", "response": "Helper function for evaluating degree 3."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef evaluate(nodes, x_val, y_val):\n _, num_nodes = nodes.shape\n if num_nodes == 1:\n raise ValueError(\"A point cannot be implicitized\")\n\n elif num_nodes == 2:\n # x(s) - x = (x0 - x) (1 - s) + (x1 - x) s\n # y(s) - y = (y0 - y) (1 - s) + (y1 - y) s\n # Modified Sylvester: [x0 - x, x1 - x]\n # [y0 - y, y1 - y]\n return (nodes[0, 0] - x_val) * (nodes[1, 1] - y_val) - (\n nodes[0, 1] - x_val\n ) * (nodes[1, 0] - y_val)\n\n elif num_nodes == 3:\n # x(s) - x = (x0 - x) (1 - s)^2 + 2 (x1 - x) s(1 - s) + (x2 - x) s^2\n # y(s) - y = (y0 - y) (1 - s)^2 + 2 (y1 - y) s(1 - s) + (y2 - y) s^2\n # Modified Sylvester: [x0 - x, 2(x1 - x), x2 - x, 0] = A|B|C|0\n # [ 0, x0 - x, 2(x1 - x), x2 - x] 0|A|B|C\n # [y0 - y, 2(y1 - y), y2 - y, 0] D|E|F|0\n # [ 0, y0 - y, 2(y1 - y), y2 - y] 0|D|E|F\n val_a, val_b, val_c = nodes[0, :] - x_val\n val_b *= 2\n val_d, val_e, val_f = nodes[1, :] - y_val\n val_e *= 2\n # [A, B, C] [E, F, 0]\n # det [E, F, 0] = - det [A, B, C] = -E (BF - CE) + F(AF - CD)\n # [D, E, F] [D, E, F]\n sub1 = val_b * val_f - val_c * val_e\n sub2 = val_a * val_f - val_c * val_d\n sub_det_a = -val_e * sub1 + val_f * sub2\n # [B, C, 0]\n # det [A, B, C] = B (BF - CE) - C (AF - CD)\n # [D, E, F]\n sub_det_d = val_b * sub1 - val_c * sub2\n return val_a * sub_det_a + val_d * sub_det_d\n\n elif num_nodes == 4:\n return _evaluate3(nodes, x_val, y_val)\n\n else:\n raise _helpers.UnsupportedDegree(num_nodes - 1, supported=(1, 2, 3))", "response": "r Evaluates the implicitized bivariate polynomial containing the curve."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _to_power_basis11(nodes1, nodes2):\n # We manually invert the Vandermonde matrix:\n # [1 0][c0] = [n0]\n # [1 1][c1] [n1]\n val0 = eval_intersection_polynomial(nodes1, nodes2, 0.0)\n val1 = eval_intersection_polynomial(nodes1, nodes2, 1.0)\n # [c0] = [ 1 0][n0]\n # [c1] = [-1 1][n1]\n return np.asfortranarray([val0, -val0 + val1])", "response": "r Converts the nodes in nodes1 and nodes2 into a power basis matrix."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _to_power_basis13(nodes1, nodes2):\n # We manually invert the Vandermonde matrix:\n # Use exact f.p. numbers to avoid round-off wherever possible.\n # [1 0 0 0 ][c0] = [n0]\n # [1 1/4 1/16 1/64 ][c1] [n1]\n # [1 3/4 9/16 27/64][c2] [n2]\n # [1 1 1 1 ][c3] [n3]\n val0 = eval_intersection_polynomial(nodes1, nodes2, 0.0)\n val1 = eval_intersection_polynomial(nodes1, nodes2, 0.25)\n val2 = eval_intersection_polynomial(nodes1, nodes2, 0.75)\n val3 = eval_intersection_polynomial(nodes1, nodes2, 1.0)\n # [c0] = [ 3 0 0 0][n0]\n # [c1] = 1 / 3 [-19 24 -8 3][n1]\n # [c2] = [ 32 -56 40 -16][n2]\n # [c3] = [-16 32 -32 16][n3]\n # Since polynomial coefficients, we don't need to divide by 3\n # to get the same polynomial. Avoid the division to avoid round-off.\n return np.asfortranarray(\n [\n 3.0 * val0,\n -19.0 * val0 + 24.0 * val1 - 8.0 * val2 + 3.0 * val3,\n 32.0 * val0 - 56.0 * val1 + 40.0 * val2 - 16.0 * val3,\n -16.0 * val0 + 32.0 * val1 - 32.0 * val2 + 16.0 * val3,\n ]\n )", "response": "r Returns the coefficients of an intersection polynomial."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _to_power_basis_degree4(nodes1, nodes2):\n # We manually invert the Vandermonde matrix:\n # [1 0 0 0 0 ][c0] = [n0]\n # [1 1/4 1/16 1/64 1/256 ][c1] [n1]\n # [1 1/2 1/4 1/8 1/16 ][c2] [n2]\n # [1 3/4 9/16 27/64 81/256][c3] [n3]\n # [1 1 1 1 1 ][c4] [n4]\n val0 = eval_intersection_polynomial(nodes1, nodes2, 0.0)\n val1 = eval_intersection_polynomial(nodes1, nodes2, 0.25)\n val2 = eval_intersection_polynomial(nodes1, nodes2, 0.5)\n val3 = eval_intersection_polynomial(nodes1, nodes2, 0.75)\n val4 = eval_intersection_polynomial(nodes1, nodes2, 1.0)\n # [c0] = [ 3 0 0 0 0 ][n0]\n # [c1] = 1 / 3 [-25 48 -36 16 -3 ][n1]\n # [c2] = [ 70 -208 228 -112 22][n2]\n # [c3] = [-80 288 -384 224 -48][n3]\n # [c4] = [ 32 -128 192 -128 32][n4]\n # Since polynomial coefficients, we don't need to divide by 3\n # to get the same polynomial. Avoid the division to avoid round-off.\n return np.asfortranarray(\n [\n 3.0 * val0,\n -25.0 * val0\n + 48.0 * val1\n - 36.0 * val2\n + 16.0 * val3\n - 3.0 * val4,\n 70.0 * val0\n - 208.0 * val1\n + 228.0 * val2\n - 112.0 * val3\n + 22.0 * val4,\n (\n -80.0 * val0\n + 288.0 * val1\n - 384.0 * val2\n + 224.0 * val3\n - 48.0 * val4\n ),\n 32.0 * val0\n - 128.0 * val1\n + 192.0 * val2\n - 128.0 * val3\n + 32.0 * val4,\n ]\n )", "response": "r Returns the coefficients of an intersection polynomial in a power basis."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _to_power_basis_degree8(nodes1, nodes2):\n evaluated = [\n eval_intersection_polynomial(nodes1, nodes2, t_val) for t_val in _CHEB9\n ]\n return polynomial.polyfit(_CHEB9, evaluated, 8)", "response": "r Helper function for to_power_basis."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _to_power_basis33(nodes1, nodes2):\n evaluated = [\n eval_intersection_polynomial(nodes1, nodes2, t_val)\n for t_val in _CHEB10\n ]\n return polynomial.polyfit(_CHEB10, evaluated, 9)", "response": "r Helper function for to_power_basis."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef to_power_basis(nodes1, nodes2):\n # NOTE: There is no corresponding \"enable\", but the disable only applies\n # in this lexical scope.\n # pylint: disable=too-many-return-statements\n _, num_nodes1 = nodes1.shape\n _, num_nodes2 = nodes2.shape\n if num_nodes1 == 2:\n if num_nodes2 == 2:\n return _to_power_basis11(nodes1, nodes2)\n\n elif num_nodes2 == 3:\n return _to_power_basis12(nodes1, nodes2)\n\n elif num_nodes2 == 4:\n return _to_power_basis13(nodes1, nodes2)\n\n elif num_nodes2 == 5:\n return _to_power_basis_degree4(nodes1, nodes2)\n\n elif num_nodes1 == 3:\n if num_nodes2 == 3:\n return _to_power_basis_degree4(nodes1, nodes2)\n\n elif num_nodes2 == 4:\n return _to_power_basis23(nodes1, nodes2)\n\n elif num_nodes2 == 5:\n return _to_power_basis_degree8(nodes1, nodes2)\n\n elif num_nodes1 == 4:\n if num_nodes2 == 4:\n return _to_power_basis33(nodes1, nodes2)\n\n raise NotImplementedError(\n \"Degree 1\",\n num_nodes1 - 1,\n \"Degree 2\",\n num_nodes2 - 1,\n _POWER_BASIS_ERR,\n )", "response": "Compute the coefficients of an intersection polynomial."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef normalize_polynomial(coeffs, threshold=_L2_THRESHOLD):\n l2_norm = polynomial_norm(coeffs)\n if l2_norm < threshold:\n return np.zeros(coeffs.shape, order=\"F\")\n\n else:\n coeffs /= l2_norm\n return coeffs", "response": "r Normalizes a polynomial in the L_2 sense."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef _get_sigma_coeffs(coeffs):\n num_nodes, = coeffs.shape\n degree = num_nodes - 1\n effective_degree = None\n for index in six.moves.range(degree, -1, -1):\n if coeffs[index] != 0.0:\n effective_degree = index\n break\n\n if effective_degree is None:\n # NOTE: This means ``np.all(coeffs == 0.0)``.\n return None, 0, 0\n\n if effective_degree == 0:\n return None, degree, 0\n\n sigma_coeffs = coeffs[:effective_degree] / coeffs[effective_degree]\n # Now we need to add the binomial coefficients, but we avoid\n # computing actual binomial coefficients. Starting from the largest\n # exponent, the first ratio of binomial coefficients is\n # (d C (e - 1)) / (d C e)\n # = e / (d - e + 1)\n binom_numerator = effective_degree\n binom_denominator = degree - effective_degree + 1\n for exponent in six.moves.xrange(effective_degree - 1, -1, -1):\n sigma_coeffs[exponent] *= binom_numerator\n sigma_coeffs[exponent] /= binom_denominator\n # We swap (d C j) with (d C (j - 1)), so `p / q` becomes\n # (p / q) (d C (j - 1)) / (d C j)\n # = (p j) / (q (d - j + 1))\n binom_numerator *= exponent\n binom_denominator *= degree - exponent + 1\n return sigma_coeffs, degree, effective_degree", "response": "Compute the sigma coefficients of the non - lead terms in the Bernstein basis."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef bernstein_companion(coeffs):\n sigma_coeffs, degree, effective_degree = _get_sigma_coeffs(coeffs)\n if effective_degree == 0:\n return np.empty((0, 0), order=\"F\"), degree, 0\n\n companion = np.zeros((effective_degree, effective_degree), order=\"F\")\n companion.flat[\n effective_degree :: effective_degree + 1 # noqa: E203\n ] = 1.0\n companion[0, :] = -sigma_coeffs[::-1]\n return companion, degree, effective_degree", "response": "r Compute a companion matrix for a polynomial in Bernstein basis."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef bezier_roots(coeffs):\n companion, degree, effective_degree = bernstein_companion(coeffs)\n if effective_degree:\n sigma_roots = np.linalg.eigvals(companion)\n # Filter out `sigma = -1`, i.e. \"points at infinity\".\n # We want the error ||(sigma - (-1))|| ~= 2^{-52}\n to_keep = np.abs(sigma_roots + 1.0) > _SIGMA_THRESHOLD\n sigma_roots = sigma_roots[to_keep]\n s_vals = sigma_roots / (1.0 + sigma_roots)\n else:\n s_vals = np.empty((0,), order=\"F\")\n if effective_degree != degree:\n delta = degree - effective_degree\n s_vals = np.hstack([s_vals, [1] * delta])\n return s_vals", "response": "r Compute polynomial roots of a Bernstein basis."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef roots_in_unit_interval(coeffs):\n all_roots = polynomial.polyroots(coeffs)\n # Only keep roots inside or very near to the unit interval.\n all_roots = all_roots[\n (_UNIT_INTERVAL_WIGGLE_START < all_roots.real)\n & (all_roots.real < _UNIT_INTERVAL_WIGGLE_END)\n ]\n # Only keep roots with very small imaginary part. (Really only\n # keep the real parts.)\n real_inds = np.abs(all_roots.imag) < _IMAGINARY_WIGGLE\n return all_roots[real_inds].real", "response": "r Compute roots of a polynomial in the unit interval."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _strip_leading_zeros(coeffs, threshold=_COEFFICIENT_THRESHOLD):\n while np.abs(coeffs[-1]) < threshold:\n coeffs = coeffs[:-1]\n return coeffs", "response": "r Strip leading zeros from a polynomial."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nresolve a computed intersection and adds to lists.", "response": "def _resolve_and_add(nodes1, s_val, final_s, nodes2, t_val, final_t):\n \"\"\"Resolve a computed intersection and add to lists.\n\n We perform one Newton step to deal with any residual issues of\n high-degree polynomial solves (one of which depends on the already\n approximate ``x_val, y_val``).\n\n Args:\n nodes1 (numpy.ndarray): The nodes in the first curve.\n s_val (float): The approximate intersection parameter\n along ``nodes1``.\n final_s (List[float]): The list of accepted intersection\n parameters ``s``.\n nodes2 (numpy.ndarray): The nodes in the second curve.\n t_val (float): The approximate intersection parameter\n along ``nodes2``.\n final_t (List[float]): The list of accepted intersection\n parameters ``t``.\n \"\"\"\n s_val, t_val = _intersection_helpers.newton_refine(\n s_val, nodes1, t_val, nodes2\n )\n s_val, success_s = _helpers.wiggle_interval(s_val)\n t_val, success_t = _helpers.wiggle_interval(t_val)\n if not (success_s and success_t):\n return\n\n final_s.append(s_val)\n final_t.append(t_val)"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef intersect_curves(nodes1, nodes2):\n nodes1 = _curve_helpers.full_reduce(nodes1)\n nodes2 = _curve_helpers.full_reduce(nodes2)\n _, num_nodes1 = nodes1.shape\n _, num_nodes2 = nodes2.shape\n swapped = False\n if num_nodes1 > num_nodes2:\n nodes1, nodes2 = nodes2, nodes1\n swapped = True\n coeffs = normalize_polynomial(to_power_basis(nodes1, nodes2))\n if np.all(coeffs == 0.0):\n raise NotImplementedError(_COINCIDENT_ERR)\n\n _check_non_simple(coeffs)\n t_vals = roots_in_unit_interval(coeffs)\n final_s = []\n final_t = []\n for t_val in t_vals:\n (x_val,), (y_val,) = _curve_helpers.evaluate_multi(\n nodes2, np.asfortranarray([t_val])\n )\n s_val = locate_point(nodes1, x_val, y_val)\n if s_val is not None:\n _resolve_and_add(nodes1, s_val, final_s, nodes2, t_val, final_t)\n result = np.zeros((2, len(final_s)), order=\"F\")\n if swapped:\n final_s, final_t = final_t, final_s\n result[0, :] = final_s\n result[1, :] = final_t\n return result", "response": "r Intersects two parametric B |eacute| zier curves."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nconverts a B |eacute| zier curve to a polynomial in power basis.", "response": "def poly_to_power_basis(bezier_coeffs):\n \"\"\"Convert a B |eacute| zier curve to polynomial in power basis.\n\n .. note::\n\n This assumes, but does not verify, that the \"B |eacute| zier\n degree\" matches the true degree of the curve. Callers can\n guarantee this by calling :func:`.full_reduce`.\n\n Args:\n bezier_coeffs (numpy.ndarray): A 1D array of coefficients in\n the Bernstein basis.\n\n Returns:\n numpy.ndarray: 1D array of coefficients in monomial basis.\n\n Raises:\n .UnsupportedDegree: If the degree of the curve is not among\n 0, 1, 2 or 3.\n \"\"\"\n num_coeffs, = bezier_coeffs.shape\n if num_coeffs == 1:\n return bezier_coeffs\n\n elif num_coeffs == 2:\n # C0 (1 - s) + C1 s = C0 + (C1 - C0) s\n coeff0, coeff1 = bezier_coeffs\n return np.asfortranarray([coeff0, coeff1 - coeff0])\n\n elif num_coeffs == 3:\n # C0 (1 - s)^2 + C1 2 (1 - s) s + C2 s^2\n # = C0 + 2(C1 - C0) s + (C2 - 2 C1 + C0) s^2\n coeff0, coeff1, coeff2 = bezier_coeffs\n return np.asfortranarray(\n [coeff0, 2.0 * (coeff1 - coeff0), coeff2 - 2.0 * coeff1 + coeff0]\n )\n\n elif num_coeffs == 4:\n # C0 (1 - s)^3 + C1 3 (1 - s)^2 + C2 3 (1 - s) s^2 + C3 s^3\n # = C0 + 3(C1 - C0) s + 3(C2 - 2 C1 + C0) s^2 +\n # (C3 - 3 C2 + 3 C1 - C0) s^3\n coeff0, coeff1, coeff2, coeff3 = bezier_coeffs\n return np.asfortranarray(\n [\n coeff0,\n 3.0 * (coeff1 - coeff0),\n 3.0 * (coeff2 - 2.0 * coeff1 + coeff0),\n coeff3 - 3.0 * coeff2 + 3.0 * coeff1 - coeff0,\n ]\n )\n\n else:\n raise _helpers.UnsupportedDegree(\n num_coeffs - 1, supported=(0, 1, 2, 3)\n )"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef all_intersections(nodes_first, nodes_second):\n # Only attempt this if the bounding boxes intersect.\n bbox_int = _geometric_intersection.bbox_intersect(\n nodes_first, nodes_second\n )\n if bbox_int == _DISJOINT:\n return np.empty((2, 0), order=\"F\"), False\n\n return intersect_curves(nodes_first, nodes_second), False", "response": "r Returns a 2x2 array of all intersections between two sets of nodes."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef _vector_close(vec1, vec2, eps=_EPS):\n # NOTE: This relies on ``vec1`` and ``vec2`` being one-dimensional\n # vectors so NumPy doesn't try to use a matrix norm.\n size1 = np.linalg.norm(vec1, ord=2)\n size2 = np.linalg.norm(vec2, ord=2)\n if size1 == 0:\n return size2 <= eps\n\n elif size2 == 0:\n return size1 <= eps\n\n else:\n upper_bound = eps * min(size1, size2)\n return np.linalg.norm(vec1 - vec2, ord=2) <= upper_bound", "response": "r Check that two vectors are close to some threshold."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef _bbox(nodes):\n left, bottom = np.min(nodes, axis=1)\n right, top = np.max(nodes, axis=1)\n return left, right, bottom, top", "response": "Returns the bounding box for a set of points."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\ncomputing the product of two Fortran contiguous matrices.", "response": "def matrix_product(mat1, mat2):\n \"\"\"Compute the product of two Fortran contiguous matrices.\n\n This is to avoid the overhead of NumPy converting to C-contiguous\n before computing a matrix product.\n\n Does so via ``A B = (B^T A^T)^T`` since ``B^T`` and ``A^T`` will be\n C-contiguous without a copy, then the product ``P = B^T A^T`` will\n be C-contiguous and we can return the view ``P^T`` without a copy.\n\n Args:\n mat1 (numpy.ndarray): The left-hand side matrix.\n mat2 (numpy.ndarray): The right-hand side matrix.\n\n Returns:\n numpy.ndarray: The product of the two matrices.\n \"\"\"\n return np.dot(mat2.T, mat1.T).T"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef cross_product_compare(start, candidate1, candidate2):\n delta1 = candidate1 - start\n delta2 = candidate2 - start\n return cross_product(delta1, delta2)", "response": "Compare two relative changes by their cross - product."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\ncheck if a value is contained in a sorted list.", "response": "def in_sorted(values, value):\n \"\"\"Checks if a value is in a sorted list.\n\n Uses the :mod:`bisect` builtin to find the insertion point for\n ``value``.\n\n Args:\n values (List[int]): Integers sorted in ascending order.\n value (int): Value to check if contained in ``values``.\n\n Returns:\n bool: Indicating if the value is contained.\n \"\"\"\n index = bisect.bisect_left(values, value)\n if index >= len(values):\n return False\n\n return values[index] == value"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef _simple_convex_hull(points):\n # NOTE: There is no corresponding \"enable\", but the disable only applies\n # in this lexical scope.\n # pylint: disable=too-many-branches\n if points.size == 0:\n return points\n\n # First, drop duplicates.\n unique_points = np.unique(points, axis=1)\n _, num_points = unique_points.shape\n if num_points < 2:\n return unique_points\n\n # Then sort the data in left-to-right order (and break ties by y-value).\n points = np.empty((2, num_points), order=\"F\")\n for index, xy_val in enumerate(\n sorted(tuple(column) for column in unique_points.T)\n ):\n points[:, index] = xy_val\n # After sorting, if there are only 2 points, return.\n if num_points < 3:\n return points\n\n # Build lower hull\n lower = [0, 1]\n for index in six.moves.xrange(2, num_points):\n point2 = points[:, index]\n while len(lower) >= 2:\n point0 = points[:, lower[-2]]\n point1 = points[:, lower[-1]]\n if cross_product_compare(point0, point1, point2) > 0:\n break\n\n else:\n lower.pop()\n lower.append(index)\n # Build upper hull\n upper = [num_points - 1]\n for index in six.moves.xrange(num_points - 2, -1, -1):\n # Don't consider indices from the lower hull (other than the ends).\n if index > 0 and in_sorted(lower, index):\n continue\n\n point2 = points[:, index]\n while len(upper) >= 2:\n point0 = points[:, upper[-2]]\n point1 = points[:, upper[-1]]\n if cross_product_compare(point0, point1, point2) > 0:\n break\n\n else:\n upper.pop()\n upper.append(index)\n # **Both** corners are double counted.\n size_polygon = len(lower) + len(upper) - 2\n polygon = np.empty((2, size_polygon), order=\"F\")\n for index, column in enumerate(lower[:-1]):\n polygon[:, index] = points[:, column]\n index_start = len(lower) - 1\n for index, column in enumerate(upper[:-1]):\n polygon[:, index + index_start] = points[:, column]\n return polygon", "response": "r Compute the convex hull for a set of points."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef is_separating(direction, polygon1, polygon2):\n # NOTE: We assume throughout that ``norm_squared != 0``. If it **were**\n # zero that would mean the ``direction`` corresponds to an\n # invalid edge.\n norm_squared = direction[0] * direction[0] + direction[1] * direction[1]\n params = []\n vertex = np.empty((2,), order=\"F\")\n for polygon in (polygon1, polygon2):\n _, polygon_size = polygon.shape\n min_param = np.inf\n max_param = -np.inf\n for index in six.moves.xrange(polygon_size):\n vertex[:] = polygon[:, index]\n param = cross_product(direction, vertex) / norm_squared\n min_param = min(min_param, param)\n max_param = max(max_param, param)\n params.append((min_param, max_param))\n # NOTE: The indexing is based on:\n # params[0] = (min_param1, max_param1)\n # params[1] = (min_param2, max_param2)\n return params[0][0] > params[1][1] or params[0][1] < params[1][0]", "response": "Checks if a given direction is a separating line for two polygons."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\ndetermines if two convex polygons collide.", "response": "def _polygon_collide(polygon1, polygon2):\n \"\"\"Determines if two **convex** polygons collide.\n\n .. _SAT: https://en.wikipedia.org/wiki/Hyperplane_separation_theorem\n .. _see also: https://hackmd.io/s/ryFmIZrsl\n\n This code uses the Separating axis theorem (`SAT`_) to quickly\n determine if the polygons intersect. `See also`_.\n\n .. note::\n\n There is also a Fortran implementation of this function, which\n will be used if it can be built.\n\n Args:\n polygon1 (numpy.ndarray): A ``2 x N`` array (``float64``) of ordered\n points in a polygon.\n polygon2 (numpy.ndarray): A ``2 x N`` array (``float64``) of ordered\n points in a polygon.\n\n Returns:\n bool: Flag indicating if the two polygons collide.\n \"\"\"\n direction = np.empty((2,), order=\"F\")\n for polygon in (polygon1, polygon2):\n _, polygon_size = polygon.shape\n for index in six.moves.xrange(polygon_size):\n # NOTE: When ``index == 0`` this will \"wrap around\" and refer\n # to index ``-1``.\n direction[:] = polygon[:, index] - polygon[:, index - 1]\n if is_separating(direction, polygon1, polygon2):\n return False\n\n return True"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef solve2x2(lhs, rhs):\n # A <--> lhs[0, 0]\n # B <--> lhs[0, 1]\n # C <--> lhs[1, 0]\n # D <--> lhs[1, 1]\n # E <--> rhs[0]\n # F <--> rhs[1]\n if np.abs(lhs[1, 0]) > np.abs(lhs[0, 0]):\n # NOTE: We know there is no division by zero here since ``C``\n # is **strictly** bigger than **some** value (in magnitude).\n # [A | B][x] = [E]\n # [C | D][y] [F]\n ratio = lhs[0, 0] / lhs[1, 0]\n # r = A / C\n # [A - rC | B - rD][x] [E - rF]\n # [C | D ][y] = [F ]\n # ==> 0x + (B - rD) y = E - rF\n denominator = lhs[0, 1] - ratio * lhs[1, 1]\n if denominator == 0.0:\n return True, None, None\n\n y_val = (rhs[0] - ratio * rhs[1]) / denominator\n # Cx + Dy = F ==> x = (F - Dy) / C\n x_val = (rhs[1] - lhs[1, 1] * y_val) / lhs[1, 0]\n return False, x_val, y_val\n\n else:\n if lhs[0, 0] == 0.0:\n return True, None, None\n\n # [A | B][x] = [E]\n # [C | D][y] [F]\n ratio = lhs[1, 0] / lhs[0, 0]\n # r = C / A\n # [A | B ][x] = [E ]\n # [C - rA | D - rB][y] [F - rE]\n # ==> 0x + (D - rB) y = F - rE\n denominator = lhs[1, 1] - ratio * lhs[0, 1]\n if denominator == 0.0:\n return True, None, None\n\n y_val = (rhs[1] - ratio * rhs[0]) / denominator\n # Ax + By = E ==> x = (E - B y) / A\n x_val = (rhs[0] - lhs[0, 1] * y_val) / lhs[0, 0]\n return False, x_val, y_val", "response": "Solve a square 2 x 2 system via LU factorization."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef add_plot_boundary(ax, padding=0.125):\n nodes = np.asfortranarray(\n np.vstack([line.get_xydata() for line in ax.lines]).T\n )\n left, right, bottom, top = _helpers.bbox(nodes)\n center_x = 0.5 * (right + left)\n delta_x = right - left\n center_y = 0.5 * (top + bottom)\n delta_y = top - bottom\n multiplier = (1.0 + padding) * 0.5\n ax.set_xlim(\n center_x - multiplier * delta_x, center_x + multiplier * delta_x\n )\n ax.set_ylim(\n center_y - multiplier * delta_y, center_y + multiplier * delta_y\n )", "response": "Add a buffer of empty space around a plot boundary."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef add_patch(ax, color, pts_per_edge, *edges):\n from matplotlib import patches\n from matplotlib import path as _path_mod\n\n s_vals = np.linspace(0.0, 1.0, pts_per_edge)\n # Evaluate points on each edge.\n all_points = []\n for edge in edges:\n points = edge.evaluate_multi(s_vals)\n # We assume the edges overlap and leave out the first point\n # in each.\n all_points.append(points[:, 1:])\n # Add first point as last point (polygon is closed).\n first_edge = all_points[0]\n all_points.append(first_edge[:, [0]])\n # Add boundary first.\n polygon = np.asfortranarray(np.hstack(all_points))\n line, = ax.plot(polygon[0, :], polygon[1, :], color=color)\n # Reset ``color`` in case it was ``None`` and set from color wheel.\n color = line.get_color()\n # ``polygon`` is stored Fortran-contiguous with ``x-y`` points in each\n # column but ``Path()`` wants ``x-y`` points in each row.\n path = _path_mod.Path(polygon.T)\n patch = patches.PathPatch(path, facecolor=color, alpha=0.625)\n ax.add_patch(patch)", "response": "Adds a polygonal surface patch to a matplotlib axis."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nvisit a literal_block node. This verifies the state of each literal block.", "response": "def visit_literal_block(self, node):\n \"\"\"Visit a ``literal_block`` node.\n\n This verifies the state of each literal / code block.\n \"\"\"\n language = node.attributes.get(\"language\", \"\")\n test_type = node.attributes.get(\"testnodetype\", \"\")\n if test_type != \"doctest\":\n if language.lower() in (\"\", \"python\"):\n msg = _LITERAL_ERR_TEMPLATE.format(\n node.rawsource, language, test_type\n )\n raise errors.ExtensionError(msg)\n\n # The base classes are not new-style, so we can't use super().\n return html.HTMLTranslator.visit_literal_block(self, node)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef linearization_error(nodes):\n _, num_nodes = nodes.shape\n degree = num_nodes - 1\n if degree == 1:\n return 0.0\n\n second_deriv = nodes[:, :-2] - 2.0 * nodes[:, 1:-1] + nodes[:, 2:]\n worst_case = np.max(np.abs(second_deriv), axis=1)\n # max_{0 <= s <= 1} s(1 - s)/2 = 1/8 = 0.125\n multiplier = 0.125 * degree * (degree - 1)\n # NOTE: worst_case is 1D due to np.max(), so this is the vector norm.\n return multiplier * np.linalg.norm(worst_case, ord=2)", "response": "r Compute the maximum error of a linear approximation."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef line_line_collide(line1, line2):\n s, t, success = segment_intersection(\n line1[:, 0], line1[:, 1], line2[:, 0], line2[:, 1]\n )\n if success:\n return _helpers.in_interval(s, 0.0, 1.0) and _helpers.in_interval(\n t, 0.0, 1.0\n )\n\n else:\n disjoint, _ = parallel_lines_parameters(\n line1[:, 0], line1[:, 1], line2[:, 0], line2[:, 1]\n )\n return not disjoint", "response": "Determines if two line segments meet."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\ndetermine if the convex hulls of two curves collide.", "response": "def convex_hull_collide(nodes1, nodes2):\n \"\"\"Determine if the convex hulls of two curves collide.\n\n .. note::\n\n This is a helper for :func:`from_linearized`.\n\n Args:\n nodes1 (numpy.ndarray): Control points of a first curve.\n nodes2 (numpy.ndarray): Control points of a second curve.\n\n Returns:\n bool: Indicating if the convex hulls collide.\n \"\"\"\n polygon1 = _helpers.simple_convex_hull(nodes1)\n _, polygon_size1 = polygon1.shape\n polygon2 = _helpers.simple_convex_hull(nodes2)\n _, polygon_size2 = polygon2.shape\n if polygon_size1 == 2 and polygon_size2 == 2:\n return line_line_collide(polygon1, polygon2)\n\n else:\n return _helpers.polygon_collide(polygon1, polygon2)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\ndetermine curve - curve intersection from two linearizations.", "response": "def from_linearized(first, second, intersections):\n \"\"\"Determine curve-curve intersection from pair of linearizations.\n\n .. note::\n\n This assumes that at least one of ``first`` and ``second`` is\n not a line. The line-line case should be handled \"early\"\n by :func:`check_lines`.\n\n .. note::\n\n This assumes the caller has verified that the bounding boxes\n for ``first`` and ``second`` actually intersect.\n\n If there is an intersection along the segments, adds that intersection\n to ``intersections``. Otherwise, returns without doing anything.\n\n Args:\n first (Linearization): First curve being intersected.\n second (Linearization): Second curve being intersected.\n intersections (list): A list of existing intersections.\n\n Raises:\n ValueError: If ``first`` and ``second`` both have linearization error\n of ``0.0`` (i.e. they are both lines). This is because this\n function expects the caller to have used :func:`check_lines`\n already.\n \"\"\"\n # NOTE: There is no corresponding \"enable\", but the disable only applies\n # in this lexical scope.\n # pylint: disable=too-many-return-statements\n s, t, success = segment_intersection(\n first.start_node, first.end_node, second.start_node, second.end_node\n )\n bad_parameters = False\n if success:\n if not (\n _helpers.in_interval(s, 0.0, 1.0)\n and _helpers.in_interval(t, 0.0, 1.0)\n ):\n bad_parameters = True\n else:\n if first.error == 0.0 and second.error == 0.0:\n raise ValueError(_UNHANDLED_LINES)\n\n # Just fall back to a Newton iteration starting in the middle of\n # the given intervals.\n bad_parameters = True\n s = 0.5\n t = 0.5\n if bad_parameters:\n # In the unlikely case that we have parallel segments or segments\n # that intersect outside of [0, 1] x [0, 1], we can still exit\n # if the convex hulls don't intersect.\n if not convex_hull_collide(first.curve.nodes, second.curve.nodes):\n return\n\n # Now, promote ``s`` and ``t`` onto the original curves.\n orig_s = (1 - s) * first.curve.start + s * first.curve.end\n orig_t = (1 - t) * second.curve.start + t * second.curve.end\n refined_s, refined_t = _intersection_helpers.full_newton(\n orig_s, first.curve.original_nodes, orig_t, second.curve.original_nodes\n )\n refined_s, success = _helpers.wiggle_interval(refined_s)\n if not success:\n return\n\n refined_t, success = _helpers.wiggle_interval(refined_t)\n if not success:\n return\n\n add_intersection(refined_s, refined_t, intersections)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef tangent_bbox_intersection(first, second, intersections):\n node_first1 = first.nodes[:, 0]\n node_first2 = first.nodes[:, -1]\n node_second1 = second.nodes[:, 0]\n node_second2 = second.nodes[:, -1]\n endpoint_check(\n first, node_first1, 0.0, second, node_second1, 0.0, intersections\n )\n endpoint_check(\n first, node_first1, 0.0, second, node_second2, 1.0, intersections\n )\n endpoint_check(\n first, node_first2, 1.0, second, node_second1, 0.0, intersections\n )\n endpoint_check(\n first, node_first2, 1.0, second, node_second2, 1.0, intersections\n )", "response": "r Check if two curves with tangent bounding boxes intersect."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef bbox_line_intersect(nodes, line_start, line_end):\n left, right, bottom, top = _helpers.bbox(nodes)\n if _helpers.in_interval(\n line_start[0], left, right\n ) and _helpers.in_interval(line_start[1], bottom, top):\n return BoxIntersectionType.INTERSECTION\n\n if _helpers.in_interval(line_end[0], left, right) and _helpers.in_interval(\n line_end[1], bottom, top\n ):\n return BoxIntersectionType.INTERSECTION\n\n # NOTE: We allow ``segment_intersection`` to fail below (i.e.\n # ``success=False``). At first, this may appear to \"ignore\"\n # some potential intersections of parallel lines. However,\n # no intersections will be missed. If parallel lines don't\n # overlap, then there is nothing to miss. If they do overlap,\n # then either the segment will have endpoints on the box (already\n # covered by the checks above) or the segment will contain an\n # entire side of the box, which will force it to intersect the 3\n # edges that meet at the two ends of those sides. The parallel\n # edge will be skipped, but the other two will be covered.\n # Bottom Edge\n s_bottom, t_bottom, success = segment_intersection(\n np.asfortranarray([left, bottom]),\n np.asfortranarray([right, bottom]),\n line_start,\n line_end,\n )\n if (\n success\n and _helpers.in_interval(s_bottom, 0.0, 1.0)\n and _helpers.in_interval(t_bottom, 0.0, 1.0)\n ):\n return BoxIntersectionType.INTERSECTION\n\n # Right Edge\n s_right, t_right, success = segment_intersection(\n np.asfortranarray([right, bottom]),\n np.asfortranarray([right, top]),\n line_start,\n line_end,\n )\n if (\n success\n and _helpers.in_interval(s_right, 0.0, 1.0)\n and _helpers.in_interval(t_right, 0.0, 1.0)\n ):\n return BoxIntersectionType.INTERSECTION\n\n # Top Edge\n s_top, t_top, success = segment_intersection(\n np.asfortranarray([right, top]),\n np.asfortranarray([left, top]),\n line_start,\n line_end,\n )\n if (\n success\n and _helpers.in_interval(s_top, 0.0, 1.0)\n and _helpers.in_interval(t_top, 0.0, 1.0)\n ):\n return BoxIntersectionType.INTERSECTION\n\n # NOTE: We skip the \"last\" edge. This is because any curve\n # that doesn't have an endpoint on a curve must cross\n # at least two, so we will have already covered such curves\n # in one of the branches above.\n return BoxIntersectionType.DISJOINT", "response": "r Determine intersection of a bounding box and a line segment."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nperform one step of the intersection process.", "response": "def intersect_one_round(candidates, intersections):\n \"\"\"Perform one step of the intersection process.\n\n .. note::\n\n This is a helper for :func:`_all_intersections` and that function\n has a Fortran equivalent.\n\n Checks if the bounding boxes of each pair in ``candidates``\n intersect. If the bounding boxes do not intersect, the pair\n is discarded. Otherwise, the pair is \"accepted\". Then we\n attempt to linearize each curve in an \"accepted\" pair and\n track the overall linearization error for every curve\n encountered.\n\n Args:\n candidates (Union[list, itertools.chain]): An iterable of\n pairs of curves (or linearized curves).\n intersections (list): A list of already encountered\n intersections. If any intersections can be readily determined\n during this round of subdivision, then they will be added\n to this list.\n\n Returns:\n list: Returns a list of the next round of ``candidates``.\n \"\"\"\n next_candidates = []\n # NOTE: In the below we replace ``isinstance(a, B)`` with\n # ``a.__class__ is B``, which is a 3-3.5x speedup.\n for first, second in candidates:\n both_linearized = False\n if first.__class__ is Linearization:\n if second.__class__ is Linearization:\n both_linearized = True\n bbox_int = bbox_intersect(\n first.curve.nodes, second.curve.nodes\n )\n else:\n bbox_int = bbox_line_intersect(\n second.nodes, first.start_node, first.end_node\n )\n else:\n if second.__class__ is Linearization:\n bbox_int = bbox_line_intersect(\n first.nodes, second.start_node, second.end_node\n )\n else:\n bbox_int = bbox_intersect(first.nodes, second.nodes)\n if bbox_int == BoxIntersectionType.DISJOINT:\n continue\n\n elif bbox_int == BoxIntersectionType.TANGENT and not both_linearized:\n # NOTE: Ignore tangent bounding boxes in the linearized case\n # because ``tangent_bbox_intersection()`` assumes that both\n # curves are not linear.\n tangent_bbox_intersection(first, second, intersections)\n continue\n\n if both_linearized:\n # If both ``first`` and ``second`` are linearizations, then\n # we can intersect them immediately.\n from_linearized(first, second, intersections)\n continue\n\n # If we haven't ``continue``-d, add the accepted pair.\n # NOTE: This may be a wasted computation, e.g. if ``first``\n # or ``second`` occur in multiple accepted pairs (the caller\n # only passes one pair at a time). However, in practice\n # the number of such pairs will be small so this cost\n # will be low.\n lin1 = six.moves.map(Linearization.from_shape, first.subdivide())\n lin2 = six.moves.map(Linearization.from_shape, second.subdivide())\n next_candidates.extend(itertools.product(lin1, lin2))\n return next_candidates"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef prune_candidates(candidates):\n pruned = []\n # NOTE: In the below we replace ``isinstance(a, B)`` with\n # ``a.__class__ is B``, which is a 3-3.5x speedup.\n for first, second in candidates:\n if first.__class__ is Linearization:\n nodes1 = first.curve.nodes\n else:\n nodes1 = first.nodes\n if second.__class__ is Linearization:\n nodes2 = second.curve.nodes\n else:\n nodes2 = second.nodes\n if convex_hull_collide(nodes1, nodes2):\n pruned.append((first, second))\n return pruned", "response": "Reduce number of candidate intersection pairs."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nmake a new set of control points that are in the same degree.", "response": "def make_same_degree(nodes1, nodes2):\n \"\"\"Degree-elevate a curve so two curves have matching degree.\n\n Args:\n nodes1 (numpy.ndarray): Set of control points for a\n B |eacute| zier curve.\n nodes2 (numpy.ndarray): Set of control points for a\n B |eacute| zier curve.\n\n Returns:\n Tuple[numpy.ndarray, numpy.ndarray]: The potentially degree-elevated\n nodes passed in.\n \"\"\"\n _, num_nodes1 = nodes1.shape\n _, num_nodes2 = nodes2.shape\n for _ in six.moves.xrange(num_nodes2 - num_nodes1):\n nodes1 = _curve_helpers.elevate_nodes(nodes1)\n for _ in six.moves.xrange(num_nodes1 - num_nodes2):\n nodes2 = _curve_helpers.elevate_nodes(nodes2)\n return nodes1, nodes2"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef check_lines(first, second):\n # NOTE: In the below we replace ``isinstance(a, B)`` with\n # ``a.__class__ is B``, which is a 3-3.5x speedup.\n if not (\n first.__class__ is Linearization\n and second.__class__ is Linearization\n and first.error == 0.0\n and second.error == 0.0\n ):\n return False, None\n\n s, t, success = segment_intersection(\n first.start_node, first.end_node, second.start_node, second.end_node\n )\n if success:\n if _helpers.in_interval(s, 0.0, 1.0) and _helpers.in_interval(\n t, 0.0, 1.0\n ):\n intersections = np.asfortranarray([[s], [t]])\n result = intersections, False\n else:\n result = np.empty((2, 0), order=\"F\"), False\n else:\n disjoint, params = parallel_lines_parameters(\n first.start_node,\n first.end_node,\n second.start_node,\n second.end_node,\n )\n if disjoint:\n result = np.empty((2, 0), order=\"F\"), False\n else:\n result = params, True\n return True, result", "response": "Checks if two curves are lines and returns a pair of two sets of intersection boolean arrays."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nsplit the curve into a left and right half.", "response": "def subdivide(self):\n \"\"\"Split the curve into a left and right half.\n\n See :meth:`.Curve.subdivide` for more information.\n\n Returns:\n Tuple[SubdividedCurve, SubdividedCurve]: The left and right\n sub-curves.\n \"\"\"\n left_nodes, right_nodes = _curve_helpers.subdivide_nodes(self.nodes)\n midpoint = 0.5 * (self.start + self.end)\n left = SubdividedCurve(\n left_nodes, self.original_nodes, start=self.start, end=midpoint\n )\n right = SubdividedCurve(\n right_nodes, self.original_nodes, start=midpoint, end=self.end\n )\n return left, right"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef from_shape(cls, shape):\n # NOTE: In the below we replace ``isinstance(a, B)`` with\n # ``a.__class__ is B``, which is a 3-3.5x speedup.\n if shape.__class__ is cls:\n return shape\n\n else:\n error = linearization_error(shape.nodes)\n if error < _ERROR_VAL:\n linearized = cls(shape, error)\n return linearized\n\n else:\n return shape", "response": "Try to linearize a curve or an already linearized curve."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef evaluate(self, s):\n return _curve_helpers.evaluate_multi(\n self._nodes, np.asfortranarray([s])\n )", "response": "Evaluate the curve along the curve."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nplots the current curve.", "response": "def plot(self, num_pts, color=None, alpha=None, ax=None):\n \"\"\"Plot the current curve.\n\n Args:\n num_pts (int): Number of points to plot.\n color (Optional[Tuple[float, float, float]]): Color as RGB profile.\n alpha (Optional[float]): The alpha channel for the color.\n ax (Optional[matplotlib.artist.Artist]): matplotlib axis object\n to add plot to.\n\n Returns:\n matplotlib.artist.Artist: The axis containing the plot. This\n may be a newly created axis.\n\n Raises:\n NotImplementedError: If the curve's dimension is not ``2``.\n \"\"\"\n if self._dimension != 2:\n raise NotImplementedError(\n \"2D is the only supported dimension\",\n \"Current dimension\",\n self._dimension,\n )\n\n s_vals = np.linspace(0.0, 1.0, num_pts)\n points = self.evaluate_multi(s_vals)\n if ax is None:\n ax = _plot_helpers.new_axis()\n ax.plot(points[0, :], points[1, :], color=color, alpha=alpha)\n return ax"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nfind the points of intersection with another curve.", "response": "def intersect(\n self, other, strategy=IntersectionStrategy.GEOMETRIC, _verify=True\n ):\n \"\"\"Find the points of intersection with another curve.\n\n See :doc:`../../algorithms/curve-curve-intersection` for more details.\n\n .. image:: ../../images/curve_intersect.png\n :align: center\n\n .. doctest:: curve-intersect\n :options: +NORMALIZE_WHITESPACE\n\n >>> nodes1 = np.asfortranarray([\n ... [0.0, 0.375, 0.75 ],\n ... [0.0, 0.75 , 0.375],\n ... ])\n >>> curve1 = bezier.Curve(nodes1, degree=2)\n >>> nodes2 = np.asfortranarray([\n ... [0.5, 0.5 ],\n ... [0.0, 0.75],\n ... ])\n >>> curve2 = bezier.Curve(nodes2, degree=1)\n >>> intersections = curve1.intersect(curve2)\n >>> 3.0 * intersections\n array([[2.],\n [2.]])\n >>> s_vals = intersections[0, :]\n >>> curve1.evaluate_multi(s_vals)\n array([[0.5],\n [0.5]])\n\n .. testcleanup:: curve-intersect\n\n import make_images\n make_images.curve_intersect(curve1, curve2, s_vals)\n\n Args:\n other (Curve): Other curve to intersect with.\n strategy (Optional[~bezier.curve.IntersectionStrategy]): The\n intersection algorithm to use. Defaults to geometric.\n _verify (Optional[bool]): Indicates if extra caution should be\n used to verify assumptions about the input and current\n curve. Can be disabled to speed up execution time.\n Defaults to :data:`True`.\n\n Returns:\n numpy.ndarray: ``2 x N`` array of ``s``- and ``t``-parameters where\n intersections occur (possibly empty).\n\n Raises:\n TypeError: If ``other`` is not a curve (and ``_verify=True``).\n NotImplementedError: If at least one of the curves\n isn't two-dimensional (and ``_verify=True``).\n ValueError: If ``strategy`` is not a valid\n :class:`.IntersectionStrategy`.\n \"\"\"\n if _verify:\n if not isinstance(other, Curve):\n raise TypeError(\n \"Can only intersect with another curve\", \"Received\", other\n )\n\n if self._dimension != 2 or other._dimension != 2:\n raise NotImplementedError(\n \"Intersection only implemented in 2D\"\n )\n\n if strategy == IntersectionStrategy.GEOMETRIC:\n all_intersections = _geometric_intersection.all_intersections\n elif strategy == IntersectionStrategy.ALGEBRAIC:\n all_intersections = _algebraic_intersection.all_intersections\n else:\n raise ValueError(\"Unexpected strategy.\", strategy)\n\n st_vals, _ = all_intersections(self._nodes, other._nodes)\n return st_vals"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef elevate(self):\n new_nodes = _curve_helpers.elevate_nodes(self._nodes)\n return Curve(new_nodes, self._degree + 1, _copy=False)", "response": "Return a degree - elevated version of the current curve."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef reduce_(self):\n new_nodes = _curve_helpers.reduce_pseudo_inverse(self._nodes)\n return Curve(new_nodes, self._degree - 1, _copy=False)", "response": "r Returns a degree - reduced version of the current curve."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef specialize(self, start, end):\n new_nodes = _curve_helpers.specialize_curve(self._nodes, start, end)\n return Curve(new_nodes, self._degree, _copy=False)", "response": "Specialize the curve to a given start and end point."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef locate(self, point):\n if point.shape != (self._dimension, 1):\n point_dimensions = \" x \".join(\n str(dimension) for dimension in point.shape\n )\n msg = _LOCATE_ERROR_TEMPLATE.format(\n self._dimension, self._dimension, point, point_dimensions\n )\n raise ValueError(msg)\n\n return _curve_helpers.locate_point(self._nodes, point)", "response": "r Finds a point on the current curve."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef clean_file(c_source, virtualenv_dirname):\n with open(c_source, \"r\") as file_obj:\n contents = file_obj.read().rstrip()\n # Replace the path to the Cython include files.\n py_version = \"python{}.{}\".format(*sys.version_info[:2])\n lib_path = os.path.join(\n \".nox\", virtualenv_dirname, \"lib\", py_version, \"site-packages\", \"\"\n )\n contents = contents.replace(lib_path, \"\")\n # Write the files back, but strip all trailing whitespace.\n lines = contents.split(\"\\n\")\n with open(c_source, \"w\") as file_obj:\n for line in lines:\n file_obj.write(line.rstrip() + \"\\n\")", "response": "Cleans up local names in a C source file."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef get_version():\n # \"Spoof\" the ``setup.py`` helper modules.\n sys.modules[\"setup_helpers\"] = object()\n sys.modules[\"setup_helpers_macos\"] = object()\n sys.modules[\"setup_helpers_windows\"] = object()\n filename = os.path.join(_ROOT_DIR, \"setup.py\")\n loader = importlib.machinery.SourceFileLoader(\"setup\", filename)\n setup_mod = loader.load_module()\n return setup_mod.VERSION", "response": "Get the current version from setup. py."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef populate_readme(\n version, circleci_build, appveyor_build, coveralls_build, travis_build\n):\n \"\"\"Populates ``README.rst`` with release-specific data.\n\n This is because ``README.rst`` is used on PyPI.\n\n Args:\n version (str): The current version.\n circleci_build (Union[str, int]): The CircleCI build ID corresponding\n to the release.\n appveyor_build (str): The AppVeyor build ID corresponding to the\n release.\n coveralls_build (Union[str, int]): The Coveralls.io build ID\n corresponding to the release.\n travis_build (int): The Travis CI build ID corresponding to\n the release.\n \"\"\"\n with open(RELEASE_README_FILE, \"r\") as file_obj:\n template = file_obj.read()\n contents = template.format(\n version=version,\n circleci_build=circleci_build,\n appveyor_build=appveyor_build,\n coveralls_build=coveralls_build,\n travis_build=travis_build,\n )\n with open(README_FILE, \"w\") as file_obj:\n file_obj.write(contents)", "response": "Populates the README. rst file with release - specific data."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef populate_index(\n version, circleci_build, appveyor_build, coveralls_build, travis_build\n):\n \"\"\"Populates ``docs/index.rst`` with release-specific data.\n\n Args:\n version (str): The current version.\n circleci_build (Union[str, int]): The CircleCI build ID corresponding\n to the release.\n appveyor_build (str): The AppVeyor build ID corresponding to the\n release.\n coveralls_build (Union[str, int]): The Coveralls.io build ID\n corresponding to the release.\n travis_build (int): The Travis CI build ID corresponding to\n the release.\n \"\"\"\n with open(RELEASE_INDEX_FILE, \"r\") as file_obj:\n template = file_obj.read()\n contents = template.format(\n version=version,\n circleci_build=circleci_build,\n appveyor_build=appveyor_build,\n coveralls_build=coveralls_build,\n travis_build=travis_build,\n )\n with open(INDEX_FILE, \"w\") as file_obj:\n file_obj.write(contents)", "response": "Populates the index. rst file with the release - specific data."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\npopulate binary - extension. rst with release - specific data.", "response": "def populate_native_libraries(version):\n \"\"\"Populates ``binary-extension.rst`` with release-specific data.\n\n Args:\n version (str): The current version.\n \"\"\"\n with open(BINARY_EXT_TEMPLATE, \"r\") as file_obj:\n template = file_obj.read()\n contents = template.format(revision=version)\n with open(BINARY_EXT_FILE, \"w\") as file_obj:\n file_obj.write(contents)"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\npopulating the DEVELOPMENT. rst file with release - specific data.", "response": "def populate_development(version):\n \"\"\"Populates ``DEVELOPMENT.rst`` with release-specific data.\n\n This is because ``DEVELOPMENT.rst`` is used in the Sphinx documentation.\n\n Args:\n version (str): The current version.\n \"\"\"\n with open(DEVELOPMENT_TEMPLATE, \"r\") as file_obj:\n template = file_obj.read()\n contents = template.format(revision=version, rtd_version=version)\n with open(DEVELOPMENT_FILE, \"w\") as file_obj:\n file_obj.write(contents)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef main():\n version = get_version()\n circleci_build = six.moves.input(\"CircleCI Build ID: \")\n appveyor_build = six.moves.input(\"AppVeyor Build ID: \")\n coveralls_build = six.moves.input(\"Coveralls Build ID: \")\n travis_build = six.moves.input(\"Travis Build ID: \")\n populate_readme(\n version, circleci_build, appveyor_build, coveralls_build, travis_build\n )\n populate_index(\n version, circleci_build, appveyor_build, coveralls_build, travis_build\n )\n populate_native_libraries(version)\n populate_development(version)", "response": "Populate the templates with release - specific fields."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef make_subdivision_matrices(degree):\n left = np.zeros((degree + 1, degree + 1), order=\"F\")\n right = np.zeros((degree + 1, degree + 1), order=\"F\")\n left[0, 0] = 1.0\n right[-1, -1] = 1.0\n for col in six.moves.xrange(1, degree + 1):\n half_prev = 0.5 * left[:col, col - 1]\n left[:col, col] = half_prev\n left[1 : col + 1, col] += half_prev # noqa: E203\n # Populate the complement col (in right) as well.\n complement = degree - col\n # NOTE: We \"should\" reverse the results when using\n # the complement, but they are symmetric so\n # that would be a waste.\n right[-(col + 1) :, complement] = left[: col + 1, col] # noqa: E203\n return left, right", "response": "This function creates the matrix used to subdivide a node into left and right nodes respectively."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nsubdividing a curve into two sub - curves.", "response": "def _subdivide_nodes(nodes):\n \"\"\"Subdivide a curve into two sub-curves.\n\n Does so by taking the unit interval (i.e. the domain of the surface) and\n splitting it into two sub-intervals by splitting down the middle.\n\n .. note::\n\n There is also a Fortran implementation of this function, which\n will be used if it can be built.\n\n Args:\n nodes (numpy.ndarray): The nodes defining a B |eacute| zier curve.\n\n Returns:\n Tuple[numpy.ndarray, numpy.ndarray]: The nodes for the two sub-curves.\n \"\"\"\n _, num_nodes = np.shape(nodes)\n if num_nodes == 2:\n left_nodes = _helpers.matrix_product(nodes, _LINEAR_SUBDIVIDE_LEFT)\n right_nodes = _helpers.matrix_product(nodes, _LINEAR_SUBDIVIDE_RIGHT)\n elif num_nodes == 3:\n left_nodes = _helpers.matrix_product(nodes, _QUADRATIC_SUBDIVIDE_LEFT)\n right_nodes = _helpers.matrix_product(\n nodes, _QUADRATIC_SUBDIVIDE_RIGHT\n )\n elif num_nodes == 4:\n left_nodes = _helpers.matrix_product(nodes, _CUBIC_SUBDIVIDE_LEFT)\n right_nodes = _helpers.matrix_product(nodes, _CUBIC_SUBDIVIDE_RIGHT)\n else:\n left_mat, right_mat = make_subdivision_matrices(num_nodes - 1)\n left_nodes = _helpers.matrix_product(nodes, left_mat)\n right_nodes = _helpers.matrix_product(nodes, right_mat)\n return left_nodes, right_nodes"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _evaluate_multi_barycentric(nodes, lambda1, lambda2):\n # NOTE: We assume but don't check that lambda2 has the same shape.\n num_vals, = lambda1.shape\n dimension, num_nodes = nodes.shape\n degree = num_nodes - 1\n # Resize as row vectors for broadcast multiplying with\n # columns of ``nodes``.\n lambda1 = lambda1[np.newaxis, :]\n lambda2 = lambda2[np.newaxis, :]\n result = np.zeros((dimension, num_vals), order=\"F\")\n result += lambda1 * nodes[:, [0]]\n binom_val = 1.0\n lambda2_pow = np.ones((1, num_vals), order=\"F\")\n for index in six.moves.xrange(1, degree):\n lambda2_pow *= lambda2\n binom_val = (binom_val * (degree - index + 1)) / index\n result += binom_val * lambda2_pow * nodes[:, [index]]\n result *= lambda1\n result += lambda2 * lambda2_pow * nodes[:, [degree]]\n return result", "response": "r Evaluates a B |eacute| zier type - function."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef vec_size(nodes, s_val):\n result_vec = evaluate_multi(nodes, np.asfortranarray([s_val]))\n # NOTE: We convert to 1D to make sure NumPy uses vector norm.\n return np.linalg.norm(result_vec[:, 0], ord=2)", "response": "r Compute the size of a set of nodes."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef _elevate_nodes(nodes):\n dimension, num_nodes = np.shape(nodes)\n new_nodes = np.empty((dimension, num_nodes + 1), order=\"F\")\n multipliers = np.arange(1, num_nodes, dtype=_FLOAT64)[np.newaxis, :]\n denominator = float(num_nodes)\n new_nodes[:, 1:-1] = (\n multipliers * nodes[:, :-1]\n + (denominator - multipliers) * nodes[:, 1:]\n )\n # Hold off on division until the end, to (attempt to) avoid round-off.\n new_nodes /= denominator\n # After setting the internal nodes (which require division), set the\n # boundary nodes.\n new_nodes[:, 0] = nodes[:, 0]\n new_nodes[:, -1] = nodes[:, -1]\n return new_nodes", "response": "r Degree - elevate a B |eacute| zier curves."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nperform one round of de Casteljau s algorithm.", "response": "def de_casteljau_one_round(nodes, lambda1, lambda2):\n \"\"\"Perform one round of de Casteljau's algorithm.\n\n .. note::\n\n This is a helper for :func:`_specialize_curve`. It does not have a\n Fortran speedup because it is **only** used by a function which has\n a Fortran speedup.\n\n The weights are assumed to sum to one.\n\n Args:\n nodes (numpy.ndarray): Control points for a curve.\n lambda1 (float): First barycentric weight on interval.\n lambda2 (float): Second barycentric weight on interval.\n\n Returns:\n numpy.ndarray: The nodes for a \"blended\" curve one degree\n lower.\n \"\"\"\n return np.asfortranarray(lambda1 * nodes[:, :-1] + lambda2 * nodes[:, 1:])"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nspecialize a specialization of a single internal control point.", "response": "def _specialize_curve(nodes, start, end):\n \"\"\"Specialize a curve to a re-parameterization\n\n .. note::\n\n This assumes the curve is degree 1 or greater but doesn't check.\n\n .. note::\n\n There is also a Fortran implementation of this function, which\n will be used if it can be built.\n\n Args:\n nodes (numpy.ndarray): Control points for a curve.\n start (float): The start point of the interval we are specializing to.\n end (float): The end point of the interval we are specializing to.\n\n Returns:\n numpy.ndarray: The control points for the specialized curve.\n \"\"\"\n # NOTE: There is no corresponding \"enable\", but the disable only applies\n # in this lexical scope.\n # pylint: disable=too-many-locals\n _, num_nodes = np.shape(nodes)\n # Uses start-->0, end-->1 to represent the specialization used.\n weights = ((1.0 - start, start), (1.0 - end, end))\n partial_vals = {\n (0,): de_casteljau_one_round(nodes, *weights[0]),\n (1,): de_casteljau_one_round(nodes, *weights[1]),\n }\n for _ in six.moves.xrange(num_nodes - 2, 0, -1):\n new_partial = {}\n for key, sub_nodes in six.iteritems(partial_vals):\n # Our keys are ascending so we increment from the last value.\n for next_id in six.moves.xrange(key[-1], 1 + 1):\n new_key = key + (next_id,)\n new_partial[new_key] = de_casteljau_one_round(\n sub_nodes, *weights[next_id]\n )\n partial_vals = new_partial\n result = np.empty(nodes.shape, order=\"F\")\n for index in six.moves.xrange(num_nodes):\n key = (0,) * (num_nodes - index - 1) + (1,) * index\n result[:, [index]] = partial_vals[key]\n return result"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _evaluate_hodograph(s, nodes):\n _, num_nodes = np.shape(nodes)\n first_deriv = nodes[:, 1:] - nodes[:, :-1]\n return (num_nodes - 1) * evaluate_multi(\n first_deriv, np.asfortranarray([s])\n )", "response": "r Evaluates the Hodograph curve at a point s."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef _newton_refine(nodes, point, s):\n pt_delta = point - evaluate_multi(nodes, np.asfortranarray([s]))\n derivative = evaluate_hodograph(s, nodes)\n # Each array is 2 x 1 (i.e. a column vector), we want the vector\n # dot product.\n delta_s = np.vdot(pt_delta[:, 0], derivative[:, 0]) / np.vdot(\n derivative[:, 0], derivative[:, 0]\n )\n return s + delta_s", "response": "r Re - fine a solution to a newton s method."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _locate_point(nodes, point):\n candidates = [(0.0, 1.0, nodes)]\n for _ in six.moves.xrange(_MAX_LOCATE_SUBDIVISIONS + 1):\n next_candidates = []\n for start, end, candidate in candidates:\n if _helpers.contains_nd(candidate, point.ravel(order=\"F\")):\n midpoint = 0.5 * (start + end)\n left, right = subdivide_nodes(candidate)\n next_candidates.extend(\n ((start, midpoint, left), (midpoint, end, right))\n )\n candidates = next_candidates\n if not candidates:\n return None\n\n params = [(start, end) for start, end, _ in candidates]\n if np.std(params) > _LOCATE_STD_CAP:\n raise ValueError(\"Parameters not close enough to one another\", params)\n\n s_approx = np.mean(params)\n s_approx = newton_refine(nodes, point, s_approx)\n # NOTE: Since ``np.mean(params)`` must be in ``[0, 1]`` it's\n # \"safe\" to push the Newton-refined value back into the unit\n # interval.\n if s_approx < 0.0:\n return 0.0\n\n elif s_approx > 1.0:\n return 1.0\n\n else:\n return s_approx", "response": "r Returns a new node that is located on a curve."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef _reduce_pseudo_inverse(nodes):\n _, num_nodes = np.shape(nodes)\n if num_nodes == 2:\n reduction = _REDUCTION0\n denom = _REDUCTION_DENOM0\n elif num_nodes == 3:\n reduction = _REDUCTION1\n denom = _REDUCTION_DENOM1\n elif num_nodes == 4:\n reduction = _REDUCTION2\n denom = _REDUCTION_DENOM2\n elif num_nodes == 5:\n reduction = _REDUCTION3\n denom = _REDUCTION_DENOM3\n else:\n raise _helpers.UnsupportedDegree(num_nodes - 1, supported=(1, 2, 3, 4))\n\n result = _helpers.matrix_product(nodes, reduction)\n result /= denom\n return result", "response": "Performs a pseudo - inverse degree reduction for a B |eacute| zier curve."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\ncomputes the error between nodes and projected nodes.", "response": "def projection_error(nodes, projected):\n \"\"\"Compute the error between ``nodes`` and the projected nodes.\n\n .. note::\n\n This is a helper for :func:`maybe_reduce`, which is in turn a helper\n for :func:`_full_reduce`. Hence there is no corresponding Fortran\n speedup.\n\n For now, just compute the relative error in the Frobenius norm. But,\n we may wish to consider the error per row / point instead.\n\n Args:\n nodes (numpy.ndarray): Nodes in a curve.\n projected (numpy.ndarray): The ``nodes`` projected into the\n space of degree-elevated nodes.\n\n Returns:\n float: The relative error.\n \"\"\"\n relative_err = np.linalg.norm(nodes - projected, ord=\"fro\")\n if relative_err != 0.0:\n relative_err /= np.linalg.norm(nodes, ord=\"fro\")\n return relative_err"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _full_reduce(nodes):\n was_reduced, nodes = maybe_reduce(nodes)\n while was_reduced:\n was_reduced, nodes = maybe_reduce(nodes)\n return nodes", "response": "Apply degree reduction to nodes until it can no longer be reduced."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef get_public_members():\n if bezier is None:\n return []\n\n local_members = []\n all_members = set()\n for name in dir(bezier):\n # Filter out non-public.\n if name.startswith(\"_\") and name not in SPECIAL_MEMBERS:\n continue\n\n value = getattr(bezier, name)\n # Filter out imported modules.\n if isinstance(value, types.ModuleType):\n continue\n\n all_members.add(name)\n # Only keep values defined in the base package.\n home = getattr(value, \"__module__\", \"bezier\")\n if home == \"bezier\":\n local_members.append(name)\n size_all = len(bezier.__all__)\n all_exports = set(bezier.__all__)\n if len(all_exports) != size_all:\n raise ValueError(\"__all__ has repeated elements\")\n\n if all_exports != all_members:\n raise ValueError(\n \"__all__ does not agree with the publicly imported members\",\n all_exports,\n all_members,\n )\n\n local_members = [\n member\n for member in local_members\n if member not in UNDOCUMENTED_SPECIAL_MEMBERS\n ]\n return local_members", "response": "Get all public members in the main package."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef get_desired():\n public_members = get_public_members()\n if public_members:\n members = \"\\n :members: {}\".format(\", \".join(public_members))\n else:\n members = \"\"\n return DESIRED_TEMPLATE.format(members=members)", "response": "Populate DESIRED_TEMPLATE with public members."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef main():\n with open(FILENAME, \"r\") as file_obj:\n contents = file_obj.read()\n desired = get_desired()\n if contents == EXPECTED:\n with open(FILENAME, \"w\") as file_obj:\n file_obj.write(desired)\n elif contents != desired:\n raise ValueError(\"Unexpected contents\", contents, \"Expected\", EXPECTED)", "response": "Main entry point to replace the generated contents."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\npatch up the Fortran compiler and ignore other compilers.", "response": "def patch_f90_compiler(f90_compiler):\n \"\"\"Patch up ``f90_compiler.library_dirs``.\n\n Updates flags in ``gfortran`` and ignores other compilers. The only\n modification is the removal of ``-fPIC`` since it is not used on Windows\n and the build flags turn warnings into errors.\n\n Args:\n f90_compiler (numpy.distutils.fcompiler.FCompiler): A Fortran compiler\n instance.\n \"\"\"\n # NOTE: NumPy may not be installed, but we don't want **this** module to\n # cause an import failure.\n from numpy.distutils.fcompiler import gnu\n\n # Only Windows.\n if os.name != \"nt\":\n return\n\n # Only ``gfortran``.\n if not isinstance(f90_compiler, gnu.Gnu95FCompiler):\n return\n\n f90_compiler.compiler_f77[:] = _remove_fpic(f90_compiler.compiler_f77)\n f90_compiler.compiler_f90[:] = _remove_fpic(f90_compiler.compiler_f90)\n c_compiler = f90_compiler.c_compiler\n if c_compiler.compiler_type != \"msvc\":\n raise NotImplementedError(\n \"MSVC is the only supported C compiler on Windows.\"\n )"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nclean up build files.", "response": "def clean(session):\n \"\"\"Clean up build files.\n\n Cleans up all artifacts that might get created during\n other ``nox`` sessions.\n\n There is no need for the session to create a ``virtualenv``\n here (we are just pretending to be ``make``).\n \"\"\"\n clean_dirs = (\n get_path(\".cache\"),\n get_path(\".coverage\"),\n get_path(\".pytest_cache\"),\n get_path(\"__pycache__\"),\n get_path(\"build\"),\n get_path(\"dist\"),\n get_path(\"docs\", \"__pycache__\"),\n get_path(\"docs\", \"build\"),\n get_path(\"scripts\", \"macos\", \"__pycache__\"),\n get_path(\"scripts\", \"macos\", \"dist_wheels\"),\n get_path(\"scripts\", \"macos\", \"fixed_wheels\"),\n get_path(\"src\", \"bezier.egg-info\"),\n get_path(\"src\", \"bezier\", \"__pycache__\"),\n get_path(\"src\", \"bezier\", \"extra-dll\"),\n get_path(\"src\", \"bezier\", \"lib\"),\n get_path(\"tests\", \"__pycache__\"),\n get_path(\"tests\", \"functional\", \"__pycache__\"),\n get_path(\"tests\", \"unit\", \"__pycache__\"),\n get_path(\"wheelhouse\"),\n )\n clean_globs = (\n get_path(\".coverage\"),\n get_path(\"*.mod\"),\n get_path(\"*.pyc\"),\n get_path(\"src\", \"bezier\", \"*.pyc\"),\n get_path(\"src\", \"bezier\", \"*.pyd\"),\n get_path(\"src\", \"bezier\", \"*.so\"),\n get_path(\"src\", \"bezier\", \"quadpack\", \"*.o\"),\n get_path(\"src\", \"bezier\", \"*.o\"),\n get_path(\"tests\", \"*.pyc\"),\n get_path(\"tests\", \"functional\", \"*.pyc\"),\n get_path(\"tests\", \"unit\", \"*.pyc\"),\n )\n for dir_path in clean_dirs:\n session.run(shutil.rmtree, dir_path, ignore_errors=True)\n for glob_path in clean_globs:\n for filename in glob.glob(glob_path):\n session.run(os.remove, filename)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef get_app_submodules(module_name):\n for name, module in get_app_modules():\n if module_has_submodule(module, module_name):\n yield name, import_module('{0}.{1}'.format(name, module_name))", "response": "Get a specific sub module of the app\n "} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nregister all hooks found in auth_hooks sub modules", "response": "def register_all_hooks():\n \"\"\"\n Register all hooks found in 'auth_hooks' sub modules\n :return:\n \"\"\"\n global _all_hooks_registered\n if not _all_hooks_registered:\n logger.debug(\"Searching for hooks\")\n hooks = list(get_app_submodules('auth_hooks'))\n logger.debug(\"Got %s hooks\" % len(hooks))\n _all_hooks_registered = True"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nsending an HTTP request to SeAT API", "response": "def exec_request(endpoint, func, raise_for_status=False, **kwargs):\n \"\"\" Send an https api request \"\"\"\n try:\n endpoint = '{0}/api/v1/{1}'.format(settings.SEAT_URL, endpoint)\n headers = {'X-Token': settings.SEAT_XTOKEN, 'Accept': 'application/json'}\n logger.debug(headers)\n logger.debug(endpoint)\n ret = getattr(requests, func)(endpoint, headers=headers, data=kwargs)\n ret.raise_for_status()\n return ret.json()\n except requests.HTTPError as e:\n if raise_for_status:\n raise e\n logger.exception(\"Error encountered while performing API request to SeAT with url {}\".format(endpoint))\n return {}"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef add_user(cls, username, email):\n sanitized = str(cls.__sanitize_username(username))\n logger.debug(\"Adding user to SeAT with username %s\" % sanitized)\n password = cls.__generate_random_pass()\n ret = cls.exec_request('user', 'post', username=sanitized, email=str(email), password=password)\n logger.debug(ret)\n if cls._response_ok(ret):\n logger.info(\"Added SeAT user with username %s\" % sanitized)\n return sanitized, password\n logger.info(\"Failed to add SeAT user with username %s\" % sanitized)\n return None, None", "response": "Add user to SeAT service"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\ndeletes user with username username", "response": "def delete_user(cls, username):\n \"\"\" Delete user \"\"\"\n ret = cls.exec_request('user/{}'.format(username), 'delete')\n logger.debug(ret)\n if cls._response_ok(ret):\n logger.info(\"Deleted SeAT user with username %s\" % username)\n return username\n return None"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nenabling user with username username", "response": "def enable_user(cls, username):\n \"\"\" Enable user \"\"\"\n ret = cls.exec_request('user/{}'.format(username), 'put', account_status=1)\n logger.debug(ret)\n if cls._response_ok(ret):\n logger.info(\"Enabled SeAT user with username %s\" % username)\n return username\n logger.info(\"Failed to enabled SeAT user with username %s\" % username)\n return None"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\ndisables user with username username", "response": "def disable_user(cls, username):\n \"\"\" Disable user \"\"\"\n cls.update_roles(username, [])\n ret = cls.exec_request('user/{}'.format(username), 'put', account_status=0)\n logger.debug(ret)\n if cls._response_ok(ret):\n logger.info(\"Disabled SeAT user with username %s\" % username)\n return username\n logger.info(\"Failed to disable SeAT user with username %s\" % username)\n return None"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\ncomparing email to one set on SeAT", "response": "def _check_email_changed(cls, username, email):\n \"\"\"Compares email to one set on SeAT\"\"\"\n ret = cls.exec_request('user/{}'.format(username), 'get', raise_for_status=True)\n return ret['email'] != email"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nreturns the name of the log entry.", "response": "def format_name(self):\n \"\"\"\n :return: str Generated name\n \"\"\"\n format_data = self.get_format_data()\n return Formatter().vformat(self.string_formatter, args=[], kwargs=format_data)"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\ninject the request user into the kwargs passed to the form", "response": "def get_form_kwargs(self):\n \"\"\"\n Inject the request user into the kwargs passed to the form\n \"\"\"\n kwargs = super(AddUpdateMixin, self).get_form_kwargs()\n kwargs.update({'user': self.request.user})\n return kwargs"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef get_object(self, queryset=None):\n if queryset is None:\n queryset = self.get_queryset()\n\n try:\n return queryset.get(user__pk=self.request.user.pk)\n except ObjectDoesNotExist:\n raise Http404", "response": "Returns the object that the view is displaying."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef can_manage_groups(cls, user):\n if user.is_authenticated:\n return cls.has_management_permission(user) or user.leads_groups.all()\n return False", "response": "Check if the user can manage groups."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef can_manage_group(cls, user, group):\n if user.is_authenticated:\n return cls.has_management_permission(user) or user.leads_groups.filter(group=group).exists()\n return False", "response": "Check if the given user can manage the given group"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef create_auth_group(sender, instance, created, **kwargs):\n if created:\n AuthGroup.objects.create(group=instance)", "response": "Create the AuthGroup model when a group is created"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef construct_command(self, command, keys=None, opts=None):\n\n cstr = [command]\n\n # Add the keys and values, escape as needed\n if keys:\n for key in keys:\n if isinstance(keys[key], list):\n ncstr = []\n for nest in keys[key]:\n ncstr.append(\"%s=%s\" % (key, self._escape_str(nest)))\n cstr.append(\"|\".join(ncstr))\n else:\n cstr.append(\"%s=%s\" % (key, self._escape_str(keys[key])))\n\n # Add in options\n if opts:\n for opt in opts:\n cstr.append(\"-%s\" % opt)\n\n return \" \".join(cstr)", "response": "Construct a TS3 formatted command string"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef parse_command(self, commandstr):\n\n if len(commandstr.split('|')) > 1:\n vals = []\n for cmd in commandstr.split('|'):\n vals.append(self.parse_command(cmd))\n return vals\n\n cmdlist = commandstr.strip().split(' ')\n command = None\n keys = {}\n opts = []\n\n for key in cmdlist:\n v = key.strip().split('=')\n if len(v) > 1:\n # Key\n if len(v) > 2:\n # Fix the stupidities in TS3 escaping\n v = [v[0], '='.join(v[1:])]\n key, value = v\n keys[key] = self._unescape_str(value)\n elif not v == ['']:\n if v[0][0] and v[0][0] == '-':\n # Option\n opts.append(v[0][1:])\n else:\n command = v[0]\n\n d = {'keys': keys, 'opts': opts}\n if command:\n d['command'] = command\n return d", "response": "Parses a TS3 command string into a dictionary of command keys and opts."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nescaping a value into a TS3 compatible string", "response": "def _escape_str(value):\n \"\"\"\n Escape a value into a TS3 compatible string\n @param value: Value\n @type value: string/int\n \"\"\"\n\n if isinstance(value, int):\n return \"%d\" % value\n value = value.replace(\"\\\\\", r'\\\\')\n for i, j in ts3_escape.items():\n value = value.replace(i, j)\n return value"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _unescape_str(value):\n\n if isinstance(value, int):\n return \"%d\" % value\n value = value.replace(r\"\\\\\", \"\\\\\")\n for i, j in ts3_escape.items():\n value = value.replace(j, i)\n return value", "response": "Unescape a TS3 compatible string into a normal string"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nlogin to the TS3 Server", "response": "def login(self, username, password):\n \"\"\"\n Login to the TS3 Server\n @param username: Username\n @type username: str\n @param password: Password\n @type password: str\n \"\"\"\n d = self.send_command('login', keys={'client_login_name': username, 'client_login_password': password})\n if d == 0:\n self._log.info('Login Successful')\n return True\n return False"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef use(self, id):\n if self._connected and id > 0:\n self.send_command('use', keys={'sid': id})", "response": "Use a particular Virtual Server instance"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef render_services_ctrl(self, request):\n urls = self.Urls()\n urls.auth_activate = 'auth_example_activate'\n urls.auth_deactivate = 'auth_example_deactivate'\n urls.auth_reset_password = 'auth_example_reset_password'\n urls.auth_set_password = 'auth_example_set_password'\n return render_to_string(self.service_ctrl_template, {\n 'service_name': self.title,\n 'urls': urls,\n 'service_url': self.service_url,\n 'username': 'example username'\n }, request=request)", "response": "Example for rendering the services control panel row"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef alliance(self) -> Union[EveAllianceInfo, None]:\n if self.alliance_id is None:\n return None\n return EveAllianceInfo.objects.get(alliance_id=self.alliance_id)", "response": "Returns the EveAllianceInfo object for this alliance."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef pre_save_config(sender, instance, *args, **kwargs):\n logger.debug(\"Received pre_save from {}\".format(instance))\n if not instance.pk:\n # new model being created\n return\n try:\n old_instance = AutogroupsConfig.objects.get(pk=instance.pk)\n\n # Check if enable was toggled, delete groups?\n if old_instance.alliance_groups is True and instance.alliance_groups is False:\n instance.delete_alliance_managed_groups()\n\n if old_instance.corp_groups is True and instance.corp_groups is False:\n instance.delete_corp_managed_groups()\n except AutogroupsConfig.DoesNotExist:\n pass", "response": "Checks if enable was toggled on group config and deletes groups if necessary."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef check_groups_on_profile_update(sender, instance, created, *args, **kwargs):\n AutogroupsConfig.objects.update_groups_for_user(instance.user)", "response": "Trigger check when main character or state changes."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\ntriggering group membership update when a state is added or removed from a group.", "response": "def autogroups_states_changed(sender, instance, action, reverse, model, pk_set, *args, **kwargs):\n \"\"\"\n Trigger group membership update when a state is added or removed from\n an autogroup config.\n \"\"\"\n if action.startswith('post_'):\n for pk in pk_set:\n try:\n state = State.objects.get(pk=pk)\n instance.update_group_membership_for_state(state)\n except State.DoesNotExist:\n # Deleted States handled by the profile state change\n pass"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nreturning a function that can be used to update the groups for the selected users", "response": "def make_service_hooks_update_groups_action(service):\n \"\"\"\n Make a admin action for the given service\n :param service: services.hooks.ServicesHook\n :return: fn to update services groups for the selected users\n \"\"\"\n def update_service_groups(modeladmin, request, queryset):\n for user in queryset: # queryset filtering doesn't work here?\n service.update_groups(user)\n\n update_service_groups.__name__ = str('update_{}_groups'.format(slugify(service.name)))\n update_service_groups.short_description = \"Sync groups for selected {} accounts\".format(service.title)\n return update_service_groups"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nreturns a function that will be used to sync the nicknames for the selected users", "response": "def make_service_hooks_sync_nickname_action(service):\n \"\"\"\n Make a sync_nickname admin action for the given service\n :param service: services.hooks.ServicesHook\n :return: fn to sync nickname for the selected users\n \"\"\"\n def sync_nickname(modeladmin, request, queryset):\n for user in queryset: # queryset filtering doesn't work here?\n service.sync_nickname(user)\n\n sync_nickname.__name__ = str('sync_{}_nickname'.format(slugify(service.name)))\n sync_nickname.short_description = \"Sync nicknames for selected {} accounts\".format(service.title)\n return sync_nickname"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nreturning the active or not active class for the current page.", "response": "def renavactive(request, pattern):\n \"\"\"\n {% renavactive request \"^/a_regex\" %}\n \"\"\"\n if re.search(pattern, request.path):\n return getattr(settings, \"NAVHELPER_ACTIVE_CLASS\", \"active\")\n return getattr(settings, \"NAVHELPER_NOT_ACTIVE_CLASS\", \"\")"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nreturn the active navclass for the given urls.", "response": "def navactive(request, urls):\n \"\"\"\n {% navactive request \"view_name another_view_name\" %}\n \"\"\"\n url_list = set(urls.split())\n\n resolved = resolve(request.path)\n resolved_urls = set()\n if resolved.url_name:\n resolved_urls.add(resolved.url_name)\n if resolved.namespaces:\n resolved_urls = resolved_urls.union([\"{}:{}\".format(namespace, resolved.url_name) for namespace in resolved.namespaces])\n resolved_urls = resolved_urls.union([\"{}:\".format(namespace) for namespace in resolved.namespaces])\n if getattr(resolved, 'app_name', None):\n resolved_urls = resolved_urls.union([\"{}:{}\".format(resolved.app_name, resolved.url_name), \"{}:\".format(resolved.app_name)])\n if getattr(resolved, 'app_names', []):\n resolved_urls = resolved_urls.union([\"{}:{}\".format(app_name, resolved.url_name) for app_name in resolved.app_names])\n resolved_urls = resolved_urls.union([\"{}:\".format(app_name) for app_name in resolved.app_names])\n if url_list and resolved_urls and bool(resolved_urls & url_list):\n return getattr(settings, \"NAVHELPER_ACTIVE_CLASS\", \"active\")\n return getattr(settings, \"NAVHELPER_NOT_ACTIVE_CLASS\", \"\")"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef update_groups_for_state(self, state: State):\n users = get_users_for_state(state)\n for config in self.filter(states=state):\n logger.debug(\"in state loop\")\n for user in users:\n logger.debug(\"in user loop for {}\".format(user))\n config.update_group_membership_for_user(user)", "response": "Update all the Group memberships for the users who have a state."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef update_groups_for_user(self, user: User, state: State = None):\n if state is None:\n state = user.profile.state\n for config in self.filter(states=state):\n # grant user new groups for their state\n config.update_group_membership_for_user(user)\n for config in self.exclude(states=state):\n # ensure user does not have groups from previous state\n config.remove_user_from_alliance_groups(user)\n config.remove_user_from_corp_groups(user)", "response": "Update the group memberships for the given users state"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef _replace_spaces(self, name: str) -> str:\n if self.replace_spaces:\n return name.strip().replace(' ', str(self.replace_spaces_with))\n return name", "response": "Replace spaces in the given name based on the config\n property."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef api_backoff(func):\n\n class PerformBackoff(Exception):\n def __init__(self, retry_after, retry_datetime, global_ratelimit):\n super(Exception, self).__init__()\n self.retry_after = int(retry_after)\n self.retry_datetime = retry_datetime\n self.global_ratelimit = global_ratelimit\n\n @wraps(func)\n def decorated(*args, **kwargs):\n blocking = kwargs.get('blocking', False)\n retries = kwargs.get('max_retries', 3)\n\n # Strip our parameters\n if 'max_retries' in kwargs:\n del kwargs['max_retries']\n if 'blocking' in kwargs:\n del kwargs['blocking']\n\n cache_key = 'DISCORD_BACKOFF_' + func.__name__\n cache_global_key = 'DISCORD_BACKOFF_GLOBAL'\n\n while retries > 0:\n try:\n try:\n # Check global backoff first, then route backoff\n existing_global_backoff = cache.get(cache_global_key)\n existing_backoff = existing_global_backoff or cache.get(cache_key)\n if existing_backoff:\n backoff_timer = datetime.datetime.strptime(existing_backoff, cache_time_format)\n if backoff_timer > datetime.datetime.utcnow():\n backoff_seconds = (backoff_timer - datetime.datetime.utcnow()).total_seconds()\n logger.debug(\"Still under backoff for %s seconds, backing off\" % backoff_seconds)\n # Still under backoff\n raise PerformBackoff(\n retry_after=backoff_seconds,\n retry_datetime=backoff_timer,\n global_ratelimit=bool(existing_global_backoff)\n )\n logger.debug(\"Calling API calling function\")\n return func(*args, **kwargs)\n except requests.HTTPError as e:\n if e.response.status_code == 429:\n try:\n retry_after = int(e.response.headers['Retry-After'])\n except (TypeError, KeyError):\n # Pick some random time\n retry_after = 5000\n\n logger.info(\"Received backoff from API of %s seconds, handling\" % retry_after)\n # Store value in redis\n backoff_until = (datetime.datetime.utcnow() +\n datetime.timedelta(milliseconds=retry_after))\n global_backoff = bool(e.response.headers.get('X-RateLimit-Global', False))\n if global_backoff:\n logger.info(\"Global backoff!!\")\n cache.set(cache_global_key, backoff_until.strftime(cache_time_format), retry_after)\n else:\n cache.set(cache_key, backoff_until.strftime(cache_time_format), retry_after)\n raise PerformBackoff(retry_after=retry_after, retry_datetime=backoff_until,\n global_ratelimit=global_backoff)\n else:\n # Not 429, re-raise\n raise e\n except PerformBackoff as bo:\n # Sleep if we're blocking\n if blocking:\n logger.info(\"Blocking Back off from API calls for %s seconds\" % bo.retry_after)\n time.sleep((10 if bo.retry_after > 10 else bo.retry_after) / 1000)\n else:\n # Otherwise raise exception and let caller handle the backoff\n raise DiscordApiBackoff(retry_after=bo.retry_after, global_ratelimit=bo.global_ratelimit)\n finally:\n retries -= 1\n if retries == 0:\n raise DiscordApiTooBusy()\n return decorated", "response": "Decorator for handling API 429 Too Many Requests messages from the Discord API."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef random_string(string_length=10):\n random = str(uuid.uuid4()) # Convert UUID format to a Python string.\n random = random.upper() # Make all characters uppercase.\n random = random.replace(\"-\", \"\") # Remove the UUID '-'.\n return random[0:string_length]", "response": "Returns a random string of length string_length."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef getGenomeList() :\n\timport rabaDB.filters as rfilt\n\tf = rfilt.RabaQuery(Genome_Raba)\n\tnames = []\n\tfor g in f.iterRun() :\n\t\tnames.append(g.name)\n\treturn names", "response": "Return the names of all imported genomes"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\niterating through the codons", "response": "def iterCodons(self) :\n\t\t\"\"\"iterates through the codons\"\"\"\n\t\tfor i in range(len(self.cDNA)/3) :\n\t\t\tyield self.getCodon(i)"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef removeDuplicates(inFileName, outFileName) :\n\tf = open(inFileName)\n\tlegend = f.readline()\n\t\n\tdata = ''\n\th = {}\n\th[legend] = 0\n\t\n\tlines = f.readlines()\n\tfor l in lines :\n\t\tif not h.has_key(l) :\n\t\t\th[l] = 0\n\t\t\tdata += l\n\t\t\t\n\tf.flush()\n\tf.close()\n\tf = open(outFileName, 'w')\n\tf.write(legend+data)\n\tf.flush()\n\tf.close()", "response": "removes duplicated lines from a inFileName CSV file"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef catCSVs(folder, ouputFileName, removeDups = False) :\n\tstrCmd = r\"\"\"cat %s/*.csv > %s\"\"\" %(folder, ouputFileName)\n\tos.system(strCmd)\n\n\tif removeDups :\n\t\tremoveDuplicates(ouputFileName, ouputFileName)", "response": "Concatenates all csv in folder and wites the results in ouputFileName."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef joinCSVs(csvFilePaths, column, ouputFileName, separator = ',') :\n\t\n\tres = ''\n\tlegend = []\n\tcsvs = []\n\t\n\tfor f in csvFilePaths :\n\t\tc = CSVFile()\n\t\tc.parse(f)\n\t\tcsvs.append(c)\n\t\tlegend.append(separator.join(c.legend.keys()))\n\tlegend = separator.join(legend)\n\t\n\tlines = []\n\tfor i in range(len(csvs[0])) :\n\t\tval = csvs[0].get(i, column)\n\t\tline = separator.join(csvs[0][i])\n\t\tfor c in csvs[1:] :\n\t\t\tfor j in range(len(c)) :\n\t\t\t\tif val == c.get(j, column) :\n\t\t\t\t\tline += separator + separator.join(c[j])\n\t\t\t\t\t\n\t\tlines.append( line )\n\t\n\tres = legend + '\\n' + '\\n'.join(lines)\n\t\n\tf = open(ouputFileName, 'w')\n\tf.write(res)\n\tf.flush()\n\tf.close()\n\t\n\treturn res", "response": "This function joins all CSVs according to the values in the column and writes the results in a new file ouputFileName."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef addField(self, field) :\n\t\tif field.lower() in self.legend :\n\t\t\traise ValueError(\"%s is already in the legend\" % field.lower())\n\t\tself.legend[field.lower()] = len(self.legend)\n\t\tif len(self.strLegend) > 0 :\n\t\t\tself.strLegend += self.separator + field\n\t\telse :\n\t\t\tself.strLegend += field", "response": "add a filed to the legend"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef parse(self, filePath, skipLines=0, separator = ',', stringSeparator = '\"', lineSeparator = '\\n') :\n\t\t\n\t\tself.filename = filePath\n\t\tf = open(filePath)\n\t\tif lineSeparator == '\\n' :\n\t\t\tlines = f.readlines()\n\t\telse :\n\t\t\tlines = f.read().split(lineSeparator)\n\t\tf.flush()\n\t\tf.close()\n\t\t\n\t\tlines = lines[skipLines:]\n\t\tself.lines = []\n\t\tself.comments = []\n\t\tfor l in lines :\n\t\t\t# print l\n\t\t\tif len(l) != 0 and l[0] != \"#\" :\n\t\t\t\tself.lines.append(l)\n\t\t\telif l[0] == \"#\" :\n\t\t\t\tself.comments.append(l)\n\n\t\tself.separator = separator\n\t\tself.lineSeparator = lineSeparator\n\t\tself.stringSeparator = stringSeparator\n\t\tself.legend = collections.OrderedDict()\n\t\t\n\t\ti = 0\n\t\tfor c in self.lines[0].lower().replace(stringSeparator, '').split(separator) :\n\t\t\tlegendElement = c.strip()\n\t\t\tif legendElement not in self.legend :\n\t\t\t\tself.legend[legendElement] = i\n\t\t\ti+=1\n\t\n\t\tself.strLegend = self.lines[0].replace('\\r', '\\n').replace('\\n', '')\n\t\tself.lines = self.lines[1:]", "response": "Loads a CSV file and parses it into a list of dictionaries."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nstart a stream to a file.", "response": "def streamToFile(self, filename, keepInMemory = False, writeRate = 1) :\n\t\t\"\"\"Starts a stream to a file. Every line must be committed (l.commit()) to be appended in to the file.\n\n\t\tIf keepInMemory is set to True, the parser will keep a version of the whole CSV in memory, writeRate is the number\n\t\tof lines that must be committed before an automatic save is triggered.\n\t\t\"\"\"\n\t\tif len(self.legend) < 1 :\n\t\t\traise ValueError(\"There's no legend defined\")\n\n\t\ttry :\n\t\t\tos.remove(filename)\n\t\texcept :\n\t\t\tpass\n\t\t\n\t\tself.streamFile = open(filename, \"a\")\n\t\tself.writeRate = writeRate\n\t\tself.streamBuffer = []\n\t\tself.keepInMemory = keepInMemory\n\n\t\tself.streamFile.write(self.strLegend + \"\\n\")"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\ncommits a line to a file and saves the current buffer if needed.", "response": "def commitLine(self, line) :\n\t\t\"\"\"Commits a line making it ready to be streamed to a file and saves the current buffer if needed. If no stream is active, raises a ValueError\"\"\"\n\t\tif self.streamBuffer is None :\n\t\t\traise ValueError(\"Commit lines is only for when you are streaming to a file\")\n\n\t\tself.streamBuffer.append(line)\n\t\tif len(self.streamBuffer) % self.writeRate == 0 :\n\t\t\tfor i in xrange(len(self.streamBuffer)) :\n\t\t\t\tself.streamBuffer[i] = str(self.streamBuffer[i])\n\t\t\tself.streamFile.write(\"%s\\n\" % ('\\n'.join(self.streamBuffer)))\n\t\t\tself.streamFile.flush()\n\t\t\tself.streamBuffer = []"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef closeStreamToFile(self) :\n\t\tif self.streamBuffer is None :\n\t\t\traise ValueError(\"Commit lines is only for when you are streaming to a file\")\n\n\t\tfor i in xrange(len(self.streamBuffer)) :\n\t\t\tself.streamBuffer[i] = str(self.streamBuffer[i])\n\t\tself.streamFile.write('\\n'.join(self.streamBuffer))\n\t\tself.streamFile.close()\n\n\t\tself.streamFile = None\n\t\tself.writeRate = None\n\t\tself.streamBuffer = None\n\t\tself.keepInMemory = True", "response": "Appends the remaining lines and closes the stream."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nappend an empty line at the end of the CSV and returns it", "response": "def newLine(self) :\n\t\t\"\"\"Appends an empty line at the end of the CSV and returns it\"\"\"\n\t\tl = CSVEntry(self)\n\t\tif self.keepInMemory :\n\t\t\tself.lines.append(l)\n\t\treturn l"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\ninserting an empty line at position i and returns it", "response": "def insertLine(self, i) :\n\t\t\"\"\"Inserts an empty line at position i and returns it\"\"\"\n\t\tself.data.insert(i, CSVEntry(self))\n\t\treturn self.lines[i]"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef save(self, filePath) :\n\t\tself.filename = filePath\n\t\tf = open(filePath, 'w')\n\t\tf.write(self.toStr())\n\t\tf.flush()\n\t\tf.close()", "response": "save the CSV to a file"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef toStr(self) :\n\t\ts = [self.strLegend]\n\t\tfor l in self.lines :\n\t\t\ts.append(l.toStr())\n\t\treturn self.lineSeparator.join(s)", "response": "returns a string version of the CSV"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nreturns the number of elements satisfying the query", "response": "def count(self, objectType, *args, **coolArgs) :\n\t\t\"\"\"Returns the number of elements satisfying the query\"\"\"\n\t\treturn self._makeLoadQuery(objectType, *args, **coolArgs).count()"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef iterGet(self, objectType, *args, **coolArgs) :\n\n\t\tfor e in self._makeLoadQuery(objectType, *args, **coolArgs).iterRun() :\n\t\t\tif issubclass(objectType, pyGenoRabaObjectWrapper) :\n\t\t\t\tyield objectType(wrapped_object_and_bag = (e, self.bagKey))\n\t\t\telse :\n\t\t\t\tyield e", "response": "Same as get but retuns the elements one by one."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef deleteSNPs(setName) :\n\tcon = conf.db\n\ttry :\n\t\tSMaster = SNPMaster(setName = setName)\n\t\tcon.beginTransaction()\n\t\tSNPType = SMaster.SNPType\n\t\tcon.delete(SNPType, 'setName = ?', (setName,))\n\t\tSMaster.delete()\n\t\tcon.endTransaction()\n\texcept KeyError :\n\t\traise KeyError(\"Can't delete the setName %s because i can't find it in SNPMaster, maybe there's not set by that name\" % setName)\n\t\t#~ printf(\"can't delete the setName %s because i can't find it in SNPMaster, maybe there's no set by that name\" % setName)\n\t\treturn False\n\treturn True", "response": "deletes a set of polymorphisms"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _importSNPs_AgnosticSNP(setName, species, genomeSource, snpsFile) :\n\t\"This function will also create an index on start->chromosomeNumber->setName. Warning : pyGeno wil interpret all positions as 0 based\"\n\tprintf('importing SNP set %s for species %s...' % (setName, species))\n\n\tsnpData = CSVFile()\n\tsnpData.parse(snpsFile, separator = \"\\t\")\n\n\tAgnosticSNP.dropIndex(('start', 'chromosomeNumber', 'setName'))\n\tconf.db.beginTransaction()\n\t\n\tpBar = ProgressBar(len(snpData))\n\tpLabel = ''\n\tcurrChrNumber = None\n\tfor snpEntry in snpData :\n\t\ttmpChr = snpEntry['chromosomeNumber']\n\t\tif tmpChr != currChrNumber :\n\t\t\tcurrChrNumber = tmpChr\n\t\t\tpLabel = 'Chr %s...' % currChrNumber\n\n\t\tsnp = AgnosticSNP()\n\t\tsnp.species = species\n\t\tsnp.setName = setName\n\t\tfor f in snp.getFields() :\n\t\t\ttry :\n\t\t\t\tsetattr(snp, f, snpEntry[f])\n\t\t\texcept KeyError :\n\t\t\t\tif f != 'species' and f != 'setName' :\n\t\t\t\t\tprintf(\"Warning filetype as no key %s\", f)\n\t\tsnp.quality = float(snp.quality)\n\t\tsnp.start = int(snp.start)\n\t\tsnp.end = int(snp.end)\n\t\tsnp.save()\n\t\tpBar.update(label = pLabel)\n\n\tpBar.close()\n\t\n\tsnpMaster = SNPMaster()\n\tsnpMaster.set(setName = setName, SNPType = 'AgnosticSNP', species = species)\n\tsnpMaster.save()\n\n\tprintf('saving...')\n\tconf.db.endTransaction()\n\tprintf('creating indexes...')\n\tAgnosticSNP.ensureGlobalIndex(('start', 'chromosomeNumber', 'setName'))\n\tprintf('importation of SNP set %s for species %s done.' %(setName, species))\n\t\n\treturn True", "response": "This function will also create an index on start -> chromosomeNumber -> setName. Warning : pyGeno wil interpret all positions as 0 based"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _importSNPs_CasavaSNP(setName, species, genomeSource, snpsFile) :\n\t\"This function will also create an index on start->chromosomeNumber->setName. Warning : pyGeno positions are 0 based\"\n\tprintf('importing SNP set %s for species %s...' % (setName, species))\n\n\tsnpData = SNPsTxtFile(snpsFile)\n\t\n\tCasavaSNP.dropIndex(('start', 'chromosomeNumber', 'setName'))\n\tconf.db.beginTransaction()\n\t\n\tpBar = ProgressBar(len(snpData))\n\tpLabel = ''\n\tcurrChrNumber = None\n\tfor snpEntry in snpData :\n\t\ttmpChr = snpEntry['chromosomeNumber']\n\t\tif tmpChr != currChrNumber :\n\t\t\tcurrChrNumber = tmpChr\n\t\t\tpLabel = 'Chr %s...' % currChrNumber\n\n\t\tsnp = CasavaSNP()\n\t\tsnp.species = species\n\t\tsnp.setName = setName\n\t\t\n\t\tfor f in snp.getFields() :\n\t\t\ttry :\n\t\t\t\tsetattr(snp, f, snpEntry[f])\n\t\t\texcept KeyError :\n\t\t\t\tif f != 'species' and f != 'setName' :\n\t\t\t\t\tprintf(\"Warning filetype as no key %s\", f)\n\t\tsnp.start -= 1\n\t\tsnp.end -= 1\n\t\tsnp.save()\n\t\tpBar.update(label = pLabel)\n\n\tpBar.close()\n\t\n\tsnpMaster = SNPMaster()\n\tsnpMaster.set(setName = setName, SNPType = 'CasavaSNP', species = species)\n\tsnpMaster.save()\n\n\tprintf('saving...')\n\tconf.db.endTransaction()\n\tprintf('creating indexes...')\n\tCasavaSNP.ensureGlobalIndex(('start', 'chromosomeNumber', 'setName'))\n\tprintf('importation of SNP set %s for species %s done.' %(setName, species))\n\t\n\treturn True", "response": "This function will also create an index on start -> chromosomeNumber -> setName. Warning : pyGeno positions are 0 based"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef _importSNPs_dbSNPSNP(setName, species, genomeSource, snpsFile) :\n\t\"This function will also create an index on start->chromosomeNumber->setName. Warning : pyGeno positions are 0 based\"\n\tsnpData = VCFFile(snpsFile, gziped = True, stream = True)\n\tdbSNPSNP.dropIndex(('start', 'chromosomeNumber', 'setName'))\n\tconf.db.beginTransaction()\n\tpBar = ProgressBar()\n\tpLabel = ''\n\tfor snpEntry in snpData :\n\t\tpBar.update(label = 'Chr %s, %s...' % (snpEntry['#CHROM'], snpEntry['ID']))\n\t\t\n\t\tsnp = dbSNPSNP()\n\t\tfor f in snp.getFields() :\n\t\t\ttry :\n\t\t\t\tsetattr(snp, f, snpEntry[f])\n\t\t\texcept KeyError :\n\t\t\t\tpass\n\t\tsnp.chromosomeNumber = snpEntry['#CHROM']\n\t\tsnp.species = species\n\t\tsnp.setName = setName\n\t\tsnp.start = snpEntry['POS']-1\n\t\tsnp.alt = snpEntry['ALT']\n\t\tsnp.ref = snpEntry['REF']\n\t\tsnp.end = snp.start+len(snp.alt)\n\t\tsnp.save()\n\t\n\tpBar.close()\n\t\n\tsnpMaster = SNPMaster()\n\tsnpMaster.set(setName = setName, SNPType = 'dbSNPSNP', species = species)\n\tsnpMaster.save()\n\t\n\tprintf('saving...')\n\tconf.db.endTransaction()\n\tprintf('creating indexes...')\n\tdbSNPSNP.ensureGlobalIndex(('start', 'chromosomeNumber', 'setName'))\n\tprintf('importation of SNP set %s for species %s done.' %(setName, species))\n\n\treturn True", "response": "This function will also create an index on start -> chromosomeNumber -> setName. Warning : pyGeno positions are 0 based"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nreturn the names of all imported snp sets", "response": "def getSNPSetsList() :\n\t\"\"\"Return the names of all imported snp sets\"\"\"\n\timport rabaDB.filters as rfilt\n\tf = rfilt.RabaQuery(SNPMaster)\n\tnames = []\n\tfor g in f.iterRun() :\n\t\tnames.append(g.setName)\n\treturn names"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nprint a list of strings", "response": "def printf(*s) :\n\t'print + sys.stdout.flush()'\n\tfor e in s[:-1] :\n\t\tprint e,\n\tprint s[-1]\n\n\tsys.stdout.flush()"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nopen a file and parses it", "response": "def parseFile(self, fil) :\n\t\t\"\"\"Opens a file and parses it\"\"\"\n\t\tf = open(fil)\n\t\tself.parseStr(f.read())\n\t\tf.close()"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nappend a new entry to the file", "response": "def add(self, header, data) :\n\t\t\"\"\"appends a new entry to the file\"\"\"\n\t\tif header[0] != '>' :\n\t\t\tself.data.append(('>'+header, data))\n\t\telse :\n\t\t\tself.data.append((header, data))"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nreturns a list of all occurances of needle in haystack", "response": "def findAll(haystack, needle) :\n\t\"\"\"returns a list of all occurances of needle in haystack\"\"\"\n\t\n\th = haystack\n\tres = []\n\tf = haystack.find(needle)\n\toffset = 0\n\twhile (f >= 0) :\n\t\t#print h, needle, f, offset\n\t\tres.append(f+offset)\n\t\toffset += f+len(needle)\n\t\th = h[f+len(needle):]\n\t\tf = h.find(needle)\n\n\treturn res"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nreturns a list of complementary sequence without inversing it", "response": "def complementTab(seq=[]):\n \"\"\"returns a list of complementary sequence without inversing it\"\"\"\n complement = {'A': 'T', 'C': 'G', 'G': 'C', 'T': 'A', 'R': 'Y', 'Y': 'R', 'M': 'K', 'K': 'M',\n 'W': 'W', 'S': 'S', 'B': 'V', 'D': 'H', 'H': 'D', 'V': 'B', 'N': 'N', 'a': 't',\n 'c': 'g', 'g': 'c', 't': 'a', 'r': 'y', 'y': 'r', 'm': 'k', 'k': 'm', 'w': 'w',\n 's': 's', 'b': 'v', 'd': 'h', 'h': 'd', 'v': 'b', 'n': 'n'}\n seq_tmp = []\n for bps in seq:\n if len(bps) == 0:\n \t#Need manage '' for deletion\n seq_tmp.append('') \n elif len(bps) == 1:\n seq_tmp.append(complement[bps])\n else:\n \t#Need manage 'ACT' for insertion\n \t#The insertion need to be reverse complement (like seq)\n seq_tmp.append(reverseComplement(bps))\n \n #Doesn't work in the second for when bps==''\n #seq = [complement[bp] if bp != '' else '' for bps in seq for bp in bps]\n\n return seq_tmp"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef translateDNA_6Frames(sequence) :\n\ttrans = (\n\t\t\t\ttranslateDNA(sequence, 'f1'),\n\t\t\t\ttranslateDNA(sequence, 'f2'),\n\t\t\t\ttranslateDNA(sequence, 'f3'),\n\n\t\t\t\ttranslateDNA(sequence, 'r1'),\n\t\t\t\ttranslateDNA(sequence, 'r2'),\n\t\t\t\ttranslateDNA(sequence, 'r3'),\n\t\t\t)\n\n\treturn trans", "response": "returns 6 translation of sequence. One for each reading frame"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\ntranslating a DNA code to a protein code.", "response": "def translateDNA(sequence, frame = 'f1', translTable_id='default') :\n\t\"\"\"Translates DNA code, frame : fwd1, fwd2, fwd3, rev1, rev2, rev3\"\"\"\n\n\tprotein = \"\"\n\n\tif frame == 'f1' :\n\t\tdna = sequence\n\telif frame == 'f2':\n\t\tdna = sequence[1:]\n\telif frame == 'f3' :\n\t\tdna = sequence[2:]\n\telif frame == 'r1' :\n\t\tdna = reverseComplement(sequence)\n\telif frame == 'r2' :\n\t\tdna = reverseComplement(sequence)\n\t\tdna = dna[1:]\n\telif frame == 'r3' :\n\t\tdna = reverseComplement(sequence)\n\t\tdna = dna[2:]\n\telse :\n\t\traise ValueError('unknown reading frame: %s, should be one of the following: fwd1, fwd2, fwd3, rev1, rev2, rev3' % frame)\n\n\tfor i in range(0, len(dna), 3) :\n\t\tcodon = dna[i:i+3]\n\n\t\t# Check if variant messed with selenocysteine codon\n\t\tif '!' in codon and codon != '!GA':\n\t\t\tcodon = codon.replace('!', 'T')\n\n\t\tif (len(codon) == 3) :\n\t\t\ttry :\n\t\t\t\t# MC\n\t\t\t\tprotein += translTable[translTable_id][codon]\n\t\t\texcept KeyError :\n\t\t\t\tcombinaisons = polymorphicCodonCombinaisons(list(codon))\n\t\t\t\ttranslations = set()\n\t\t\t\tfor ci in range(len(combinaisons)):\n\t\t\t\t\ttranslations.add(translTable[translTable_id][combinaisons[ci]])\n\t\t\t\tprotein += '/'.join(translations)\n\n\treturn protein"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\ntake a dna sequence with polymorphismes and returns all the possible sequences that it can yield", "response": "def getSequenceCombinaisons(polymorphipolymorphicDnaSeqSeq, pos = 0) :\n\t\"\"\"Takes a dna sequence with polymorphismes and returns all the possible sequences that it can yield\"\"\"\n\n\tif type(polymorphipolymorphicDnaSeqSeq) is not types.ListType :\n\t\tseq = list(polymorphipolymorphicDnaSeqSeq)\n\telse :\n\t\tseq = polymorphipolymorphicDnaSeqSeq\n\n\tif pos >= len(seq) :\n\t\treturn [''.join(seq)]\n\n\tvariants = []\n\tif seq[pos] in polymorphicNucleotides :\n\t\tchars = decodePolymorphicNucleotide(seq[pos])\n\telse :\n\t\tchars = seq[pos]#.split('/')\n\n\tfor c in chars :\n\t\trseq = copy.copy(seq)\n\t\trseq[pos] = c\n\t\tvariants.extend(getSequenceCombinaisons(rseq, pos + 1))\n\n\treturn variants"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nreturns a single character encoding all nucletides of a given polySeq in a single character.", "response": "def encodePolymorphicNucleotide(polySeq) :\n\t\"\"\"returns a single character encoding all nucletides of polySeq \n\tin a single character. PolySeq must have one of the following forms: \n\t['A', 'T', 'G'], 'ATG', 'A/T/G'\"\"\"\n\t\n\tif type(polySeq) is types.StringType :\n\t\tif polySeq.find(\"/\") < 0 :\n\t\t\tsseq = list(polySeq)\n\t\telse :\n\t\t\tsseq = polySeq.split('/')\n\t\t\t\n\telse :\n\t\tsseq = polySeq\n\t\n\tseq = []\n\tfor n in sseq :\n\t\ttry :\n\t\t\tfor n2 in polymorphicNucleotides[n] :\n\t\t\t\tseq.append(n2)\n\t\texcept KeyError :\n\t\t\tseq.append(n)\n\t\n\tseq = set(seq)\n\t\n\tif len(seq) == 4:\n\t\treturn 'N'\n\telif len(seq) == 3 :\n\t\tif 'T' not in seq :\n\t\t\treturn 'V'\n\t\telif 'G' not in seq :\n\t\t\treturn 'H'\n\t\telif 'C' not in seq :\n\t\t\treturn 'D'\n\t\telif 'A' not in seq :\n\t\t\treturn 'B'\n\telif len(seq) == 2 :\n\t\tif 'A' in seq and 'G' in seq :\n\t\t\treturn 'R'\n\t\telif 'C' in seq and 'T' in seq :\n\t\t\treturn 'Y'\n\t\telif 'A' in seq and 'C' in seq :\n\t\t\treturn 'M'\n\t\telif 'T' in seq and 'G' in seq :\n\t\t\treturn 'K'\n\t\telif 'A' in seq and 'T' in seq :\n\t\t\treturn 'W'\n\t\telif 'C' in seq and 'G' in seq :\n\t\t\treturn 'S'\n\telif polySeq[0] in nucleotides :\n\t\treturn polySeq[0]\n\telse :\n\t\traise UnknownNucleotide(polySeq)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef decodePolymorphicNucleotide(nuc) :\n\tif nuc in polymorphicNucleotides :\n\t\treturn polymorphicNucleotides[nuc]\n\n\tif nuc in nucleotides :\n\t\treturn nuc\n\n\traise ValueError('nuc: %s, is not a valid nucleotide' % nuc)", "response": "decode a polymorphic nucleotide into a sequence of nucleotides"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef getNucleotideCodon(sequence, x1) :\n\n\tif x1 < 0 or x1 >= len(sequence) :\n\t\treturn None\n\n\tp = x1%3\n\tif p == 0 :\n\t\treturn (sequence[x1: x1+3], 0)\n\telif p ==1 :\n\t\treturn (sequence[x1-1: x1+2], 1)\n\telif p == 2 :\n\t\treturn (sequence[x1-2: x1+1], 2)", "response": "Returns the entire codon of the nucleotide at pos x1 in sequence and \n\t and the position of that nocleotide in the codon in a tuple"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef showDifferences(seq1, seq2) :\n\tret = []\n\tfor i in range(max(len(seq1), len(seq2))) :\n\n\t\tif i >= len(seq1) :\n\t\t\tc1 = '#'\n\t\telse :\n\t\t\tc1 = seq1[i]\n\t\tif i >= len(seq2) :\n\t\t\tc2 = '#'\n\t\telse :\n\t\t\tc2 = seq2[i]\n\n\t\tif c1 != c2 :\n\t\t\tret.append('%s|%s' % (c1, c2))\n\t\telse :\n\t\t\tret.append('-')\n\n\treturn ''.join(ret)", "response": "Returns a string that shows the differences between seq1 and seq2."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef highlightSubsequence(sequence, x1, x2, start=' [', stop = '] ') :\n\n\tseq = list(sequence)\n\tprint x1, x2-1, len(seq)\n\tseq[x1] = start + seq[x1]\n\tseq[x2-1] = seq[x2-1] + stop\n\treturn ''.join(seq)", "response": "returns a sequence where the subsequence in [ x1 x2 is placed \n\tin bewteen start and stop"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef deleteGenome(species, name) :\n\n printf('deleting genome (%s, %s)...' % (species, name))\n\n conf.db.beginTransaction()\n objs = []\n allGood = True\n try :\n genome = Genome_Raba(name = name, species = species.lower())\n objs.append(genome)\n pBar = ProgressBar(label = 'preparing')\n for typ in (Chromosome_Raba, Gene_Raba, Transcript_Raba, Exon_Raba, Protein_Raba) :\n pBar.update()\n f = RabaQuery(typ, namespace = genome._raba_namespace)\n f.addFilter({'genome' : genome})\n for e in f.iterRun() :\n objs.append(e)\n pBar.close()\n \n pBar = ProgressBar(nbEpochs = len(objs), label = 'deleting objects')\n for e in objs :\n pBar.update()\n e.delete()\n pBar.close()\n \n except KeyError as e :\n #~ printf(\"\\tWARNING, couldn't remove genome form db, maybe it's not there: \", e)\n raise KeyError(\"\\tWARNING, couldn't remove genome form db, maybe it's not there: \", e)\n allGood = False\n printf('\\tdeleting folder')\n try :\n shutil.rmtree(conf.getGenomeSequencePath(species, name))\n except OSError as e:\n #~ printf('\\tWARNING, Unable to delete folder: ', e)\n OSError('\\tWARNING, Unable to delete folder: ', e)\n allGood = False\n \n conf.db.endTransaction()\n return allGood", "response": "Removes a genome from the database"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nimports a pyGeno genome package.", "response": "def importGenome(packageFile, batchSize = 50, verbose = 0) :\n \"\"\"Import a pyGeno genome package. A genome packages is folder or a tar.gz ball that contains at it's root:\n\n * gziped fasta files for all chromosomes, or URLs from where them must be downloaded\n \n * gziped GTF gene_set file from Ensembl, or an URL from where it must be downloaded\n \n * a manifest.ini file such as::\n \n [package_infos]\n description = Test package. This package installs only chromosome Y of mus musculus\n maintainer = Tariq Daouda\n maintainer_contact = tariq.daouda [at] umontreal\n version = GRCm38.73\n\n [genome]\n species = Mus_musculus\n name = GRCm38_test\n source = http://useast.ensembl.org/info/data/ftp/index.html\n\n [chromosome_files]\n Y = Mus_musculus.GRCm38.73.dna.chromosome.Y.fa.gz / or an url such as ftp://... or http://\n\n [gene_set]\n gtf = Mus_musculus.GRCm38.73_Y-only.gtf.gz / or an url such as ftp://... or http://\n\n All files except the manifest can be downloaded from: http://useast.ensembl.org/info/data/ftp/index.html\n \n A rollback is performed if an exception is caught during importation\n \n batchSize sets the number of genes to parse before performing a database save. PCs with little ram like\n small values, while those endowed with more memory may perform faster with higher ones.\n \n Verbose must be an int [0, 4] for various levels of verbosity\n \"\"\"\n\n def reformatItems(items) :\n s = str(items)\n s = s.replace('[', '').replace(']', '').replace(\"',\", ': ').replace('), ', '\\n').replace(\"'\", '').replace('(', '').replace(')', '')\n return s\n\n printf('Importing genome package: %s... (This may take a while)' % packageFile)\n\n isDir = False\n if not os.path.isdir(packageFile) :\n packageDir = _decompressPackage(packageFile)\n else :\n isDir = True\n packageDir = packageFile\n\n parser = SafeConfigParser()\n parser.read(os.path.normpath(packageDir+'/manifest.ini'))\n packageInfos = parser.items('package_infos')\n\n genomeName = parser.get('genome', 'name')\n species = parser.get('genome', 'species')\n genomeSource = parser.get('genome', 'source')\n \n seqTargetDir = conf.getGenomeSequencePath(species.lower(), genomeName)\n if os.path.isdir(seqTargetDir) :\n raise KeyError(\"The directory %s already exists, Please call deleteGenome() first if you want to reinstall\" % seqTargetDir)\n \n gtfFile = _getFile(parser.get('gene_set', 'gtf'), packageDir)\n \n chromosomesFiles = {}\n chromosomeSet = set()\n for key, fil in parser.items('chromosome_files') :\n chromosomesFiles[key] = _getFile(fil, packageDir)\n chromosomeSet.add(key)\n\n try :\n genome = Genome(name = genomeName, species = species)\n except KeyError:\n pass\n else :\n raise KeyError(\"There seems to be already a genome (%s, %s), please call deleteGenome() first if you want to reinstall it\" % (genomeName, species))\n\n genome = Genome_Raba()\n genome.set(name = genomeName, species = species, source = genomeSource, packageInfos = packageInfos)\n\n printf(\"Importing:\\n\\t%s\\nGenome:\\n\\t%s\\n...\" % (reformatItems(packageInfos).replace('\\n', '\\n\\t'), reformatItems(parser.items('genome')).replace('\\n', '\\n\\t')))\n\n chros = _importGenomeObjects(gtfFile, chromosomeSet, genome, batchSize, verbose)\n os.makedirs(seqTargetDir)\n startChro = 0\n pBar = ProgressBar(nbEpochs = len(chros))\n for chro in chros :\n pBar.update(label = \"Importing DNA, chro %s\" % chro.number)\n length = _importSequence(chro, chromosomesFiles[chro.number.lower()], seqTargetDir)\n chro.start = startChro\n chro.end = startChro+length\n startChro = chro.end\n chro.save()\n pBar.close()\n \n if not isDir :\n shutil.rmtree(packageDir)\n \n #~ objgraph.show_most_common_types(limit=20)\n return True"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nimports Chromosome objects from a GTF file.", "response": "def _importGenomeObjects(gtfFilePath, chroSet, genome, batchSize, verbose = 0) :\n \"\"\"verbose must be an int [0, 4] for various levels of verbosity\"\"\"\n\n class Store(object) :\n \n def __init__(self, conf) :\n self.conf = conf\n self.chromosomes = {}\n \n self.genes = {}\n self.transcripts = {}\n self.proteins = {}\n self.exons = {}\n\n def batch_save(self) :\n self.conf.db.beginTransaction()\n \n for c in self.genes.itervalues() :\n c.save()\n conf.removeFromDBRegistery(c)\n \n for c in self.transcripts.itervalues() :\n c.save()\n conf.removeFromDBRegistery(c.exons)\n conf.removeFromDBRegistery(c)\n \n for c in self.proteins.itervalues() :\n c.save()\n conf.removeFromDBRegistery(c)\n \n self.conf.db.endTransaction()\n \n del(self.genes)\n del(self.transcripts)\n del(self.proteins)\n del(self.exons)\n \n self.genes = {}\n self.transcripts = {}\n self.proteins = {}\n self.exons = {}\n\n gc.collect()\n\n def save_chros(self) :\n pBar = ProgressBar(nbEpochs = len(self.chromosomes))\n for c in self.chromosomes.itervalues() :\n pBar.update(label = 'Chr %s' % c.number)\n c.save()\n pBar.close()\n \n printf('Importing gene set infos from %s...' % gtfFilePath)\n \n printf('Backuping indexes...')\n indexes = conf.db.getIndexes()\n printf(\"Droping all your indexes, (don't worry i'll restore them later)...\")\n Genome_Raba.flushIndexes()\n Chromosome_Raba.flushIndexes()\n Gene_Raba.flushIndexes()\n Transcript_Raba.flushIndexes()\n Protein_Raba.flushIndexes()\n Exon_Raba.flushIndexes()\n \n printf(\"Parsing gene set...\")\n gtf = GTFFile(gtfFilePath, gziped = True)\n printf('Done. Importation begins!')\n \n store = Store(conf)\n chroNumber = None\n pBar = ProgressBar(nbEpochs = len(gtf))\n for line in gtf :\n chroN = line['seqname']\n pBar.update(label = \"Chr %s\" % chroN)\n \n if (chroN.upper() in chroSet or chroN.lower() in chroSet):\n strand = line['strand']\n gene_biotype = line['gene_biotype']\n regionType = line['feature']\n frame = line['frame']\n\n start = int(line['start']) - 1\n end = int(line['end'])\n if start > end :\n start, end = end, start\n\n chroNumber = chroN.upper()\n if chroNumber not in store.chromosomes :\n store.chromosomes[chroNumber] = Chromosome_Raba()\n store.chromosomes[chroNumber].set(genome = genome, number = chroNumber)\n \n try :\n geneId = line['gene_id']\n geneName = line['gene_name']\n except KeyError :\n geneId = None\n geneName = None\n if verbose :\n printf('Warning: no gene_id/name found in line %s' % gtf[i])\n \n if geneId is not None :\n if geneId not in store.genes :\n if len(store.genes) > batchSize :\n store.batch_save()\n \n if verbose > 0 :\n printf('\\tGene %s, %s...' % (geneId, geneName))\n store.genes[geneId] = Gene_Raba()\n store.genes[geneId].set(genome = genome, id = geneId, chromosome = store.chromosomes[chroNumber], name = geneName, strand = strand, biotype = gene_biotype)\n if start < store.genes[geneId].start or store.genes[geneId].start is None :\n store.genes[geneId].start = start\n if end > store.genes[geneId].end or store.genes[geneId].end is None :\n store.genes[geneId].end = end\n try :\n transId = line['transcript_id']\n transName = line['transcript_name']\n try :\n transcript_biotype = line['transcript_biotype']\n except KeyError :\n transcript_biotype = None\n except KeyError :\n transId = None\n transName = None\n if verbose > 2 :\n printf('\\t\\tWarning: no transcript_id, name found in line %s' % gtf[i])\n \n if transId is not None :\n if transId not in store.transcripts :\n if verbose > 1 :\n printf('\\t\\tTranscript %s, %s...' % (transId, transName))\n store.transcripts[transId] = Transcript_Raba()\n store.transcripts[transId].set(genome = genome, id = transId, chromosome = store.chromosomes[chroNumber], gene = store.genes.get(geneId, None), name = transName, biotype=transcript_biotype)\n if start < store.transcripts[transId].start or store.transcripts[transId].start is None:\n store.transcripts[transId].start = start\n if end > store.transcripts[transId].end or store.transcripts[transId].end is None:\n store.transcripts[transId].end = end\n \n try :\n protId = line['protein_id']\n except KeyError :\n protId = None\n if verbose > 2 :\n printf('Warning: no protein_id found in line %s' % gtf[i])\n\n # Store selenocysteine positions in transcript\n if regionType == 'Selenocysteine':\n store.transcripts[transId].selenocysteine.append(start)\n \n if protId is not None and protId not in store.proteins :\n if verbose > 1 :\n printf('\\t\\tProtein %s...' % (protId))\n store.proteins[protId] = Protein_Raba()\n store.proteins[protId].set(genome = genome, id = protId, chromosome = store.chromosomes[chroNumber], gene = store.genes.get(geneId, None), transcript = store.transcripts.get(transId, None), name = transName)\n store.transcripts[transId].protein = store.proteins[protId]\n\n try :\n exonNumber = int(line['exon_number']) - 1\n exonKey = (transId, exonNumber)\n except KeyError :\n exonNumber = None\n exonKey = None\n if verbose > 2 :\n printf('Warning: no exon number or id found in line %s' % gtf[i])\n \n if exonKey is not None :\n if verbose > 3 :\n printf('\\t\\t\\texon %s...' % (exonId))\n \n if exonKey not in store.exons and regionType == 'exon' :\n store.exons[exonKey] = Exon_Raba()\n store.exons[exonKey].set(genome = genome, chromosome = store.chromosomes[chroNumber], gene = store.genes.get(geneId, None), transcript = store.transcripts.get(transId, None), protein = store.proteins.get(protId, None), strand = strand, number = exonNumber, start = start, end = end)\n store.transcripts[transId].exons.append(store.exons[exonKey])\n \n try :\n store.exons[exonKey].id = line['exon_id']\n except KeyError :\n pass\n \n if regionType == 'exon' :\n if start < store.exons[exonKey].start or store.exons[exonKey].start is None:\n store.exons[exonKey].start = start\n if end > store.transcripts[transId].end or store.exons[exonKey].end is None:\n store.exons[exonKey].end = end\n elif regionType == 'CDS' :\n store.exons[exonKey].CDS_start = start\n store.exons[exonKey].CDS_end = end\n store.exons[exonKey].frame = frame\n elif regionType == 'stop_codon' :\n if strand == '+' :\n if store.exons[exonKey].CDS_end != None :\n store.exons[exonKey].CDS_end += 3\n if store.exons[exonKey].end < store.exons[exonKey].CDS_end :\n store.exons[exonKey].end = store.exons[exonKey].CDS_end\n if store.transcripts[transId].end < store.exons[exonKey].CDS_end :\n store.transcripts[transId].end = store.exons[exonKey].CDS_end\n if store.genes[geneId].end < store.exons[exonKey].CDS_end :\n store.genes[geneId].end = store.exons[exonKey].CDS_end\n if strand == '-' :\n if store.exons[exonKey].CDS_start != None :\n store.exons[exonKey].CDS_start -= 3\n if store.exons[exonKey].start > store.exons[exonKey].CDS_start :\n store.exons[exonKey].start = store.exons[exonKey].CDS_start\n if store.transcripts[transId].start > store.exons[exonKey].CDS_start :\n store.transcripts[transId].start = store.exons[exonKey].CDS_start\n if store.genes[geneId].start > store.exons[exonKey].CDS_start :\n store.genes[geneId].start = store.exons[exonKey].CDS_start\n pBar.close()\n \n store.batch_save()\n \n conf.db.beginTransaction()\n printf('almost done saving chromosomes...')\n store.save_chros()\n \n printf('saving genome object...')\n genome.save()\n conf.db.endTransaction()\n \n conf.db.beginTransaction()\n printf('restoring core indexes...')\n # Genome.ensureGlobalIndex(('name', 'species'))\n # Chromosome.ensureGlobalIndex('genome')\n # Gene.ensureGlobalIndex('genome')\n # Transcript.ensureGlobalIndex('genome')\n # Protein.ensureGlobalIndex('genome')\n # Exon.ensureGlobalIndex('genome')\n Transcript.ensureGlobalIndex('exons')\n \n printf('commiting changes...')\n conf.db.endTransaction()\n \n conf.db.beginTransaction()\n printf('restoring user indexes')\n pBar = ProgressBar(label = \"restoring\", nbEpochs = len(indexes))\n for idx in indexes :\n pBar.update()\n conf.db.execute(idx[-1].replace('CREATE INDEX', 'CREATE INDEX IF NOT EXISTS'))\n pBar.close()\n \n printf('commiting changes...')\n conf.db.endTransaction()\n \n return store.chromosomes.values()"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nserializes fastas into. dat files", "response": "def _importSequence(chromosome, fastaFile, targetDir) :\n \"Serializes fastas into .dat files\"\n\n f = gzip.open(fastaFile)\n header = f.readline()\n strRes = f.read().upper().replace('\\n', '').replace('\\r', '')\n f.close()\n\n fn = '%s/chromosome%s.dat' % (targetDir, chromosome.number)\n f = open(fn, 'w')\n f.write(strRes)\n f.close()\n chromosome.dataFile = fn\n chromosome.header = header\n return len(strRes)"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\ncreates a default configuration file", "response": "def createDefaultConfigFile() :\n\t\"\"\"Creates a default configuration file\"\"\"\n\ts = \"[pyGeno_config]\\nsettings_dir=%s\\nremote_location=%s\" % (pyGeno_SETTINGS_DIR, pyGeno_REMOTE_LOCATION)\n\tf = open('%s/config.ini' % pyGeno_SETTINGS_DIR, 'w')\n\tf.write(s)\n\tf.close()"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef getSettingsPath() :\n\tparser = SafeConfigParser()\n\ttry :\n\t\tparser.read(os.path.normpath(pyGeno_SETTINGS_DIR+'/config.ini'))\n\t\treturn parser.get('pyGeno_config', 'settings_dir')\n\texcept :\n\t\tcreateDefaultConfigFile()\n\t\treturn getSettingsPath()", "response": "Returns the path where the settings are stored"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef pyGeno_init() :\n\t\n\tglobal db, dbConf\n\t\n\tglobal pyGeno_SETTINGS_PATH\n\tglobal pyGeno_RABA_DBFILE\n\tglobal pyGeno_DATA_PATH\n\t\n\tif not checkPythonVersion() :\n\t\traise PythonVersionError(\"==> FATAL: pyGeno only works with python 2.7 and above, please upgrade your python version\")\n\n\tif not os.path.exists(pyGeno_SETTINGS_DIR) :\n\t\tos.makedirs(pyGeno_SETTINGS_DIR)\n\t\n\tpyGeno_SETTINGS_PATH = getSettingsPath()\n\tpyGeno_RABA_DBFILE = os.path.normpath( os.path.join(pyGeno_SETTINGS_PATH, \"pyGenoRaba.db\") )\n\tpyGeno_DATA_PATH = os.path.normpath( os.path.join(pyGeno_SETTINGS_PATH, \"data\") )\n\t\n\tif not os.path.exists(pyGeno_SETTINGS_PATH) :\n\t\tos.makedirs(pyGeno_SETTINGS_PATH)\n\n\tif not os.path.exists(pyGeno_DATA_PATH) :\n\t\tos.makedirs(pyGeno_DATA_PATH)\n\n\t#launching the db\n\trabaDB.rabaSetup.RabaConfiguration(pyGeno_RABA_NAMESPACE, pyGeno_RABA_DBFILE)\n\tdb = rabaDB.rabaSetup.RabaConnection(pyGeno_RABA_NAMESPACE)\n\tdbConf = rabaDB.rabaSetup.RabaConfiguration(pyGeno_RABA_NAMESPACE)", "response": "This function is automatically called at launch"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef pluck(self) :\n\t\te = copy.copy(self)\n\t\te.transcript = str(self.transcript)\n\t\treturn e", "response": "Returns a copy of the exon and sets the value of self. transcript into str."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nreturns the previous exon of the transcript or None if there is no exon.", "response": "def previousExon(self) :\n\t\t\"\"\"Returns the previous exon of the transcript, or None if there is none\"\"\"\n\t\t\n\t\tif self.number == 0 :\n\t\t\treturn None\n\t\t\n\t\ttry :\n\t\t\treturn self.transcript.exons[self.number-1]\n\t\texcept IndexError :\n\t\t\treturn None"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef get(self, li) :\n\t\ti = li*4\n\t\tself.__splitEntry(i)\n\t\treturn self.data[i]", "response": "returns the ith entry"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef newEntry(self, ident = \"\", seq = \"\", plus = \"\", qual = \"\") :\n\t\te = FastqEntry()\n\t\tself.data.append(e)\n\t\treturn e", "response": "Returns a new entry in the CSV"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nlists all the datawraps availabe from a remote location.", "response": "def listRemoteDatawraps(location = conf.pyGeno_REMOTE_LOCATION) :\n\t\"\"\"Lists all the datawraps availabe from a remote a remote location.\"\"\"\n\tloc = location + \"/datawraps.json\"\n\tresponse = urllib2.urlopen(loc)\n\tjs = json.loads(response.read())\n\t\n\treturn js"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef printRemoteDatawraps(location = conf.pyGeno_REMOTE_LOCATION) :\n\t\n\tl = listRemoteDatawraps(location)\n\tprintf(\"Available datawraps for bootstraping\\n\")\n\tprint json.dumps(l[\"Ordered\"], sort_keys=True, indent=4, separators=(',', ': '))", "response": "Print all available datawraps from a remote location."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nimport a genome from the remote genomes.", "response": "def importRemoteGenome(name, batchSize = 100) :\n\t\"\"\"Import a genome available from http://pygeno.iric.ca (might work).\"\"\"\n\ttry :\n\t\tdw = listRemoteDatawraps()[\"Flat\"][\"Reference genomes\"][name]\n\texcept AttributeError :\n\t\traise AttributeError(\"There's no remote genome datawrap by the name of: '%s'\" % name)\n\n\tfinalFile = _DW(name, dw[\"url\"])\n\tPG.importGenome(finalFile, batchSize)"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nimport a remote SNP set from the IRAF CA.", "response": "def importRemoteSNPs(name) :\n\t\"\"\"Import a SNP set available from http://pygeno.iric.ca (might work).\"\"\"\n\ttry :\n\t\tdw = listRemoteDatawraps()[\"Flat\"][\"SNPs\"]\n\texcept AttributeError :\n\t\traise AttributeError(\"There's no remote genome datawrap by the name of: '%s'\" % name)\n\n\tfinalFile = _DW(name, dw[\"url\"])\n\tPS.importSNPs(finalFile)"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef listDatawraps() :\n\tl = {\"Genomes\" : [], \"SNPs\" : []}\n\tfor f in os.listdir(os.path.join(this_dir, \"bootstrap_data/genomes\")) :\n\t\tif f.find(\".tar.gz\") > -1 :\n\t\t\tl[\"Genomes\"].append(f)\n\t\n\tfor f in os.listdir(os.path.join(this_dir, \"bootstrap_data/SNPs\")) :\n\t\tif f.find(\".tar.gz\") > -1 :\n\t\t\tl[\"SNPs\"].append(f)\n\n\treturn l", "response": "Lists all the datawraps pyGeno comes with"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nprints all available datawraps for bootstrapping", "response": "def printDatawraps() :\n\t\"\"\"print all available datawraps for bootstraping\"\"\"\n\tl = listDatawraps()\n\tprintf(\"Available datawraps for boostraping\\n\")\n\tfor k, v in l.iteritems() :\n\t\tprintf(k)\n\t\tprintf(\"~\"*len(k) + \"|\")\n\t\tfor vv in v :\n\t\t\tprintf(\" \"*len(k) + \"|\" + \"~~~:> \" + vv)\n\t\tprintf('\\n')"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nimports a genome shipped with pyGeno. Most of the datawraps only contain URLs towards data provided by third parties.", "response": "def importGenome(name, batchSize = 100) :\n\t\"\"\"Import a genome shipped with pyGeno. Most of the datawraps only contain URLs towards data provided by third parties.\"\"\"\n\tpath = os.path.join(this_dir, \"bootstrap_data\", \"genomes/\" + name)\n\tPG.importGenome(path, batchSize)"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nimports a SNP set shipped with pyGeno. Most of the datawraps only contain URLs towards data provided by third parties.", "response": "def importSNPs(name) :\n\t\"\"\"Import a SNP set shipped with pyGeno. Most of the datawraps only contain URLs towards data provided by third parties.\"\"\"\n\tpath = os.path.join(this_dir, \"bootstrap_data\", \"SNPs/\" + name)\n\tPS.importSNPs(path)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef log(self) :\n\t\t\n\t\tself.logs['epochDuration'].append(self.lastEpochDuration)\n\t\tself.logs['avg'].append(self.avg)\n\t\tself.logs['runtime'].append(self.runtime)\n\t\tself.logs['remtime'].append(self.remtime)", "response": "logs stats about the progression without printing anything on screen"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef saveLogs(self, filename) :\n\t\tf = open(filename, 'wb')\n\t\tcPickle.dump(self.logs, f)\n\t\tf.close()", "response": "dumps logs into a nice pickle"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef update(self, label = '', forceRefresh = False, log = False) :\n\t\tself.currEpoch += 1\n\t\ttim = time.time()\n\t\tif (tim - self.lastPrintTime > self.minRefeshTime) or forceRefresh :\n\t\t\tself._update()\n\t\t\t\n\t\t\twheelState = self.wheel[self.currEpoch%len(self.wheel)]\n\t\t\t\n\t\t\tif label == '' :\n\t\t\t\tslabel = self.label\n\t\t\telse :\n\t\t\t\tslabel = label\n\t\t\t\n\t\t\tif self.nbEpochs > 1 :\n\t\t\t\tratio = self.currEpoch/self.nbEpochs\n\t\t\t\tsnakeLen = int(self.width*ratio)\n\t\t\t\tvoidLen = int(self.width - (self.width*ratio))\n\n\t\t\t\tif snakeLen + voidLen < self.width :\n\t\t\t\t\tsnakeLen = self.width - voidLen\n\t\t\t\t\n\t\t\t\tself.bar = \"%s %s[%s:>%s] %.2f%% (%d/%d) runtime: %s, remaining: %s, avg: %s\" %(wheelState, slabel, \"~-\" * snakeLen, \" \" * voidLen, ratio*100, self.currEpoch, self.nbEpochs, self.runtime_hr, self.remtime_hr, self.formatTime(self.avg))\n\t\t\t\tif self.currEpoch == self.nbEpochs :\n\t\t\t\t\tself.close()\n\t\t\t\t\n\t\t\telse :\n\t\t\t\tw = self.width - len(self.miniSnake)\n\t\t\t\tv = self.currEpoch%(w+1)\n\t\t\t\tsnake = \"%s%s%s\" %(\" \" * (v), self.miniSnake, \" \" * (w-v))\n\t\t\t\tself.bar = \"%s %s[%s] %s%% (%d/%s) runtime: %s, remaining: %s, avg: %s\" %(wheelState, slabel, snake, '?', self.currEpoch, '?', self.runtime_hr, '?', self.formatTime(self.avg))\n\t\t\t\n\t\t\tsys.stdout.write(\"\\b\" * (len(self.bar)+1))\n\t\t\tsys.stdout.write(\" \" * (len(self.bar)+1))\n\t\t\tsys.stdout.write(\"\\b\" * (len(self.bar)+1))\n\t\t\tsys.stdout.write(self.bar)\n\t\t\tsys.stdout.flush()\n\t\t\tself.lastPrintTime = time.time()\n\t\t\t\n\t\t\tif log :\n\t\t\t\tself.log()", "response": "update the internal state of the object"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef aux_insertTree(childTree, parentTree):\n\tif childTree.x1 != None and childTree.x2 != None :\n\t\tparentTree.insert(childTree.x1, childTree.x2, childTree.name, childTree.referedObject)\n\n\tfor c in childTree.children:\n\t\taux_insertTree(c, parentTree)", "response": "This a private function that inserts a child tree into a parent tree."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef aux_moveTree(offset, tree):\n\tif tree.x1 != None and tree.x2 != None :\n\t\ttree.x1, tree.x2 = tree.x1+offset, tree.x2+offset\n\t\t\n\tfor c in tree.children:\n\t\taux_moveTree(offset, c)", "response": "This function is a private recursive function that translates tree to a given x1 and tree s children to a given x2"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef insert(self, x1, x2, name = '', referedObject = []) :\n\t\t\n\t\tif x1 > x2 :\n\t\t\txx1, xx2 = x2, x1\n\t\telse :\n\t\t\txx1, xx2 = x1, x2\n\n\t\trt = None\n\t\tinsertId = None\n\t\tchildrenToRemove = []\n\t\tfor i in range(len(self.children)) :\n\t\t\tif self.children[i].x1 == xx1 and xx2 == self.children[i].x2 :\n\t\t\t\tself.children[i].name = self.children[i].name + ' U ' + name\n\t\t\t\tself.children[i].referedObject.append(referedObject)\n\t\t\t\treturn self.children[i]\n\t\t\t\n\t\t\tif self.children[i].x1 <= xx1 and xx2 <= self.children[i].x2 :\n\t\t\t\treturn self.children[i].insert(x1, x2, name, referedObject)\n\t\t\t\n\t\t\telif xx1 <= self.children[i].x1 and self.children[i].x2 <= xx2 :\n\t\t\t\tif rt == None :\n\t\t\t\t\tif type(referedObject) is types.ListType :\n\t\t\t\t\t\trt = SegmentTree(xx1, xx2, name, referedObject, self, self.level+1)\n\t\t\t\t\telse :\n\t\t\t\t\t\trt = SegmentTree(xx1, xx2, name, [referedObject], self, self.level+1)\n\t\t\t\t\t\n\t\t\t\t\tinsertId = i\n\t\t\t\t\t\n\t\t\t\trt.__addChild(self.children[i])\n\t\t\t\tself.children[i].father = rt\n\t\t\t\tchildrenToRemove.append(self.children[i])\n\t\t\n\t\t\telif xx1 <= self.children[i].x1 and xx2 <= self.children[i].x2 :\n\t\t\t\tinsertId = i\n\t\t\t\tbreak\n\t\t\t\t\n\t\tif rt != None :\n\t\t\tself.__addChild(rt, insertId)\n\t\t\tfor c in childrenToRemove :\n\t\t\t\tself.children.remove(c)\n\t\telse :\n\t\t\tif type(referedObject) is types.ListType :\n\t\t\t\trt = SegmentTree(xx1, xx2, name, referedObject, self, self.level+1)\n\t\t\telse :\n\t\t\t\trt = SegmentTree(xx1, xx2, name, [referedObject], self, self.level+1)\n\t\t\t\n\t\t\tif insertId != None :\n\t\t\t\tself.__addChild(rt, insertId)\n\t\t\telse :\n\t\t\t\tself.__addChild(rt)\n\t\t\n\t\treturn rt", "response": "Insert the segment in it s right place and returns it."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef intersect(self, x1, x2 = None) :\n\t\t\n\t\tdef condition(x1, x2, tree) :\n\t\t\t#print self.id, tree.x1, tree.x2, x1, x2\n\t\t\tif (tree.x1 != None and tree.x2 != None) and (tree.x1 <= x1 and x1 < tree.x2 or tree.x1 <= x2 and x2 < tree.x2) :\n\t\t\t\treturn True\n\t\t\treturn False\n\t\t\t\n\t\tif x2 == None :\n\t\t\txx1, xx2 = x1, x1\n\t\telif x1 > x2 :\n\t\t\txx1, xx2 = x2, x1\n\t\telse :\n\t\t\txx1, xx2 = x1, x2\n\t\t\t\n\t\tc1 = self.__dichotomicSearch(xx1)\n\t\tc2 = self.__dichotomicSearch(xx2)\n\t\t\n\t\tif c1 == -1 or c2 == -1 :\n\t\t\treturn []\n\t\t\t\n\t\tif xx1 < self.children[c1].x1 :\n\t\t\tc1 -= 1\n\t\t\t\n\t\tinter = self.__radiateDown(x1, x2, c1, condition)\n\t\tif self.children[c1].id == self.children[c2].id :\n\t\t\tinter.extend(self.__radiateUp(x1, x2, c2+1, condition))\n\t\telse :\n\t\t\tinter.extend(self.__radiateUp(x1, x2, c2, condition))\n\t\t\n\t\tret = []\n\t\tfor c in inter :\n\t\t\tret.extend(c.intersect(x1, x2))\n\t\t\n\t\tinter.extend(ret)\n\t\treturn inter", "response": "Returns a list of all segments intersected by x1 and x2."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef __radiateDown(self, x1, x2, childId, condition) :\n\t\t\"Radiates down: walks self.children downward until condition is no longer verifed or there's no childrens left \"\n\t\tret = []\n\t\ti = childId\n\t\twhile 0 <= i :\n\t\t\tif condition(x1, x2, self.children[i]) :\n\t\t\t\tret.append(self.children[i])\n\t\t\telse :\n\t\t\t\tbreak\n\t\t\ti -= 1\n\t\treturn ret", "response": "Radiates down : walks self. children downward until condition is no longer verifed until there s no childrens left"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef __radiateUp(self, x1, x2, childId, condition) :\n\t\t\"Radiates uo: walks self.children upward until condition is no longer verifed or there's no childrens left \"\n\t\tret = []\n\t\ti = childId\n\t\twhile i < len(self.children):\n\t\t\tif condition(x1, x2, self.children[i]) :\n\t\t\t\tret.append(self.children[i])\n\t\t\telse :\n\t\t\t\tbreak\n\t\t\ti += 1\n\t\treturn ret", "response": "Radiates uo : walks self. children upward until condition is no longer verifed until there s no childrens left"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nremove all gaps between regions", "response": "def removeGaps(self) :\n\t\t\"\"\"Remove all gaps between regions\"\"\"\n\t\t\n\t\tfor i in range(1, len(self.children)) :\n\t\t\tif self.children[i].x1 > self.children[i-1].x2:\t\t\t\t\n\t\t\t\taux_moveTree(self.children[i-1].x2-self.children[i].x1, self.children[i])"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nreturns the total length of indexed regions", "response": "def getIndexedLength(self) :\n\t\t\"\"\"Returns the total length of indexed regions\"\"\"\n\t\tif self.x1 != None and self.x2 != None:\n\t\t\treturn self.x2 - self.x1\n\t\telse :\n\t\t\tif len(self.children) == 0 :\n\t\t\t\treturn 0\n\t\t\telse :\n\t\t\t\tl = self.children[0].x2 - self.children[0].x1\n\t\t\t\tfor i in range(1, len(self.children)) :\n\t\t\t\t\tl += self.children[i].x2 - self.children[i].x1 - max(0, self.children[i-1].x2 - self.children[i].x1)\n\t\t\t\treturn l"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nreturn a list of couples ( x1 x2 ) of all the first level indexed regions", "response": "def getFirstLevel(self) :\n\t\t\"\"\"returns a list of couples (x1, x2) of all the first level indexed regions\"\"\"\n\t\tres = []\n\t\tif len(self.children) > 0 :\n\t\t\tfor c in self.children:\n\t\t\t\tres.append((c.x1, c.x2)) \n\t\telse :\n\t\t\tif self.x1 != None :\n\t\t\t\tres = [(self.x1, self.x2)]\n\t\t\telse :\n\t\t\t\tres = None\n\t\treturn res"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nflatten the tree. The tree become a tree of depth 1 where overlapping regions have been merged together.", "response": "def flatten(self) :\n\t\t\"\"\"Flattens the tree. The tree become a tree of depth 1 where overlapping regions have been merged together\"\"\"\n\t\tif len(self.children) > 1 :\n\t\t\tchildren = self.children\n\t\t\tself.emptyChildren()\n\t\t\t\n\t\t\tchildren[0].emptyChildren()\n\t\t\tx1 = children[0].x1\n\t\t\tx2 = children[0].x2\n\t\t\trefObjs = [children[0].referedObject]\n\t\t\tname = children[0].name\n\t\t\t\n\t\t\tfor i in range(1, len(children)) :\n\t\t\t\tchildren[i].emptyChildren()\n\t\t\t\tif children[i-1] >= children[i] :\n\t\t\t\t\tx2 = children[i].x2\n\t\t\t\t\trefObjs.append(children[i].referedObject)\n\t\t\t\t\tname += \" U \" + children[i].name\n\t\t\t\telse :\n\t\t\t\t\tif len(refObjs) == 1 :\n\t\t\t\t\t\trefObjs = refObjs[0]\n\t\t\n\t\t\t\t\tself.insert(x1, x2, name, refObjs)\n\t\t\t\t\tx1 = children[i].x1\n\t\t\t\t\tx2 = children[i].x2\n\t\t\t\t\trefObjs = [children[i].referedObject]\n\t\t\t\t\tname = children[i].name\n\t\t\t\n\t\t\tif len(refObjs) == 1 :\n\t\t\t\trefObjs = refObjs[0]\n\t\t\n\t\t\tself.insert(x1, x2, name, refObjs)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef move(self, newX1) :\n\t\tif self.x1 != None and self.x2 != None :\n\t\t\toffset = newX1-self.x1\n\t\t\taux_moveTree(offset, self)\n\t\telif len(self.children) > 0 :\n\t\t\toffset = newX1-self.children[0].x1\n\t\t\taux_moveTree(offset, self)", "response": "Moves tree to a new starting position updates x1s of children"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nencoding a binary reprensentation default sequence and polymorphisms list", "response": "def encode(self, sequence):\n\t\t\"\"\"Returns a tuple (binary reprensentation, default sequence, polymorphisms list)\"\"\"\n\t\t\n\t\tpolymorphisms = []\n\t\tdefaultSequence = '' \n\t\tbinSequence = array.array(self.forma.typecode)\n\t\tb = 0\n\t\ti = 0\n\t\ttrueI = 0 #not inc in case if poly\n\t\tpoly = set()\n\t\twhile i < len(sequence)-1:\n\t\t\tb = b | self.forma[self.charToBin[sequence[i]]]\n\t\t\tif sequence[i+1] == '/' :\n\t\t\t\tpoly.add(sequence[i])\n\t\t\t\ti += 2\n\t\t\telse :\n\t\t\t\tbinSequence.append(b)\n\t\t\t\tif len(poly) > 0 :\n\t\t\t\t\tpoly.add(sequence[i])\n\t\t\t\t\tpolymorphisms.append((trueI, poly))\n\t\t\t\t\tpoly = set()\n\t\t\t\t\n\t\t\t\tbb = 0\n\t\t\t\twhile b % 2 != 0 :\n\t\t\t\t\tb = b/2\n\t\t\t\t\t\n\t\t\t\tdefaultSequence += sequence[i]\n\t\t\t\tb = 0\n\t\t\t\ti += 1\n\t\t\t\ttrueI += 1\n\t\t\n\t\tif i < len(sequence) :\n\t\t\tb = b | self.forma[self.charToBin[sequence[i]]]\n\t\t\tbinSequence.append(b)\n\t\t\tif len(poly) > 0 :\n\t\t\t\tif sequence[i] not in poly :\n\t\t\t\t\tpoly.add(sequence[i])\n\t\t\t\tpolymorphisms.append((trueI, poly))\n\t\t\tdefaultSequence += sequence[i]\n\t\t\n\t\treturn (binSequence, defaultSequence, polymorphisms)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\ngetting the list of variants from a polymorphisme", "response": "def __getSequenceVariants(self, x1, polyStart, polyStop, listSequence) :\n\t\t\"\"\"polyStop, is the polymorphisme at wixh number where the calcul of combinaisons stops\"\"\"\n\t\tif polyStart < len(self.polymorphisms) and polyStart < polyStop: \n\t\t\tsequence = copy.copy(listSequence)\n\t\t\tret = []\n\t\t\t\n\t\t\tpk = self.polymorphisms[polyStart]\n\t\t\tposInSequence = pk[0]-x1\n\t\t\tif posInSequence < len(listSequence) : \n\t\t\t\tfor allele in pk[1] :\n\t\t\t\t\tsequence[posInSequence] = allele\n\t\t\t\t\tret.extend(self.__getSequenceVariants(x1, polyStart+1, polyStop, sequence))\n\t\t\t\n\t\t\treturn ret\n\t\telse :\n\t\t\treturn [''.join(listSequence)]"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef getSequenceVariants(self, x1 = 0, x2 = -1, maxVariantNumber = 128) :\n\t\tif x2 == -1 :\n\t\t\txx2 = len(self.defaultSequence)\n\t\telse :\n\t\t\txx2 = x2\n\t\t\n\t\tpolyStart = None\n\t\tnbP = 1\n\t\tstopped = False\n\t\tj = 0\n\t\tfor p in self.polymorphisms :\n\t\t\tif p[0] >= xx2 :\n\t\t\t\tbreak\n\t\t\t\n\t\t\tif x1 <= p[0] :\n\t\t\t\tif polyStart == None :\n\t\t\t\t\tpolyStart = j\n\t\t\t\t\n\t\t\t\tnbP *= len(p[1])\n\t\t\t\t\n\t\t\t\tif nbP > maxVariantNumber :\n\t\t\t\t\tstopped = True\n\t\t\t\t\tbreak\n\t\t\tj += 1\n\t\t\n\t\tif polyStart == None :\n\t\t\treturn (stopped, [self.defaultSequence[x1:xx2]])\n\t\t\n\t\treturn (stopped, self.__getSequenceVariants(x1, polyStart, j, list(self.defaultSequence[x1:xx2])))", "response": "returns the sequences resulting from all combinaisons between x1 and x2"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef getNbVariants(self, x1, x2 = -1) :\n\t\tif x2 == -1 :\n\t\t\txx2 = len(self.defaultSequence)\n\t\telse :\n\t\t\txx2 = x2\n\t\t\n\t\tnbP = 1\n\t\tfor p in self.polymorphisms:\n\t\t\tif x1 <= p[0] and p[0] <= xx2 :\n\t\t\t\tnbP *= len(p[1])\n\t\t\n\t\treturn nbP", "response": "returns the nb of variants of sequences between x1 and x2"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _dichFind(self, needle, currHaystack, offset, lst = None) :\n\t\t\n\t\tif len(currHaystack) == 1 :\n\t\t\tif (offset <= (len(self) - len(needle))) and (currHaystack[0] & needle[0]) > 0 and (self[offset+len(needle)-1] & needle[-1]) > 0 :\n\t\t\t\tfound = True\n\t\t\t\tfor i in xrange(1, len(needle)-1) :\n\t\t\t\t\tif self[offset + i] & needle[i] == 0 :\n\t\t\t\t\t\tfound = False\n\t\t\t\t\t\tbreak\n\t\t\t\tif found :\n\t\t\t\t\tif lst is not None :\n\t\t\t\t\t\tlst.append(offset)\n\t\t\t\t\telse :\n\t\t\t\t\t\treturn offset\n\t\t\t\telse :\n\t\t\t\t\tif lst is None :\n\t\t\t\t\t\treturn -1\n\t\telse :\n\t\t\tif (offset <= (len(self) - len(needle))) :\n\t\t\t\tif lst is not None :\n\t\t\t\t\tself._dichFind(needle, currHaystack[:len(currHaystack)/2], offset, lst)\n\t\t\t\t\tself._dichFind(needle, currHaystack[len(currHaystack)/2:], offset + len(currHaystack)/2, lst)\n\t\t\t\telse :\n\t\t\t\t\tv1 = self._dichFind(needle, currHaystack[:len(currHaystack)/2], offset, lst)\n\t\t\t\t\tif v1 > -1 :\n\t\t\t\t\t\treturn v1\n\t\t\t\t\t\n\t\t\t\t\treturn self._dichFind(needle, currHaystack[len(currHaystack)/2:], offset + len(currHaystack)/2, lst)\n\t\t\treturn -1", "response": "dichotomic search, if lst is None, will return the first position found. If it's a list, will return a list of all positions in lst. returns -1 or [] if no match found"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nusing KMP algorithm to search the first occurrence in the input_sequence of the pattern. both arguments are integer arrays. return the position of the first occurrence if found ; otherwise - 1.", "response": "def _kmp_search_first(self, pInput_sequence, pPattern):\n \"\"\"use KMP algorithm to search the first occurrence in the input_sequence of the pattern. both arguments are integer arrays. return the position of the occurence if found; otherwise, -1.\"\"\"\n input_sequence, pattern = pInput_sequence, [len(bin(e)) for e in pPattern]\n n, m = len(input_sequence), len(pattern)\n d = p = 0\n next = self._kmp_construct_next(pattern)\n while d < n and p < m:\n p = next[len(bin(input_sequence[d]))][p]\n d += 1\n if p == m: return d - p\n else: return -1"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nuse KMP algorithm to search all occurences in the input_sequence of the pattern. both arguments are integer arrays. return a list of the positions of the occurences if found otherwise [].", "response": "def _kmp_search_all(self, pInput_sequence, pPattern):\n \"\"\"use KMP algorithm to search all occurrence in the input_sequence of the pattern. both arguments are integer arrays. return a list of the positions of the occurences if found; otherwise, [].\"\"\"\n r = []\n input_sequence, pattern = [len(bin(e)) for e in pInput_sequence], [len(bin(e)) for e in pPattern]\n n, m = len(input_sequence), len(pattern)\n d = p = 0\n next = self._kmp_construct_next(pattern)\n while d < n:\n p = next[input_sequence[d]][p]\n d += 1\n if p == m:\n r.append(d - m)\n p = 0\n return r"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _kmp_find(self, needle, haystack, lst = None):\n if lst != None: return self._kmp_search_all(haystack, needle)\n else: return self._kmp_search_first(haystack, needle)", "response": "find with KMP - searching."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef findByBiSearch(self, strSeq) :\n\t\tarr = self.encode(strSeq)\n\t\treturn self._dichFind(arr[0], self, 0, lst = None)", "response": "returns the first occurence of strSeq in self. Takes polymorphisms into account"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef findAllByBiSearch(self, strSeq) :\n\t\tarr = self.encode(strSeq)\n\t\tlst = []\n\t\tself._dichFind(arr[0], self, 0, lst)\n\t\treturn lst", "response": "Same as find but returns a list of all occurences"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef find(self, strSeq) :\n\t\tarr = self.encode(strSeq)\n return self._kmp_find(arr[0], self)", "response": "returns the first occurence of strSeq in self. Takes polymorphisms into account"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef findAll(self, strSeq) :\n\t\tarr = self.encode(strSeq)\n\t\tlst = []\n lst = self._kmp_find(arr[0], self, lst)\n\t\treturn lst", "response": "Same as find but returns a list of all occurences"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef decode(self, binSequence):\n\t\ttry:\n\t\t\tbinSeq = iter(binSequence[0])\n\t\texcept TypeError, te:\n\t\t\tbinSeq = binSequence\n \n\t\tret = ''\n\t\tfor b in binSeq :\n\t\t\tch = ''\n\t\t\tfor c in self.charToBin :\n\t\t\t\tif b & self.forma[self.charToBin[c]] > 0 :\n\t\t\t\t\tch += c +'/'\n\t\t\tif ch == '' :\n\t\t\t\traise KeyError('Key %d unkowom, bad format' % b)\n\t\t\t\n\t\t\tret += ch[:-1]\n\t\treturn ret", "response": "decodes a binary sequence to return a string"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nparse a file and returns a dictionary of the object properties", "response": "def parse(self, filename, gziped = False, stream = False) :\n\t\t\"\"\"opens a file\"\"\"\n\t\tself.stream = stream\n\t\t\n\t\tif gziped :\n\t\t\tself.f = gzip.open(filename)\n\t\telse :\n\t\t\tself.f = open(filename)\n\t\t\n\t\tself.filename = filename\n\t\tself.gziped = gziped\n\t\t\n\t\tlineId = 0\n\t\tinLegend = True\n\t\twhile inLegend :\n\t\t\tll = self.f.readline()\n\t\t\tl = ll.replace('\\r', '\\n').replace('\\n', '')\n\t\t\tif l[:2] == '##' :\n\t\t\t\teqPos = l.find('=')\n\t\t\t\tkey = l[2:eqPos]\n\t\t\t\tvalues = l[eqPos+1:].strip()\n\t\t\t\t\n\t\t\t\tif l[eqPos+1] != '<' :\n\t\t\t\t\tself.meta[key] = values\n\t\t\t\telse :\n\t\t\t\t\tif key not in self.meta :\n\t\t\t\t\t\tself.meta[key] = {}\n\t\t\t\t\tsvalues = l[eqPos+2:-1].split(',') #remove the < and > that surounf the string \n\t\t\t\t\tidKey = svalues[0].split('=')[1]\n\t\t\t\t\tself.meta[key][idKey] = {} \n\t\t\t\t\ti = 1\n\t\t\t\t\tfor v in svalues[1:] :\n\t\t\t\t\t\tsv = v.split(\"=\")\n\t\t\t\t\t\tfield = sv[0]\n\t\t\t\t\t\tvalue = sv[1]\n\t\t\t\t\t\tif field.lower() == 'description' :\n\t\t\t\t\t\t\tself.meta[key][idKey][field] = ','.join(svalues[i:])[len(field)+2:-1]\n\t\t\t\t\t\t\tbreak\n\t\t\t\t\t\tself.meta[key][idKey][field] = value\n\t\t\t\t\t\ti += 1\n\t\t\telif l[:6] == '#CHROM': #we are in legend\n\t\t\t\tsl = l.split('\\t')\n\t\t\t\tfor i in range(len(sl)) :\n\t\t\t\t\tself.legend[sl[i]] = i\n\t\t\t\t\tself.dnegel[i] = sl[i]\n\t\t\t\tbreak\n\t\t\t\n\t\t\tlineId += 1\n\t\t\n\t\tif not stream :\n\t\t\tself.lines = self.f.readlines()\n\t\t\tself.f.close()"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef _decode8(self, offset):\n # UTF-8 Strings contain two lengths, as they might differ:\n # 1) the UTF-16 length\n str_len, skip = self._decode_length(offset, 1)\n offset += skip\n\n # 2) the utf-8 string length\n encoded_bytes, skip = self._decode_length(offset, 1)\n offset += skip\n\n data = self.m_charbuff[offset: offset + encoded_bytes]\n\n assert self.m_charbuff[offset + encoded_bytes] == 0, \\\n \"UTF-8 String is not null terminated! At offset={}\".format(offset)\n\n return self._decode_bytes(data, 'utf-8', str_len)", "response": "Decode an UTF - 8 String at the given offset."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef _decode16(self, offset):\n str_len, skip = self._decode_length(offset, 2)\n offset += skip\n\n # The len is the string len in utf-16 units\n encoded_bytes = str_len * 2\n\n data = self.m_charbuff[offset: offset + encoded_bytes]\n\n assert self.m_charbuff[offset + encoded_bytes:offset + encoded_bytes + 2] == b\"\\x00\\x00\", \\\n \"UTF-16 String is not null terminated! At offset={}\".format(offset)\n\n return self._decode_bytes(data, 'utf-16', str_len)", "response": "Decode an UTF - 16 String at the given offset."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\ndecode the length of the string at the given offset.", "response": "def _decode_length(self, offset, sizeof_char):\n \"\"\"\n Generic Length Decoding at offset of string\n\n The method works for both 8 and 16 bit Strings.\n Length checks are enforced:\n * 8 bit strings: maximum of 0x7FFF bytes (See\n http://androidxref.com/9.0.0_r3/xref/frameworks/base/libs/androidfw/ResourceTypes.cpp#692)\n * 16 bit strings: maximum of 0x7FFFFFF bytes (See\n http://androidxref.com/9.0.0_r3/xref/frameworks/base/libs/androidfw/ResourceTypes.cpp#670)\n\n :param offset: offset into the string data section of the beginning of\n the string\n :param sizeof_char: number of bytes per char (1 = 8bit, 2 = 16bit)\n :returns: tuple of (length, read bytes)\n \"\"\"\n sizeof_2chars = sizeof_char << 1\n fmt = \"<2{}\".format('B' if sizeof_char == 1 else 'H')\n highbit = 0x80 << (8 * (sizeof_char - 1))\n\n length1, length2 = unpack(fmt, self.m_charbuff[offset:(offset + sizeof_2chars)])\n\n if (length1 & highbit) != 0:\n length = ((length1 & ~highbit) << (8 * sizeof_char)) | length2\n size = sizeof_2chars\n else:\n length = length1\n size = sizeof_char\n\n if sizeof_char == 1:\n assert length <= 0x7FFF, \"length of UTF-8 string is too large! At offset={}\".format(offset)\n else:\n assert length <= 0x7FFFFFFF, \"length of UTF-16 string is too large! At offset={}\".format(offset)\n\n return length, size"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\napplies some fixes to element named and attribute names.", "response": "def _fix_name(self, name):\n \"\"\"\n Apply some fixes to element named and attribute names.\n Try to get conform to:\n > Like element names, attribute names are case-sensitive and must start with a letter or underscore.\n > The rest of the name can contain letters, digits, hyphens, underscores, and periods.\n See: https://msdn.microsoft.com/en-us/library/ms256152(v=vs.110).aspx\n\n :param name: Name of the attribute\n :return: a fixed version of the name\n \"\"\"\n if not name[0].isalpha() and name[0] != \"_\":\n log.warning(\"Invalid start for name '{}'\".format(name))\n self.packerwarning = True\n name = \"_{}\".format(name)\n if name.startswith(\"android:\"):\n # Seems be a common thing...\n # Actually this means that the Manifest is likely to be broken, as\n # usually no namespace URI is set in this case.\n log.warning(\n \"Name '{}' starts with 'android:' prefix! \"\n \"The Manifest seems to be broken? Removing prefix.\".format(\n name\n )\n )\n self.packerwarning = True\n name = name[len(\"android:\"):]\n if \":\" in name:\n # Print out an extra warning\n log.warning(\"Name seems to contain a namespace prefix: '{}'\".format(name))\n if not re.match(r\"^[a-zA-Z0-9._-]*$\", name):\n log.warning(\"Name '{}' contains invalid characters!\".format(name))\n self.packerwarning = True\n name = re.sub(r\"[^a-zA-Z0-9._-]\", \"_\", name)\n\n return name"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nfix the value of the attribute to be a string that is not part of the XML.", "response": "def _fix_value(self, value):\n \"\"\"\n Return a cleaned version of a value\n according to the specification:\n > Char\t ::= \t#x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF]\n\n See https://www.w3.org/TR/xml/#charsets\n\n :param value: a value to clean\n :return: the cleaned value\n \"\"\"\n if not self.__charrange or not self.__replacement:\n if sys.maxunicode == 0xFFFF:\n # Fix for python 2.x, surrogate pairs does not match in regex\n self.__charrange = re.compile(\n u'^([\\u0020-\\uD7FF\\u0009\\u000A\\u000D\\uE000-\\uFFFD]|[\\uD800-\\uDBFF][\\uDC00-\\uDFFF])*$')\n # TODO: this regex is slightly wrong... surrogates are not matched as pairs.\n self.__replacement = re.compile(u'[^\\u0020-\\uDBFF\\u0009\\u000A\\u000D\\uE000-\\uFFFD\\uDC00-\\uDFFF]')\n else:\n self.__charrange = re.compile(u'^[\\u0020-\\uD7FF\\u0009\\u000A\\u000D\\uE000-\\uFFFD\\U00010000-\\U0010FFFF]*$')\n self.__replacement = re.compile(u'[^\\u0020-\\uD7FF\\u0009\\u000A\\u000D\\uE000-\\uFFFD\\U00010000-\\U0010FFFF]')\n\n # Reading string until \\x00. This is the same as aapt does.\n if \"\\x00\" in value:\n self.packerwarning = True\n log.warning(\n \"Null byte found in attribute value at position {}: \"\n \"Value(hex): '{}'\".format(\n value.find(\"\\x00\"),\n binascii.hexlify(value.encode(\"utf-8\"))\n )\n )\n value = value[:value.find(\"\\x00\")]\n\n if not self.__charrange.match(value):\n log.warning(\"Invalid character in value found. Replacing with '_'.\")\n self.packerwarning = True\n value = self.__replacement.sub('_', value)\n return value"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _apk_analysis(self):\n i = \"AndroidManifest.xml\"\n try:\n manifest_data = self.zip.read(i)\n except KeyError:\n log.warning(\"Missing AndroidManifest.xml. Is this an APK file?\")\n else:\n ap = AXMLPrinter(manifest_data)\n\n if not ap.is_valid():\n log.error(\"Error while parsing AndroidManifest.xml - is the file valid?\")\n return\n\n self.axml[i] = ap\n self.xml[i] = self.axml[i].get_xml_obj()\n\n if self.axml[i].is_packed():\n log.warning(\"XML Seems to be packed, operations on the AndroidManifest.xml might fail.\")\n\n if self.xml[i] is not None:\n if self.xml[i].tag != \"manifest\":\n log.error(\"AndroidManifest.xml does not start with a tag! Is this a valid APK?\")\n return\n\n self.package = self.get_attribute_value(\"manifest\", \"package\")\n self.androidversion[\"Code\"] = self.get_attribute_value(\"manifest\", \"versionCode\")\n self.androidversion[\"Name\"] = self.get_attribute_value(\"manifest\", \"versionName\")\n permission = list(self.get_all_attribute_value(\"uses-permission\", \"name\"))\n self.permissions = list(set(self.permissions + permission))\n\n for uses_permission in self.find_tags(\"uses-permission\"):\n self.uses_permissions.append([\n self.get_value_from_tag(uses_permission, \"name\"),\n self._get_permission_maxsdk(uses_permission)\n ])\n\n # getting details of the declared permissions\n for d_perm_item in self.find_tags('permission'):\n d_perm_name = self._get_res_string_value(\n str(self.get_value_from_tag(d_perm_item, \"name\")))\n d_perm_label = self._get_res_string_value(\n str(self.get_value_from_tag(d_perm_item, \"label\")))\n d_perm_description = self._get_res_string_value(\n str(self.get_value_from_tag(d_perm_item, \"description\")))\n d_perm_permissionGroup = self._get_res_string_value(\n str(self.get_value_from_tag(d_perm_item, \"permissionGroup\")))\n d_perm_protectionLevel = self._get_res_string_value(\n str(self.get_value_from_tag(d_perm_item, \"protectionLevel\")))\n\n d_perm_details = {\n \"label\": d_perm_label,\n \"description\": d_perm_description,\n \"permissionGroup\": d_perm_permissionGroup,\n \"protectionLevel\": d_perm_protectionLevel,\n }\n self.declared_permissions[d_perm_name] = d_perm_details\n\n self.valid_apk = True", "response": "Run analysis on the APK file and set all the attributes in the class attribute."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\ngetting the filetype guessed for a buffer.", "response": "def _get_file_magic_name(self, buffer):\n \"\"\"\n Return the filetype guessed for a buffer\n :param buffer: bytes\n :return: str of filetype\n \"\"\"\n default = \"Unknown\"\n ftype = None\n\n try:\n # Magic is optional\n import magic\n except ImportError:\n return default\n\n try:\n # There are several implementations of magic,\n # unfortunately all called magic\n # We use this one: https://github.com/ahupp/python-magic/\n getattr(magic, \"MagicException\")\n except AttributeError:\n # Looks like no magic was installed\n return default\n\n try:\n ftype = magic.from_buffer(buffer[:1024])\n except magic.MagicError as e:\n log.exception(\"Error getting the magic type!\")\n return default\n\n if not ftype:\n return default\n else:\n return self._patch_magic(buffer, ftype)"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef get_files_types(self):\n if self._files == {}:\n # Generate File Types / CRC List\n for i in self.get_files():\n buffer = self.zip.read(i)\n self.files_crc32[i] = crc32(buffer)\n # FIXME why not use the crc from the zipfile?\n # should be validated as well.\n # crc = self.zip.getinfo(i).CRC\n self._files[i] = self._get_file_magic_name(buffer)\n\n return self._files", "response": "Return the files inside the APK with their associated types"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\npatches magic to return correct guess by mime libraries", "response": "def _patch_magic(self, buffer, orig):\n \"\"\"\n Overwrite some probably wrong detections by mime libraries\n\n :param buffer: bytes of the file to detect\n :param orig: guess by mime libary\n :return: corrected guess\n \"\"\"\n if (\"Zip\" in orig) or ('(JAR)' in orig):\n val = is_android_raw(buffer)\n if val == \"APK\":\n return \"Android application package file\"\n\n return orig"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef get_files_crc32(self):\n if self.files_crc32 == {}:\n for i in self.get_files():\n buffer = self.zip.read(i)\n self.files_crc32[i] = crc32(buffer)\n\n return self.files_crc32", "response": "Calculates and returns a dictionary of filenames and CRC32s"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef is_multidex(self):\n dexre = re.compile(\"^classes(\\d+)?.dex$\")\n return len([instance for instance in self.get_files() if dexre.search(instance)]) > 1", "response": "Test if the APK has multiple DEX files"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\ngets the elements in xml files which match with the tag name and the specific attribute name.", "response": "def get_elements(self, tag_name, attribute, with_namespace=True):\n \"\"\"\n Deprecated: use `get_all_attribute_value()` instead\n Return elements in xml files which match with the tag name and the specific attribute\n :param tag_name: a string which specify the tag name\n :param attribute: a string which specify the attribute\n \"\"\"\n for i in self.xml:\n if self.xml[i] is None:\n continue\n for item in self.xml[i].findall('.//' + tag_name):\n if with_namespace:\n value = item.get(self._ns(attribute))\n else:\n value = item.get(attribute)\n # There might be an attribute without the namespace\n if value:\n yield self._format_value(value)"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _format_value(self, value):\n if len(value) > 0:\n if value[0] == \".\":\n value = self.package + value\n else:\n v_dot = value.find(\".\")\n if v_dot == 0:\n value = self.package + \".\" + value\n elif v_dot == -1:\n value = self.package + \".\" + value\n return value", "response": "Format a value with packagename if not already set"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef get_element(self, tag_name, attribute, **attribute_filter):\n for i in self.xml:\n if self.xml[i] is None:\n continue\n tag = self.xml[i].findall('.//' + tag_name)\n if len(tag) == 0:\n return None\n for item in tag:\n skip_this_item = False\n for attr, val in list(attribute_filter.items()):\n attr_val = item.get(self._ns(attr))\n if attr_val != val:\n skip_this_item = True\n break\n\n if skip_this_item:\n continue\n\n value = item.get(self._ns(attribute))\n\n if value is not None:\n return value\n return None", "response": "Return the element in xml files which match with the tag name and the specific attribute name."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nreturn true if the attributes of the tag match the ones in attribute filter", "response": "def is_tag_matched(self, tag, **attribute_filter):\n \"\"\"\n Return true if the attributes matches in attribute filter\n :param tag: specify the tag element\n :type tag: Element\n :param attribute: specify the attribute\n :type attribute: string\n \"\"\"\n if len(attribute_filter) <= 0:\n return True\n for attr, value in attribute_filter.items():\n # TODO: figure out if both android:name and name tag exist which one to give preference\n _value = tag.get(self._ns(attr))\n if _value is None:\n log.warning(\"Failed to get the attribute with namespace\")\n _value = tag.get(attr)\n if _value != value:\n return False\n return True"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef name(self):\n if self.m_name == -1 or (self.m_event != const.START_TAG and self.m_event != const.END_TAG):\n return u''\n\n return self.sb[self.m_name]", "response": "Return the name of the tag"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef namespace(self):\n if self.m_name == -1 or (self.m_event != const.START_TAG and self.m_event != const.END_TAG):\n return u''\n\n # No Namespace\n if self.m_namespaceUri == 0xFFFFFFFF:\n return u''\n\n return self.sb[self.m_namespaceUri]", "response": "Return the Namespace URI of the current tag."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nreturning the current text assosicated with the current text", "response": "def text(self):\n \"\"\"\n Return the String assosicated with the current text\n \"\"\"\n if self.m_name == -1 or self.m_event != const.TEXT:\n return u''\n\n return self.sb[self.m_name]"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nreturning the Namespace URI for the attribute at the given index.", "response": "def getAttributeNamespace(self, index):\n \"\"\"\n Return the Namespace URI (if any) for the attribute\n \"\"\"\n uri = self.getAttributeUri(index)\n\n # No Namespace\n if uri == 0xFFFFFFFF:\n return u''\n\n return self.sb[uri]"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nreturns the String which represents the attribute name", "response": "def getAttributeName(self, index):\n \"\"\"\n Returns the String which represents the attribute name\n \"\"\"\n offset = self._get_attribute_offset(index)\n name = self.m_attributes[offset + const.ATTRIBUTE_IX_NAME]\n\n res = self.sb[name]\n # If the result is a (null) string, we need to look it up.\n if not res:\n attr = self.m_resourceIDs[name]\n if attr in public.SYSTEM_RESOURCES['attributes']['inverse']:\n res = 'android:' + public.SYSTEM_RESOURCES['attributes']['inverse'][attr]\n else:\n # Attach the HEX Number, so for multiple missing attributes we do not run\n # into problems.\n res = 'android:UNKNOWN_SYSTEM_ATTRIBUTE_{:08x}'.format(attr)\n\n return res"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nreturn the type of the attribute at the given index", "response": "def getAttributeValueType(self, index):\n \"\"\"\n Return the type of the attribute at the given index\n\n :param index: index of the attribute\n \"\"\"\n offset = self._get_attribute_offset(index)\n return self.m_attributes[offset + const.ATTRIBUTE_IX_VALUE_TYPE]"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef getAttributeValueData(self, index):\n offset = self._get_attribute_offset(index)\n return self.m_attributes[offset + const.ATTRIBUTE_IX_VALUE_DATA]", "response": "Returns the data of the attribute at the given index"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef get_qualifier(self):\n res = []\n\n mcc = self.imsi & 0xFFFF\n mnc = (self.imsi & 0xFFFF0000) >> 16\n if mcc != 0:\n res.append(\"mcc%d\" % mcc)\n if mnc != 0:\n res.append(\"mnc%d\" % mnc)\n\n if self.locale != 0:\n res.append(self.get_language_and_region())\n\n screenLayout = self.screenConfig & 0xff\n if (screenLayout & const.MASK_LAYOUTDIR) != 0:\n if screenLayout & const.MASK_LAYOUTDIR == const.LAYOUTDIR_LTR:\n res.append(\"ldltr\")\n elif screenLayout & const.MASK_LAYOUTDIR == const.LAYOUTDIR_RTL:\n res.append(\"ldrtl\")\n else:\n res.append(\"layoutDir_%d\" % (screenLayout & const.MASK_LAYOUTDIR))\n\n smallestScreenWidthDp = (self.screenConfig & 0xFFFF0000) >> 16\n if smallestScreenWidthDp != 0:\n res.append(\"sw%ddp\" % smallestScreenWidthDp)\n\n screenWidthDp = self.screenSizeDp & 0xFFFF\n screenHeightDp = (self.screenSizeDp & 0xFFFF0000) >> 16\n if screenWidthDp != 0:\n res.append(\"w%ddp\" % screenWidthDp)\n if screenHeightDp != 0:\n res.append(\"h%ddp\" % screenHeightDp)\n\n if (screenLayout & const.MASK_SCREENSIZE) != const.SCREENSIZE_ANY:\n if screenLayout & const.MASK_SCREENSIZE == const.SCREENSIZE_SMALL:\n res.append(\"small\")\n elif screenLayout & const.MASK_SCREENSIZE == const.SCREENSIZE_NORMAL:\n res.append(\"normal\")\n elif screenLayout & const.MASK_SCREENSIZE == const.SCREENSIZE_LARGE:\n res.append(\"large\")\n elif screenLayout & const.MASK_SCREENSIZE == const.SCREENSIZE_XLARGE:\n res.append(\"xlarge\")\n else:\n res.append(\"screenLayoutSize_%d\" % (screenLayout & const.MASK_SCREENSIZE))\n if (screenLayout & const.MASK_SCREENLONG) != 0:\n if screenLayout & const.MASK_SCREENLONG == const.SCREENLONG_NO:\n res.append(\"notlong\")\n elif screenLayout & const.MASK_SCREENLONG == const.SCREENLONG_YES:\n res.append(\"long\")\n else:\n res.append(\"screenLayoutLong_%d\" % (screenLayout & const.MASK_SCREENLONG))\n\n density = (self.screenType & 0xffff0000) >> 16\n if density != const.DENSITY_DEFAULT:\n if density == const.DENSITY_LOW:\n res.append(\"ldpi\")\n elif density == const.DENSITY_MEDIUM:\n res.append(\"mdpi\")\n elif density == const.DENSITY_TV:\n res.append(\"tvdpi\")\n elif density == const.DENSITY_HIGH:\n res.append(\"hdpi\")\n elif density == const.DENSITY_XHIGH:\n res.append(\"xhdpi\")\n elif density == const.DENSITY_XXHIGH:\n res.append(\"xxhdpi\")\n elif density == const.DENSITY_XXXHIGH:\n res.append(\"xxxhdpi\")\n elif density == const.DENSITY_NONE:\n res.append(\"nodpi\")\n elif density == const.DENSITY_ANY:\n res.append(\"anydpi\")\n else:\n res.append(\"%ddpi\" % (density))\n\n touchscreen = (self.screenType & 0xff00) >> 8\n if touchscreen != const.TOUCHSCREEN_ANY:\n if touchscreen == const.TOUCHSCREEN_NOTOUCH:\n res.append(\"notouch\")\n elif touchscreen == const.TOUCHSCREEN_FINGER:\n res.append(\"finger\")\n elif touchscreen == const.TOUCHSCREEN_STYLUS:\n res.append(\"stylus\")\n else:\n res.append(\"touchscreen_%d\" % touchscreen)\n\n screenSize = self.screenSize\n if screenSize != 0:\n screenWidth = self.screenSize & 0xffff\n screenHeight = (self.screenSize & 0xffff0000) >> 16\n res.append(\"%dx%d\" % (screenWidth, screenHeight))\n\n version = self.version\n if version != 0:\n sdkVersion = self.version & 0xffff\n minorVersion = (self.version & 0xffff0000) >> 16\n res.append(\"v%d\" % sdkVersion)\n if minorVersion != 0:\n res.append(\".%d\" % minorVersion)\n\n return \"-\".join(res)", "response": "Get the resource name qualifier for the current configuration."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef make_cache_key(request):\n headers = frozenset(request._p['header'].items())\n path = frozenset(request._p['path'].items())\n query = frozenset(request._p['query'])\n return (request.url, headers, path, query)", "response": "Generate a cache key from request object data"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef check_cache(cache):\n if isinstance(cache, BaseCache):\n return cache\n elif cache is False:\n return DictCache()\n elif cache is None:\n return DummyCache()\n else:\n raise ValueError('Provided cache must implement BaseCache')", "response": "check if a cache fits esipy needs or not"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nreturning the time left in second for an expires header", "response": "def get_cache_time_left(expires_header):\n \"\"\" return the time left in second for an expires header \"\"\"\n epoch = datetime(1970, 1, 1)\n # this date is ALWAYS in UTC (RFC 7231)\n expire = (\n datetime(\n *parsedate(expires_header)[:6]\n ) - epoch\n ).total_seconds()\n now = (datetime.utcnow() - epoch).total_seconds()\n return int(expire) - int(now)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\ngenerate a code_verifier for the current user.", "response": "def generate_code_verifier(n_bytes=64):\n \"\"\"\n source: https://github.com/openstack/deb-python-oauth2client\n Generates a 'code_verifier' as described in section 4.1 of RFC 7636.\n This is a 'high-entropy cryptographic random string' that will be\n impractical for an attacker to guess.\n Args:\n n_bytes: integer between 31 and 96, inclusive. default: 64\n number of bytes of entropy to include in verifier.\n Returns:\n Bytestring, representing urlsafe base64-encoded random data.\n \"\"\"\n verifier = base64.urlsafe_b64encode(\n os.urandom(n_bytes)\n ).rstrip(b'=').decode('utf-8')\n\n # https://tools.ietf.org/html/rfc7636#section-4.1\n # minimum length of 43 characters and a maximum length of 128 characters.\n if len(verifier) < 43:\n raise ValueError(\"Verifier too short. n_bytes must be > 30.\")\n elif len(verifier) > 128:\n raise ValueError(\"Verifier too long. n_bytes must be < 97.\")\n else:\n return verifier"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef generate_code_challenge(verifier):\n digest = hashlib.sha256(verifier.encode('utf-8')).digest()\n return base64.urlsafe_b64encode(digest).rstrip(b'=').decode('utf-8')", "response": "Generate a code_challenge for a given token."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef add_receiver(self, receiver):\r\n if not callable(receiver):\r\n raise TypeError(\"receiver must be callable\")\r\n self.event_receivers.append(receiver)", "response": "Adds a receiver to the list of receivers."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nremoves a receiver from the list of receivers.", "response": "def remove_receiver(self, receiver):\r\n \"\"\" Remove a receiver to the list of receivers.\r\n\r\n :param receiver: a callable variable\r\n \"\"\"\r\n if receiver in self.event_receivers:\r\n self.event_receivers.remove(receiver)"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nsending a message to all event receivers that have been triggered and pass them the parameters.", "response": "def send_robust(self, **kwargs):\r\n \"\"\" Trigger all receiver and pass them the parameters\r\n If an exception is raised it will be catched and displayed as error\r\n in the logger (if defined).\r\n\r\n :param kwargs: all arguments from the event.\r\n \"\"\"\r\n for receiver in self.event_receivers:\r\n try:\r\n receiver(**kwargs)\r\n except Exception as err: # pylint: disable=W0703\r\n if not hasattr(err, '__traceback__'):\r\n LOGGER.error(sys.exc_info()[2])\r\n else:\r\n LOGGER.error(getattr(err, '__traceback__'))"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\ngets the app from cache or generate a new one if required", "response": "def __get_or_create_app(self, url, cache_key):\n \"\"\" Get the app from cache or generate a new one if required\n\n Because app object doesn't have etag/expiry, we have to make\n a head() call before, to have these informations first... \"\"\"\n headers = {\"Accept\": \"application/json\"}\n app_url = '%s?datasource=%s' % (url, self.datasource)\n\n cached = self.cache.get(cache_key, (None, None, 0))\n if cached is None or len(cached) != 3:\n self.cache.invalidate(cache_key)\n cached_app, cached_headers, cached_expiry = (cached, None, 0)\n else:\n cached_app, cached_headers, cached_expiry = cached\n\n if cached_app is not None and cached_headers is not None:\n # we didn't set custom expire, use header expiry\n expires = cached_headers.get('expires', None)\n cache_timeout = -1\n if self.expire is None and expires is not None:\n cache_timeout = get_cache_time_left(\n cached_headers['expires']\n )\n if cache_timeout >= 0:\n return cached_app\n\n # we set custom expire, check this instead\n else:\n if self.expire == 0 or cached_expiry >= time.time():\n return cached_app\n\n # if we have etags, add the header to use them\n etag = cached_headers.get('etag', None)\n if etag is not None:\n headers['If-None-Match'] = etag\n\n # if nothing makes us use the cache, invalidate it\n if ((expires is None or cache_timeout < 0 or\n cached_expiry < time.time()) and etag is None):\n self.cache.invalidate(cache_key)\n\n # set timeout value in case we have to cache it later\n timeout = 0\n if self.expire is not None and self.expire > 0:\n timeout = time.time() + self.expire\n\n # we are here, we know we have to make a head call...\n res = requests.head(app_url, headers=headers)\n if res.status_code == 304 and cached_app is not None:\n self.cache.set(\n cache_key,\n (cached_app, res.headers, timeout)\n )\n return cached_app\n\n # ok, cache is not accurate, make the full stuff\n app = App.create(app_url)\n if self.caching:\n self.cache.set(cache_key, (app, res.headers, timeout))\n\n return app"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\ninvalidate all cached endpoints meta included", "response": "def clear_cached_endpoints(self, prefix=None):\n \"\"\" Invalidate all cached endpoints, meta included\n\n Loop over all meta endpoints to generate all cache key the\n invalidate each of them. Doing it this way will prevent the\n app not finding keys as the user may change its prefixes\n Meta endpoint will be updated upon next call.\n :param: prefix the prefix for the cache key (default is cache_prefix)\n \"\"\"\n prefix = prefix if prefix is not None else self.cache_prefix\n for endpoint in self.app.op.values():\n cache_key = '%s:app:%s' % (prefix, endpoint.url)\n self.cache.invalidate(cache_key)\n self.cache.invalidate('%s:app:meta_swagger_url' % self.cache_prefix)\n self.app = None"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef __get_jwks_key(self, jwks_uri, **kwargs):\r\n jwks_key = kwargs.pop('jwks_key', None)\r\n if not jwks_key:\r\n res = self._session.get(jwks_uri)\r\n res.raise_for_status()\r\n jwks_key = res.json()\r\n\r\n self.jwks_key_set = None\r\n self.jwks_key = None\r\n if 'keys' in jwks_key:\r\n self.jwks_key_set = {}\r\n for jwks in jwks_key['keys']:\r\n self.jwks_key_set[jwks['kid']] = jwks\r\n else:\r\n self.jwks_key = jwks_key", "response": "Get from jwks_ui all the JWKS keys required to decode JWT Token."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nprepares the request parameters to execute the POST call to the OAuth token endpoint.", "response": "def __prepare_token_request(self, params, url=None):\r\n \"\"\"Generate the request parameters to execute the POST call to\r\n get or refresh a token.\r\n\r\n Parameters\r\n ----------\r\n params : dictionary\r\n Contains the parameters that will be given as data to the oauth\r\n endpoint\r\n\r\n Returns\r\n -------\r\n type\r\n The filled request parameters with all required informations\r\n\r\n \"\"\"\r\n if self.secret_key is None:\r\n params['code_verifier'] = self.code_verifier\r\n params['client_id'] = self.client_id\r\n\r\n request_params = {\r\n 'headers': self.__get_basic_auth_header(),\r\n 'data': params,\r\n 'url': self.oauth_token if url is None else url,\r\n }\r\n\r\n return request_params"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nconstruct the auth uri and returns it", "response": "def get_auth_uri(self, state, scopes=None, implicit=False):\r\n \"\"\"Constructs the full auth uri and returns it..\r\n\r\n Parameters\r\n ----------\r\n state : string\r\n The state to pass through the auth process\r\n scopes : list (optional)\r\n The list of scope to have\r\n implicit : Boolean (optional)\r\n Activate or not the implicit flow\r\n\r\n Returns\r\n -------\r\n String\r\n The generated URL for the user to log into EVE SSO\r\n\r\n \"\"\"\r\n if state is None or state == '':\r\n raise AttributeError('\"state\" must be non empty, non None string')\r\n\r\n scopes_list = [] if not scopes else scopes\r\n response_type = 'code' if not implicit else 'token'\r\n\r\n # generate the auth URI\r\n auth_uri = '%s?response_type=%s&redirect_uri=%s&client_id=%s%s%s' % (\r\n self.oauth_authorize,\r\n response_type,\r\n quote(self.redirect_uri, safe=''),\r\n self.client_id,\r\n '&scope=%s' % '+'.join(scopes_list) if scopes else '',\r\n '&state=%s' % state\r\n )\r\n\r\n # add code challenge if we have one\r\n if self.secret_key is None and not implicit:\r\n auth_uri += '&code_challenge_method=S256&code_challenge=%s' % (\r\n generate_code_challenge(self.code_verifier)\r\n )\r\n return auth_uri"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nreturning the params dict for the post request to get the access_token from the refresh_token", "response": "def get_refresh_token_params(self, scope_list=None):\r\n \"\"\" Return the param object for the post() call to get the access_token\r\n from the refresh_token\r\n\r\n :param code: the refresh token\r\n :return: a dict with the url, params and header\r\n \"\"\"\r\n if self.refresh_token is None:\r\n raise AttributeError('No refresh token is defined.')\r\n\r\n params = {\r\n 'grant_type': 'refresh_token',\r\n 'refresh_token': self.refresh_token,\r\n }\r\n\r\n if scope_list:\r\n if isinstance(scope_list, list):\r\n scopes = '+'.join(scope_list)\r\n else:\r\n raise AttributeError('scope_list must be a list of scope.')\r\n params['scope'] = scopes\r\n\r\n return self.__prepare_token_request(params)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nupdate the access_token refresh_token and token_expiry attributes from the response body.", "response": "def update_token(self, response_json, **kwargs):\r\n \"\"\" Update access_token, refresh_token and token_expiry from the\r\n response body.\r\n The response must be converted to a json object before being passed as\r\n a parameter\r\n\r\n :param response_json: the response body to use.\r\n :param token_identifier: the user identifier for the token\r\n \"\"\"\r\n self.token_identifier = kwargs.pop(\r\n 'token_identifier',\r\n self.token_identifier\r\n )\r\n self.access_token = response_json['access_token']\r\n self.token_expiry = int(time.time()) + response_json['expires_in']\r\n\r\n if 'refresh_token' in response_json:\r\n self.refresh_token = response_json['refresh_token']"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nreturn true if the token is expired.", "response": "def is_token_expired(self, offset=0):\r\n \"\"\" Return true if the token is expired.\r\n\r\n The offset can be used to change the expiry time:\r\n - positive value decrease the time (sooner)\r\n - negative value increase the time (later)\r\n If the expiry is not set, always return True. This case allow the users\r\n to define a security object, only knowing the refresh_token and get\r\n a new access_token / expiry_time without errors.\r\n\r\n :param offset: the expiry offset (in seconds) [default: 0]\r\n :return: boolean true if expired, else false.\r\n \"\"\"\r\n if self.token_expiry is None:\r\n return True\r\n return int(time.time()) >= (self.token_expiry - offset)"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nrefreshes the auth data using the refresh token in auth.", "response": "def refresh(self, scope_list=None):\r\n \"\"\" Update the auth data (tokens) using the refresh token in auth.\r\n \"\"\"\r\n request_data = self.get_refresh_token_params(scope_list)\r\n res = self._session.post(**request_data)\r\n if res.status_code != 200:\r\n raise APIException(\r\n request_data['url'],\r\n res.status_code,\r\n response=res.content,\r\n request_param=request_data,\r\n response_header=res.headers\r\n )\r\n json_res = res.json()\r\n self.update_token(json_res)\r\n return json_res"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nrequest the token to the auth endpoint.", "response": "def auth(self, code):\r\n \"\"\" Request the token to the /oauth/token endpoint.\r\n Update the security tokens.\r\n\r\n :param code: the code you get from the auth process\r\n \"\"\"\r\n request_data = self.get_access_token_params(code)\r\n\r\n res = self._session.post(**request_data)\r\n if res.status_code != 200:\r\n raise APIException(\r\n request_data['url'],\r\n res.status_code,\r\n response=res.content,\r\n request_param=request_data,\r\n response_header=res.headers\r\n )\r\n\r\n json_res = res.json()\r\n self.update_token(json_res)\r\n return json_res"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nrevoke the current tokens then empty all stored tokens", "response": "def revoke(self):\r\n \"\"\"Revoke the current tokens then empty all stored tokens\r\n This returns nothing since the endpoint return HTTP/200\r\n whatever the result is...\r\n\r\n Currently not working with JWT, left here for compatibility.\r\n \"\"\"\r\n if not self.refresh_token and not self.access_token:\r\n raise AttributeError('No access/refresh token are defined.')\r\n\r\n if self.refresh_token:\r\n data = {\r\n 'token_type_hint': 'refresh_token',\r\n 'token': self.refresh_token,\r\n }\r\n else:\r\n data = {\r\n 'token_type_hint': 'access_token',\r\n 'token': self.access_token,\r\n }\r\n\r\n request_data = self.__prepare_token_request(data, self.oauth_revoke)\r\n\r\n self._session.post(**request_data)\r\n self.access_token = None\r\n self.refresh_token = None\r\n self.token_expiry = None"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\ndecode and verify the token and return the decoded informations.", "response": "def verify(self, kid='JWT-Signature-Key', options=None):\r\n \"\"\"Decode and verify the token and return the decoded informations\r\n\r\n Parameters\r\n ----------\r\n kid : string\r\n The JWKS key id to identify the key to decode the token.\r\n Default is 'JWT-Signature-Key'. Only change if CCP changes it.\r\n options : Dict\r\n The dictionary of options for skipping validation steps. See\r\n https://python-jose.readthedocs.io/en/latest/jwt/api.html#jose.jwt.decode\r\n\r\n Returns\r\n -------\r\n Dict\r\n The JWT informations from the token, such as character name etc.\r\n\r\n Raises\r\n ------\r\n jose.exceptions.JWTError: If the signature is invalid in any way.\r\n jose.exceptions.ExpiredSignatureError: If the signature has expired\r\n jose.exceptions.JWTClaimsError: If any claim is invalid in any way.\r\n \"\"\"\r\n if self.access_token is None or self.access_token == \"\":\r\n raise AttributeError('No access token are available at this time')\r\n if options is None:\r\n options = {}\r\n\r\n if self.jwks_key_set is None:\r\n key = self.jwks_key\r\n else:\r\n key = self.jwks_key_set[kid]\r\n\r\n return jwt.decode(\r\n self.access_token,\r\n key,\r\n issuer=self.oauth_issuer,\r\n options=options\r\n )"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\ntries to send a request and retries the response.", "response": "def _retry_request(self, req_and_resp, _retry=0, **kwargs):\r\n \"\"\"Uses self._request in a sane retry loop (for 5xx level errors).\r\n\r\n Do not use the _retry parameter, use the same params as _request\r\n\r\n Used when ESIClient is initialized with retry_requests=True\r\n if raise_on_error is True, this will only raise exception after\r\n all retry have been done\r\n\r\n \"\"\"\r\n raise_on_error = kwargs.pop('raise_on_error', False)\r\n\r\n if _retry:\r\n # backoff delay loop in seconds: 0.01, 0.16, 0.81, 2.56, 6.25\r\n time.sleep(_retry ** 4 / 100)\r\n\r\n res = self._request(req_and_resp, **kwargs)\r\n\r\n if 500 <= res.status <= 599:\r\n _retry += 1\r\n if _retry < 5:\r\n LOGGER.warning(\r\n \"[failure #%d] %s %d: %r\",\r\n _retry,\r\n req_and_resp[0].url,\r\n res.status,\r\n res.data,\r\n )\r\n return self._retry_request(\r\n req_and_resp,\r\n _retry=_retry,\r\n raise_on_error=raise_on_error,\r\n **kwargs\r\n )\r\n\r\n if res.status >= 400 and raise_on_error:\r\n raise APIException(\r\n req_and_resp[0].url,\r\n res.status,\r\n response=res.raw,\r\n request_param=req_and_resp[0].query,\r\n response_header=res.header\r\n )\r\n\r\n return res"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef multi_request(self, reqs_and_resps, threads=20, **kwargs):\r\n\r\n opt = kwargs.pop('opt', {})\r\n raw_body_only = kwargs.pop('raw_body_only', self.raw_body_only)\r\n # you shouldnt need more than 100, 20 is probably fine in most cases\r\n threads = max(min(threads, 100), 1)\r\n\r\n def _multi_shim(req_and_resp):\r\n \"\"\"Shim self.request to also return the original request.\"\"\"\r\n\r\n return req_and_resp[0], self.request(\r\n req_and_resp,\r\n raw_body_only=raw_body_only,\r\n opt=opt,\r\n )\r\n\r\n results = []\r\n\r\n with ThreadPoolExecutor(max_workers=threads) as pool:\r\n for result in pool.map(_multi_shim, reqs_and_resps):\r\n results.append(result)\r\n\r\n return results", "response": "Use a threadpool to send multiple requests in parallel."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nmake a request and fill the response with the fields from the request and response.", "response": "def _request(self, req_and_resp, **kwargs):\r\n \"\"\" Take a request_and_response object from pyswagger.App and\r\n check auth, token, headers, prepare the actual request and fill the\r\n response\r\n\r\n Note on performance : if you need more performance (because you are\r\n using this in a batch) you'd rather set raw_body_only=True, as parsed\r\n body is really slow. You'll then have to get data from response.raw\r\n and convert it to json using \"json.loads(response.raw)\"\r\n\r\n :param req_and_resp: the request and response object from pyswagger.App\r\n :param raw_body_only: define if we want the body to be parsed as object\r\n instead of staying a raw dict. [Default: False]\r\n :param opt: options, see pyswagger/blob/master/pyswagger/io.py#L144\r\n :param raise_on_error: boolean to raise an error if HTTP Code >= 400\r\n\r\n :return: the final response.\r\n \"\"\"\r\n\r\n opt = kwargs.pop('opt', {})\r\n\r\n # reset the request and response to reuse existing req_and_resp\r\n req_and_resp[0].reset()\r\n req_and_resp[1].reset()\r\n\r\n # required because of inheritance\r\n request, response = super(EsiClient, self).request(req_and_resp, opt)\r\n\r\n # check cache here so we have all headers, formed url and params\r\n cache_key = make_cache_key(request)\r\n res = self.__make_request(request, opt, cache_key)\r\n\r\n if res.status_code == 200:\r\n self.__cache_response(cache_key, res, request.method.upper())\r\n\r\n # generate the Response object from requests response\r\n response.raw_body_only = kwargs.pop(\r\n 'raw_body_only',\r\n self.raw_body_only\r\n )\r\n\r\n try:\r\n response.apply_with(\r\n status=res.status_code,\r\n header=res.headers,\r\n raw=six.BytesIO(res.content).getvalue()\r\n )\r\n\r\n except ValueError:\r\n # catch JSONDecodeError/ValueError when response is not JSON\r\n raise APIException(\r\n request.url,\r\n res.status_code,\r\n response=res.content,\r\n request_param=request.query,\r\n response_header=res.headers\r\n )\r\n\r\n if 'warning' in res.headers:\r\n # send in logger and warnings, so the user doesn't have to use\r\n # logging to see it (at least once)\r\n LOGGER.warning(\"[%s] %s\", res.url, res.headers['warning'])\r\n warnings.warn(\"[%s] %s\" % (res.url, res.headers['warning']))\r\n\r\n if res.status_code >= 400 and kwargs.pop('raise_on_error', False):\r\n raise APIException(\r\n request.url,\r\n res.status_code,\r\n response=response.raw,\r\n request_param=request.query,\r\n response_header=response.header\r\n )\r\n\r\n return response"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nmakes a HEAD request and return a response object.", "response": "def head(self, req_and_resp, **kwargs):\r\n \"\"\" Take a request_and_response object from pyswagger.App, check\r\n and prepare everything to make a valid HEAD request\r\n\r\n :param req_and_resp: the request and response object from pyswagger.App\r\n :param opt: options, see pyswagger/blob/master/pyswagger/io.py#L144\r\n :param raise_on_error: boolean to raise an error if HTTP Code >= 400\r\n\r\n :return: the final response.\r\n \"\"\"\r\n\r\n opt = kwargs.pop('opt', {})\r\n\r\n # reset the request and response to reuse existing req_and_resp\r\n req_and_resp[0].reset()\r\n req_and_resp[1].reset()\r\n\r\n # required because of inheritance\r\n request, response = super(EsiClient, self).request(req_and_resp, opt)\r\n\r\n res = self.__make_request(request, opt, method='HEAD')\r\n\r\n response.apply_with(\r\n status=res.status_code,\r\n header=res.headers,\r\n raw=None,\r\n )\r\n\r\n if 'warning' in res.headers:\r\n # send in logger and warnings, so the user doesn't have to use\r\n # logging to see it (at least once)\r\n LOGGER.warning(\"[%s] %s\", res.url, res.headers['warning'])\r\n warnings.warn(\"[%s] %s\" % (res.url, res.headers['warning']))\r\n\r\n if res.status_code >= 400 and kwargs.pop('raise_on_error', False):\r\n raise APIException(\r\n request.url,\r\n res.status_code,\r\n response='',\r\n request_param=request.query,\r\n response_header=response.header\r\n )\r\n\r\n return response"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\ncache the response and check if it is expired", "response": "def __cache_response(self, cache_key, res, method):\r\n \"\"\" cache the response\r\n\r\n if method is one of self.__uncached_method__, don't cache anything\r\n \"\"\"\r\n if ('expires' in res.headers\r\n and method not in self.__uncached_methods__):\r\n cache_timeout = get_cache_time_left(res.headers.get('expires'))\r\n\r\n # Occasionally CCP swagger will return an outdated expire\r\n # warn and skip cache if timeout is <0\r\n if cache_timeout >= 0:\r\n self.cache.set(\r\n cache_key,\r\n CachedResponse(\r\n status_code=res.status_code,\r\n headers=res.headers,\r\n content=res.content,\r\n url=res.url,\r\n ),\r\n )\r\n else:\r\n LOGGER.warning(\r\n \"[%s] returned expired result: %s\", res.url,\r\n res.headers)\r\n warnings.warn(\"[%s] returned expired result\" % res.url)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nmake a new request and return the response or cached response.", "response": "def __make_request(self, request, opt, cache_key=None, method=None):\r\n \"\"\" Check cache, deal with expiration and etag, make the request and\r\n return the response or cached response.\r\n\r\n :param request: the pyswagger.io.Request object to prepare the request\r\n :param opt: options, see pyswagger/blob/master/pyswagger/io.py#L144\r\n :param cache_key: the cache key used for the cache stuff.\r\n :param method: [default:None] allows to force the method, especially\r\n useful if you want to make a HEAD request.\r\n Default value will use endpoint method\r\n\r\n \"\"\"\r\n # check expiration and etags\r\n opt_headers = {}\r\n cached_response = self.cache.get(cache_key, None)\r\n if cached_response is not None:\r\n # if we have expires cached, and still validd\r\n expires = cached_response.headers.get('expires', None)\r\n if expires is not None:\r\n cache_timeout = get_cache_time_left(\r\n cached_response.headers['expires']\r\n )\r\n if cache_timeout >= 0:\r\n return cached_response\r\n\r\n # if we have etags, add the header to use them\r\n etag = cached_response.headers.get('etag', None)\r\n if etag is not None:\r\n opt_headers['If-None-Match'] = etag\r\n\r\n # if nothing makes us use the cache, invalidate everything\r\n if (expires is None or cache_timeout < 0) and etag is None:\r\n self.cache.invalidate(cache_key)\r\n\r\n # apply request-related options before preparation.\r\n request.prepare(\r\n scheme=self.prepare_schemes(request).pop(),\r\n handle_files=False\r\n )\r\n request._patch(opt)\r\n\r\n # prepare the request and make it.\r\n method = method or request.method.upper()\r\n request.header.update(opt_headers)\r\n prepared_request = self._session.prepare_request(\r\n Request(\r\n method=method,\r\n url=request.url,\r\n params=request.query,\r\n data=request.data,\r\n headers=request.header\r\n )\r\n )\r\n start_api_call = time.time()\r\n\r\n try:\r\n res = self._session.send(\r\n prepared_request,\r\n timeout=self.timeout,\r\n )\r\n\r\n except (RequestsConnectionError, Timeout) as exc:\r\n # timeout issue, generate a fake response to finish the process\r\n # as a normal error 500\r\n res = CachedResponse(\r\n status_code=500,\r\n headers={},\r\n content=('{\"error\": \"%s\"}' % str(exc)).encode('latin-1'),\r\n url=prepared_request.url\r\n )\r\n\r\n # event for api call stats\r\n self.signal_api_call_stats.send(\r\n url=res.url,\r\n status_code=res.status_code,\r\n elapsed_time=time.time() - start_api_call,\r\n message=res.content if res.status_code != 200 else None\r\n )\r\n\r\n # if we have HTTP 304 (content didn't change), return the cached\r\n # response updated with the new headers\r\n if res.status_code == 304 and cached_response is not None:\r\n cached_response.headers['Expires'] = res.headers.get('Expires')\r\n cached_response.headers['Date'] = res.headers.get('Date')\r\n return cached_response\r\n return res"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef _hash(data):\r\n hash_algo = hashlib.new('md5')\r\n hash_algo.update(pickle.dumps(data))\r\n # prefix allows possibility of multiple applications\r\n # sharing same keyspace\r\n return 'esi_' + hash_algo.hexdigest()", "response": "generate a hash from data object to be used as cache key"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef fit_from_image(self, data, voxelsize, seeds, unique_cls):\n fvs, clsselected = self.features_from_image(data, voxelsize, seeds, unique_cls)\n self.fit(fvs, clsselected)", "response": "This Method computes feature vector and train model from the data."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nloading the current model from a file", "response": "def load(self, mdl_file):\n \"\"\"\n load model from file. fv_type is not set with this function. It is expected to set it before.\n \"\"\"\n import dill as pickle\n\n mdl_file_e = op.expanduser(mdl_file)\n\n sv = pickle.load(open(mdl_file_e, \"rb\"))\n self.mdl = sv[\"mdl\"]\n # self.mdl[2] = self.mdl[0]\n # try:\n # eval(sv['fv_extern_src'])\n # eval(\"fv_extern_temp_name = \" + sv['fv_extern_src_name'])\n # sv['fv_extern'] = fv_extern_temp_name\n # except:\n # print \"pomoc,necoje blbe\"\n # pass\n\n self.modelparams.update(sv[\"modelparams\"])\n logger.debug(\"loaded model from path: \" + mdl_file_e)"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nfunction to compute feature vectors from image data.", "response": "def features_from_image(\n self, data, voxelsize, seeds=None, unique_cls=None\n ): # , voxels=None):\n \"\"\"\n Input data is 3d image\n\n :param data: is 3d image\n :param seeds: ndimage with same shape as data, nonzero values means seeds.\n :param unique_cls: can select only fv for seeds from specific class.\n f.e. unique_cls = [1, 2] ignores label 0\n\n funcion is called twice in graph cut\n first call is with all params, second is only with data.\n\n based on self.modelparams['fv_type'] the feature vector is computed\n keywords \"intensity\", \"voxels\", \"fv001\", \"fv_extern\" can be used.\n modelparams['fv_type'] = 'fv_extern' allows to use external fv function\n\n Example of exter feature function. For easier implementation of return values use function return_fv_by_seeds().\n\n def fv_function(data, voxelsize, seeds=None, cl=None):\n data2 = scipy.ndimage.filters.gaussian_filter(data, sigma=5)\n arrs = [data.reshape(-1, 1), data2.reshape(-1, 1)]\n fv = np.concatenate(arrs, axis=1)\n return imcut.features.return_fv_by_seeds(fv, seeds, unique_cls)\n\n modelparams['fv_extern'] = fv_function\n \"\"\"\n\n fv_type = self.modelparams[\"fv_type\"]\n logger.debug(\"fv_type \" + fv_type)\n fv = []\n if fv_type == \"intensity\":\n fv = data.reshape(-1, 1)\n\n if seeds is not None:\n logger.debug(\"seeds: %s\", scipy.stats.describe(seeds, axis=None))\n sd = seeds.reshape(-1, 1)\n selection = np.in1d(sd, unique_cls)\n fv = fv[selection]\n sd = sd[selection]\n # sd = sd[]\n return fv, sd\n return fv\n\n # elif fv_type in (\"voxels\"):\n # if seeds is not None:\n # fv = np.asarray(voxels).reshape(-1, 1)\n # else:\n # fv = data\n # fv = fv.reshape(-1, 1)\n elif fv_type in (\"fv001\", \"FV001\", \"intensity_and_blur\"):\n\n # intensity in pixel, gaussian blur intensity\n return features.fv_function_intensity_and_smoothing(\n data, voxelsize, seeds, unique_cls\n )\n\n # from PyQt4.QtCore import pyqtRemoveInputHook\n # pyqtRemoveInputHook()\n\n # print fv1.shape\n # print fv2.shape\n # print fv.shape\n elif fv_type == \"fv_extern\":\n fv_function = self.modelparams[\"fv_extern\"]\n return fv_function(data, voxelsize, seeds, unique_cls)\n\n else:\n logger.error(\"Unknown feature vector type: \" + self.modelparams[\"fv_type\"])\n return fv"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nfitting one class in the database.", "response": "def _fit_one_class(self, clx, cl):\n \"\"\" Train clas number cl with data clx.\n\n Use trainFromImageAndSeeds() function if you want to use 3D image data\n as an input.\n\n clx: data, 2d matrix\n cl: label, integer\n\n label: gmmsame, gaussian_kde, dpgmm, stored\n \"\"\"\n\n logger.debug(\"clx \" + str(clx[:10, :]))\n logger.debug(\"clx type\" + str(clx.dtype))\n # name = 'clx' + str(cl) + '.npy'\n # print name\n # np.save(name, clx)\n logger.debug(\"_fit()\")\n if self.modelparams[\"adaptation\"] == \"original_data\":\n if cl in self.mdl.keys():\n return\n # if True:\n # return\n\n logger.debug(\"training continues\")\n\n if self.modelparams[\"type\"] == \"gmmsame\":\n if len(clx.shape) == 1:\n logger.warning(\n \"reshaping in train will be removed. Use \\\n \\ntrainFromImageAndSeeds() function\"\n )\n\n print(\"Warning deprecated feature in train() function\")\n # je to jen jednorozm\u011brn\u00fd vektor, tak je pot\u0159eba to\n # p\u0159ev\u00e9st na 2d matici\n clx = clx.reshape(-1, 1)\n gmmparams = self.modelparams[\"params\"]\n self.mdl[cl] = sklearn.mixture.GaussianMixture(**gmmparams)\n self.mdl[cl].fit(clx)\n\n elif self.modelparams[\"type\"] == \"kernel\":\n # Not working (probably) in old versions of scikits\n # from sklearn.neighbors.kde import KernelDensity\n from sklearn.neighbors import KernelDensity\n\n # kernelmodelparams = {'kernel': 'gaussian', 'bandwidth': 0.2}\n kernelmodelparams = self.modelparams[\"params\"]\n self.mdl[cl] = KernelDensity(**kernelmodelparams).fit(clx)\n elif self.modelparams[\"type\"] == \"gaussian_kde\":\n # print clx\n import scipy.stats\n\n # from PyQt4.QtCore import pyqtRemoveInputHook\n # pyqtRemoveInputHook()\n\n # gaussian_kde works only with floating point types\n self.mdl[cl] = scipy.stats.gaussian_kde(clx.astype(np.float))\n elif self.modelparams[\"type\"] == \"dpgmm\":\n # print 'clx.shape ', clx.shape\n # print 'cl ', cl\n gmmparams = self.modelparams[\"params\"]\n self.mdl[cl] = sklearn.mixture.DPGMM(**gmmparams)\n # todo here is a hack\n # dpgmm z n\u011bjak\u00e9ho d\u016fvodu nefunguje pro na\u0161e data\n # v\u017edy natr\u00e9nuje jednu slo\u017eku v bl\u00edzkosti nuly\n # patrn\u011b to bude m\u00edt n\u011bco spole\u010dn\u00e9ho s parametrem alpha\n # p\u0159en\u00e1sob\u00ed-li se to mal\u00fdm \u010d\u00edslem, z\u00e1zra\u010dn\u011b to chod\u00ed\n self.mdl[cl].fit(clx * 0.001)\n elif self.modelparams[\"type\"] == \"stored\":\n # Classifer is trained before segmentation and stored to pickle\n import pickle\n\n print(\"stored\")\n logger.warning(\"deprecated use of stored parameters\")\n\n mdl_file = self.modelparams[\"params\"][\"mdl_file\"]\n self.mdl = pickle.load(open(mdl_file, \"rb\"))\n\n else:\n raise NameError(\"Unknown model type\")"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef likelihood(self, x, cl):\n\n # sha = x.shape\n # xr = x.reshape(-1, sha[-1])\n # outsha = sha[:-1]\n # from PyQt4.QtCore import pyqtRemoveInputHook\n # pyqtRemoveInputHook()\n logger.debug(\"likel \" + str(x.shape))\n if self.modelparams[\"type\"] == \"gmmsame\":\n\n px = self.mdl[cl].score_samples(x)\n\n # todo o\u0161et\u0159it v\u00edce dimenzion\u00e1ln\u00ed fv\n # px = px.reshape(outsha)\n elif self.modelparams[\"type\"] == \"kernel\":\n px = self.mdl[cl].score_samples(x)\n elif self.modelparams[\"type\"] == \"gaussian_kde\":\n # print x\n # np.log because it is likelihood\n # @TODO Zde je patrn\u011b probl\u00e9m s reshape\n # old\n # px = np.log(self.mdl[cl](x.reshape(-1)))\n # new\n px = np.log(self.mdl[cl](x))\n # px = px.reshape(outsha)\n # from PyQt4.QtCore import pyqtRemoveInputHook\n # pyqtRemoveInputHook()\n elif self.modelparams[\"type\"] == \"dpgmm\":\n # todo here is a hack\n # dpgmm z n\u011bjak\u00e9ho d\u016fvodu nefunguje pro na\u0161e data\n # v\u017edy natr\u00e9nuje jednu slo\u017eku v bl\u00edzkosti nuly\n # patrn\u011b to bude m\u00edt n\u011bco spole\u010dn\u00e9ho s parametrem alpha\n # p\u0159en\u00e1sob\u00ed-li se to mal\u00fdm \u010d\u00edslem, z\u00e1zra\u010dn\u011b to chod\u00ed\n logger.warning(\".score() replaced with .score_samples() . Check it.\")\n # px = self.mdl[cl].score(x * 0.01)\n px = self.mdl[cl].score_samples(x * 0.01)\n elif self.modelparams[\"type\"] == \"stored\":\n px = self.mdl[cl].score(x)\n return px", "response": "likelihood from data x with fature vector with 4 fatures"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\ngets information about one node in graph", "response": "def inspect_node_neighborhood(nlinks, msinds, node_msindex):\n \"\"\"\n Get information about one node in graph\n\n :param nlinks: neighboorhood edges\n :param msinds: indexes in 3d image\n :param node_msindex: int, multiscale index of selected voxel\n :return: node_neighboor_edges_and_weights, node_neighboor_seeds\n \"\"\"\n # seed_indexes = np.nonzero(node_seed)\n # selected_inds = msinds[seed_indexes]\n # node_msindex = selected_inds[0]\n\n node_neighbor_edges = get_neighborhood_edes(nlinks, node_msindex)\n\n node_neighbor_seeds = np.zeros_like(msinds, dtype=np.int8)\n for neighboor_ind in np.unique(node_neighbor_edges[:, :2].ravel()):\n node_neighbor_ind = np.where(msinds == neighboor_ind)\n node_neighbor_seeds[node_neighbor_ind] = 2\n\n node_neighbor_seeds[np.where(msinds == node_msindex)] = 1\n\n # node_coordinates = np.unravel_index(selected_voxel_ind, msinds.shape)\n # node_neighbor_coordinates = np.unravel_index(np.unique(node_neighbor_edges[:, :2].ravel()), msinds.shape)\n return node_neighbor_edges, node_neighbor_seeds"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef inspect_node(nlinks, unariesalt, msinds, node_msindex):\n\n node_unariesalt = unariesalt[node_msindex]\n neigh_edges, neigh_seeds = inspect_node_neighborhood(nlinks, msinds, node_msindex)\n return node_unariesalt, neigh_edges, neigh_seeds", "response": "Inspects one node in graph and returns unariesalt edges and weights for each node in graph"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef get_node_msindex(msinds, node_seed):\n if type(node_seed) == np.ndarray:\n seed_indexes = np.nonzero(node_seed)\n elif type(node_seed) == list:\n seed_indexes = node_seed\n else:\n seed_indexes = [node_seed]\n\n selected_nodes_msinds = msinds[seed_indexes]\n node_msindex = selected_nodes_msinds[0]\n return node_msindex", "response": "Convert seeds - like selection of voxel to multiscale index."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef relabel_squeeze(data):\n palette, index = np.unique(data, return_inverse=True)\n data = index.reshape(data.shape)\n # realy slow solution\n # unq = np.unique(data)\n # actual_label = 0\n # for lab in unq:\n # data[data == lab] = actual_label\n # actual_label += 1\n\n # one another solution probably slower\n # arr = data\n # data = (np.digitize(arr.reshape(-1,),np.unique(arr))-1).reshape(arr.shape)\n\n return data", "response": "Makes relabeling of data if there are unused values."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef load(self, filename, fv_extern=None):\n self.modelparams[\"mdl_stored_file\"] = filename\n if fv_extern is not None:\n self.modelparams[\"fv_extern\"] = fv_extern\n\n # segparams['modelparams'] = {\n # 'mdl_stored_file': mdl_stored_file,\n # # 'fv_extern': fv_function\n # }\n self.mdl = Model(modelparams=self.modelparams)", "response": "Load the model stored in the file."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nget the segmentation and the :return:", "response": "def __msgc_step12_low_resolution_segmentation(self):\n \"\"\"\n Get the segmentation and the\n :return:\n \"\"\"\n import scipy\n\n start = self._start_time\n # ===== low resolution data processing\n # default parameters\n # TODO segparams_lo and segparams_hi je tam asi zbytecn\u011b\n sparams_lo = {\n \"boundary_dilatation_distance\": 2,\n \"block_size\": 6,\n \"use_boundary_penalties\": True,\n \"boundary_penalties_weight\": 1,\n \"tile_zoom_constant\": 1,\n }\n\n sparams_lo.update(self.segparams)\n sparams_hi = copy.copy(sparams_lo)\n # sparams_lo['boundary_penalties_weight'] = (\n # sparams_lo['boundary_penalties_weight'] *\n # sparams_lo['block_size'])\n self.segparams = sparams_lo\n\n self.stats[\"t1\"] = time.time() - start\n # step 1: low res GC\n hiseeds = self.seeds\n # ms_zoom = 4 # 0.125 #self.segparams['scale']\n # ms_zoom = self.segparams['block_size']\n # loseeds = pyed.getSeeds()\n # logger.debug(\"msc \" + str(np.unique(hiseeds)))\n loseeds = seed_zoom(hiseeds, self.segparams[\"block_size\"])\n\n hard_constraints = True\n\n self.seeds = loseeds\n\n modelparams_hi = self.modelparams.copy()\n # feature vector will be computed from selected voxels\n self.modelparams[\"use_extra_features_for_training\"] = True\n\n # TODO what with voxels? It is used from here\n # hiseeds and hiimage is used to create intensity model\n self.voxels1 = self.img[hiseeds == 1].reshape(-1, 1)\n self.voxels2 = self.img[hiseeds == 2].reshape(-1, 1)\n # this is how to compute with loseeds resolution but in wrong way\n # self.voxels1 = self.img[self.seeds == 1]\n # self.voxels2 = self.img[self.seeds == 2]\n\n # self.voxels1 = pyed.getSeedsVal(1)\n # self.voxels2 = pyed.getSeedsVal(2)\n\n img_orig = self.img\n\n # TODO this should be done with resize_to_shape_whith_zoom\n zoom = np.asarray(loseeds.shape).astype(np.float) / img_orig.shape\n self.img = scipy.ndimage.interpolation.zoom(img_orig, zoom, order=0)\n voxelsize_orig = self.voxelsize\n logger.debug(\"zoom %s\", zoom)\n logger.debug(\"vs %s\", self.voxelsize)\n self.voxelsize = self.voxelsize * zoom\n\n # self.img = resize_to_shape_with_zoom(img_orig, loseeds.shape, 1.0 / ms_zoom, order=0)\n\n # this step set the self.segmentation\n self.__single_scale_gc_run()\n # logger.debug(\n # 'segmentation - max: %d min: %d' % (\n # np.max(self.segmentation),\n # np.min(self.segmentation)\n # )\n # )\n logger.debug(\n \"segmentation: %s\", scipy.stats.describe(self.segmentation, axis=None)\n )\n\n self.modelparams = modelparams_hi\n self.voxelsize = voxelsize_orig\n self.img = img_orig\n self.seeds = hiseeds\n self.stats[\"t2\"] = time.time() - start\n return hard_constraints"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nestimate discontinuity in basis of low resolution image segmentation. :return: discontinuity in low resolution", "response": "def __msgc_step3_discontinuity_localization(self):\n \"\"\"\n Estimate discontinuity in basis of low resolution image segmentation.\n :return: discontinuity in low resolution\n \"\"\"\n import scipy\n\n start = self._start_time\n seg = 1 - self.segmentation.astype(np.int8)\n self.stats[\"low level object voxels\"] = np.sum(seg)\n self.stats[\"low level image voxels\"] = np.prod(seg.shape)\n # in seg is now stored low resolution segmentation\n # back to normal parameters\n # step 2: discontinuity localization\n # self.segparams = sparams_hi\n seg_border = scipy.ndimage.filters.laplace(seg, mode=\"constant\")\n logger.debug(\"seg_border: %s\", scipy.stats.describe(seg_border, axis=None))\n # logger.debug(str(np.max(seg_border)))\n # logger.debug(str(np.min(seg_border)))\n seg_border[seg_border != 0] = 1\n logger.debug(\"seg_border: %s\", scipy.stats.describe(seg_border, axis=None))\n # scipy.ndimage.morphology.distance_transform_edt\n boundary_dilatation_distance = self.segparams[\"boundary_dilatation_distance\"]\n seg = scipy.ndimage.morphology.binary_dilation(\n seg_border,\n # seg,\n np.ones(\n [\n (boundary_dilatation_distance * 2) + 1,\n (boundary_dilatation_distance * 2) + 1,\n (boundary_dilatation_distance * 2) + 1,\n ]\n ),\n )\n if self.keep_temp_properties:\n self.temp_msgc_lowres_discontinuity = seg\n else:\n self.temp_msgc_lowres_discontinuity = None\n\n if self.debug_images:\n import sed3\n\n pd = sed3.sed3(seg_border) # ), contour=seg)\n pd.show()\n pd = sed3.sed3(seg) # ), contour=seg)\n pd.show()\n # segzoom = scipy.ndimage.interpolation.zoom(seg.astype('float'), zoom,\n # order=0).astype('int8')\n self.stats[\"t3\"] = time.time() - start\n return seg"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef __multiscale_gc_lo2hi_run(self): # , pyed):\n # from PyQt4.QtCore import pyqtRemoveInputHook\n # pyqtRemoveInputHook()\n self._msgc_lo2hi_resize_init()\n self.__msgc_step0_init()\n\n hard_constraints = self.__msgc_step12_low_resolution_segmentation()\n # ===== high resolution data processing\n seg = self.__msgc_step3_discontinuity_localization()\n\n self.stats[\"t3.1\"] = (time.time() - self._start_time)\n graph = Graph(\n seg,\n voxelsize=self.voxelsize,\n nsplit=self.segparams[\"block_size\"],\n edge_weight_table=self._msgc_npenalty_table,\n compute_low_nodes_index=True,\n )\n\n # graph.run() = graph.generate_base_grid() + graph.split_voxels()\n # graph.run()\n graph.generate_base_grid()\n self.stats[\"t3.2\"] = (time.time() - self._start_time)\n graph.split_voxels()\n\n self.stats[\"t3.3\"] = (time.time() - self._start_time)\n\n self.stats.update(graph.stats)\n self.stats[\"t4\"] = (time.time() - self._start_time)\n mul_mask, mul_val = self.__msgc_tlinks_area_weight_from_low_segmentation(seg)\n area_weight = 1\n unariesalt = self.__create_tlinks(\n self.img,\n self.voxelsize,\n self.seeds,\n area_weight=area_weight,\n hard_constraints=hard_constraints,\n mul_mask=None,\n mul_val=None,\n )\n # N-links prepared\n self.stats[\"t5\"] = (time.time() - self._start_time)\n un, ind = np.unique(graph.msinds, return_index=True)\n self.stats[\"t6\"] = (time.time() - self._start_time)\n\n self.stats[\"t7\"] = (time.time() - self._start_time)\n unariesalt2_lo2hi = np.hstack(\n [unariesalt[ind, 0, 0].reshape(-1, 1), unariesalt[ind, 0, 1].reshape(-1, 1)]\n )\n nlinks_lo2hi = np.hstack([graph.edges, graph.edges_weights.reshape(-1, 1)])\n if self.debug_images:\n import sed3\n\n ed = sed3.sed3(unariesalt[:, :, 0].reshape(self.img.shape))\n ed.show()\n import sed3\n\n ed = sed3.sed3(unariesalt[:, :, 1].reshape(self.img.shape))\n ed.show()\n # ed = sed3.sed3(seg)\n # ed.show()\n # import sed3\n # ed = sed3.sed3(graph.data)\n # ed.show()\n # import sed3\n # ed = sed3.sed3(graph.msinds)\n # ed.show()\n\n # nlinks, unariesalt2, msinds = self.__msgc_step45678_construct_graph(area_weight, hard_constraints, seg)\n # self.__msgc_step9_finish_perform_gc_and_reshape(nlinks, unariesalt2, msinds)\n self.__msgc_step9_finish_perform_gc_and_reshape(\n nlinks_lo2hi, unariesalt2_lo2hi, graph.msinds\n )\n self._msgc_lo2hi_resize_clean_finish()", "response": "Run Graph - Cut segmentation with refinement of low resolution multiscale graph."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef __multiscale_gc_hi2lo_run(self): # , pyed):\n # from PyQt4.QtCore import pyqtRemoveInputHook\n # pyqtRemoveInputHook()\n\n self.__msgc_step0_init()\n hard_constraints = self.__msgc_step12_low_resolution_segmentation()\n # ===== high resolution data processing\n seg = self.__msgc_step3_discontinuity_localization()\n nlinks, unariesalt2, msinds = self.__msgc_step45678_hi2lo_construct_graph(\n hard_constraints, seg\n )\n self.__msgc_step9_finish_perform_gc_and_reshape(nlinks, unariesalt2, msinds)", "response": "Run Graph - Cut segmentation with simplifiyng of high resolution multiscale graph."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef __ordered_values_by_indexes(self, data, inds):\n # get unique labels and their first indexes\n # lab, linds = np.unique(inds, return_index=True)\n # compute values by indexes\n # values = data.reshape(-1)[linds]\n\n # alternative slow implementation\n # if there are different data on same index, it will take\n # maximal value\n # lab = np.unique(inds)\n # values = [0]*len(lab)\n # for label in lab:\n # values[label] = np.max(data[inds == label])\n #\n # values = np.asarray(values)\n\n # yet another implementation\n values = [None] * (np.max(inds) + 1)\n\n linear_inds = inds.ravel()\n linear_data = data.ravel()\n for i in range(0, len(linear_inds)):\n # going over all data pixels\n\n if values[linear_inds[i]] is None:\n # this index is found for first\n values[linear_inds[i]] = linear_data[i]\n elif values[linear_inds[i]] < linear_data[i]:\n # here can be changed maximal or minimal value\n values[linear_inds[i]] = linear_data[i]\n\n values = np.asarray(values)\n\n return values", "response": "Return values by indexes."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nfunctioning computes multiscale indexes of ndarray.", "response": "def __hi2lo_multiscale_indexes(self, mask, orig_shape): # , zoom):\n \"\"\"\n Function computes multiscale indexes of ndarray.\n\n mask: Says where is original resolution (0) and where is small\n resolution (1). Mask is in small resolution.\n\n orig_shape: Original shape of input data.\n zoom: Usually number greater then 1\n\n result = [[0 1 2],\n [3 4 4],\n [5 4 4]]\n \"\"\"\n\n mask_orig = zoom_to_shape(mask, orig_shape, dtype=np.int8)\n\n inds_small = np.arange(mask.size).reshape(mask.shape)\n inds_small_in_orig = zoom_to_shape(inds_small, orig_shape, dtype=np.int8)\n inds_orig = np.arange(np.prod(orig_shape)).reshape(orig_shape)\n\n # inds_orig = inds_orig * mask_orig\n inds_orig += np.max(inds_small_in_orig) + 1\n # print 'indexes'\n # import py3DSeedEditor as ped\n # import pdb; pdb.set_trace() # BREAKPOINT\n\n # '==' is not the same as 'is' for numpy.array\n inds_small_in_orig[mask_orig == True] = inds_orig[mask_orig == True] # noqa\n inds = inds_small_in_orig\n # print np.max(inds)\n # print np.min(inds)\n inds = relabel_squeeze(inds)\n logger.debug(\n \"Index after relabeling: %s\", scipy.stats.describe(inds, axis=None)\n )\n # logger.debug(\"Minimal index after relabeling: \" + str(np.min(inds)))\n # inds_orig[mask_orig==True] = 0\n # inds_small_in_orig[mask_orig==False] = 0\n # inds = (inds_orig + np.max(inds_small_in_orig) + 1) + inds_small_in_orig\n\n return inds, mask_orig"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef interactivity(self, min_val=None, max_val=None, qt_app=None):\n from .seed_editor_qt import QTSeedEditor\n from PyQt4.QtGui import QApplication\n\n if min_val is None:\n min_val = np.min(self.img)\n\n if max_val is None:\n max_val = np.max(self.img)\n\n window_c = (max_val + min_val) / 2 # .astype(np.int16)\n window_w = max_val - min_val # .astype(np.int16)\n\n if qt_app is None:\n qt_app = QApplication(sys.argv)\n\n pyed = QTSeedEditor(\n self.img,\n modeFun=self.interactivity_loop,\n voxelSize=self.voxelsize,\n seeds=self.seeds,\n volume_unit=self.volume_unit,\n )\n\n pyed.changeC(window_c)\n pyed.changeW(window_w)\n\n qt_app.exec_()", "response": "Interactivity setting with 3d seed editor"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nfunctioning for manual seed setting. Sets variable seeds and prepares voxels for density model.", "response": "def set_seeds(self, seeds):\n \"\"\"\n Function for manual seed setting. Sets variable seeds and prepares\n voxels for density model.\n :param seeds: ndarray (0 - nothing, 1 - object, 2 - background,\n 3 - object just hard constraints, no model training, 4 - background \n just hard constraints, no model training)\n \"\"\"\n if self.img.shape != seeds.shape:\n raise Exception(\"Seeds must be same size as input image\")\n\n self.seeds = seeds.astype(\"int8\")\n self.voxels1 = self.img[self.seeds == 1]\n self.voxels2 = self.img[self.seeds == 2]"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef run(self, run_fit_model=True):\n\n if run_fit_model:\n self.fit_model(self.img, self.voxelsize, self.seeds)\n\n self._start_time = time.time()\n if self.segparams[\"method\"].lower() in (\"graphcut\", \"gc\"):\n self.__single_scale_gc_run()\n elif self.segparams[\"method\"].lower() in (\n \"multiscale_graphcut\",\n \"multiscale_gc\",\n \"msgc\",\n \"msgc_lo2hi\",\n \"lo2hi\",\n \"multiscale_graphcut_lo2hi\",\n ):\n logger.debug(\"performing multiscale Graph-Cut lo2hi\")\n self.__multiscale_gc_lo2hi_run()\n elif self.segparams[\"method\"].lower() in (\n \"msgc_hi2lo\",\n \"hi2lo\",\n \"multiscale_graphcut_hi2lo\",\n ):\n logger.debug(\"performing multiscale Graph-Cut hi2lo\")\n self.__multiscale_gc_hi2lo_run()\n else:\n logger.error(\"Unknown segmentation method: \" + self.segparams[\"method\"])", "response": "Run the Graph Cut segmentation according to preset parameters."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nsetting the hard constraints for the tdata1 and tdata2", "response": "def __set_hard_hard_constraints(self, tdata1, tdata2, seeds):\n \"\"\"\n it works with seed labels:\n 0: nothing\n 1: object 1 - full seeds\n 2: object 2 - full seeds\n 3: object 1 - not a training seeds\n 4: object 2 - not a training seeds\n \"\"\"\n seeds_mask = (seeds == 1) | (seeds == 3)\n tdata2[seeds_mask] = np.max(tdata2) + 1\n tdata1[seeds_mask] = 0\n\n seeds_mask = (seeds == 2) | (seeds == 4)\n tdata1[seeds_mask] = np.max(tdata1) + 1\n tdata2[seeds_mask] = 0\n\n return tdata1, tdata2"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef __similarity_for_tlinks_obj_bgr(\n self,\n data,\n voxelsize,\n # voxels1, voxels2,\n # seeds, otherfeatures=None\n ):\n \"\"\"\n Compute edge values for graph cut tlinks based on image intensity\n and texture.\n \"\"\"\n # self.fit_model(data, voxelsize, seeds)\n # There is a need to have small vaues for good fit\n # R(obj) = -ln( Pr (Ip | O) )\n # R(bck) = -ln( Pr (Ip | B) )\n # Boykov2001b\n # ln is computed in likelihood\n tdata1 = (-(self.mdl.likelihood_from_image(data, voxelsize, 1))) * 10\n tdata2 = (-(self.mdl.likelihood_from_image(data, voxelsize, 2))) * 10\n\n # to spare some memory\n dtype = np.int16\n if np.any(tdata1 > 32760):\n dtype = np.float32\n if np.any(tdata2 > 32760):\n dtype = np.float32\n\n if self.segparams[\"use_apriori_if_available\"] and self.apriori is not None:\n logger.debug(\"using apriori information\")\n gamma = self.segparams[\"apriori_gamma\"]\n a1 = (-np.log(self.apriori * 0.998 + 0.001)) * 10\n a2 = (-np.log(0.999 - (self.apriori * 0.998))) * 10\n # logger.debug('max ' + str(np.max(tdata1)) + ' min ' + str(np.min(tdata1)))\n # logger.debug('max ' + str(np.max(tdata2)) + ' min ' + str(np.min(tdata2)))\n # logger.debug('max ' + str(np.max(a1)) + ' min ' + str(np.min(a1)))\n # logger.debug('max ' + str(np.max(a2)) + ' min ' + str(np.min(a2)))\n tdata1u = (((1 - gamma) * tdata1) + (gamma * a1)).astype(dtype)\n tdata2u = (((1 - gamma) * tdata2) + (gamma * a2)).astype(dtype)\n tdata1 = tdata1u\n tdata2 = tdata2u\n # logger.debug(' max ' + str(np.max(tdata1)) + ' min ' + str(np.min(tdata1)))\n # logger.debug(' max ' + str(np.max(tdata2)) + ' min ' + str(np.min(tdata2)))\n # logger.debug('gamma ' + str(gamma))\n\n # import sed3\n # ed = sed3.show_slices(tdata1)\n # ed = sed3.show_slices(tdata2)\n del tdata1u\n del tdata2u\n del a1\n del a2\n\n # if np.any(tdata1 < 0) or np.any(tdata2 <0):\n # logger.error(\"Problem with tlinks. Likelihood is < 0\")\n\n # if self.debug_images:\n # self.__show_debug_tdata_images(tdata1, tdata2, suptitle=\"likelihood\")\n return tdata1, tdata2", "response": "Compute edge values for tlinks based on image intensity\n voxels1 voxels2 and otherfeatures."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef __create_nlinks(self, data, inds=None, boundary_penalties_fcn=None):\n # use the gerneral graph algorithm\n # first, we construct the grid graph\n start = time.time()\n if inds is None:\n inds = np.arange(data.size).reshape(data.shape)\n # if not self.segparams['use_boundary_penalties'] and \\\n # boundary_penalties_fcn is None :\n if boundary_penalties_fcn is None:\n # This is faster for some specific format\n edgx = np.c_[inds[:, :, :-1].ravel(), inds[:, :, 1:].ravel()]\n edgy = np.c_[inds[:, :-1, :].ravel(), inds[:, 1:, :].ravel()]\n edgz = np.c_[inds[:-1, :, :].ravel(), inds[1:, :, :].ravel()]\n\n else:\n logger.info(\"use_boundary_penalties\")\n\n bpw = self.segparams[\"boundary_penalties_weight\"]\n\n bpa = boundary_penalties_fcn(2)\n # id1=inds[:, :, :-1].ravel()\n edgx = np.c_[\n inds[:, :, :-1].ravel(),\n inds[:, :, 1:].ravel(),\n # cc * np.ones(id1.shape)\n bpw * bpa[:, :, 1:].ravel(),\n ]\n\n bpa = boundary_penalties_fcn(1)\n # id1 =inds[:, 1:, :].ravel()\n edgy = np.c_[\n inds[:, :-1, :].ravel(),\n inds[:, 1:, :].ravel(),\n # cc * np.ones(id1.shape)]\n bpw * bpa[:, 1:, :].ravel(),\n ]\n\n bpa = boundary_penalties_fcn(0)\n # id1 = inds[1:, :, :].ravel()\n edgz = np.c_[\n inds[:-1, :, :].ravel(),\n inds[1:, :, :].ravel(),\n # cc * np.ones(id1.shape)]\n bpw * bpa[1:, :, :].ravel(),\n ]\n\n # import pdb; pdb.set_trace()\n edges = np.vstack([edgx, edgy, edgz]).astype(np.int32)\n # edges - seznam indexu hran, kteres spolu sousedi\\\n elapsed = time.time() - start\n self.stats[\"_create_nlinks time\"] = elapsed\n logger.info(\"__create nlinks time \" + str(elapsed))\n return edges", "response": "Compute nlinks grid from data shape information."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nusing actual model to calculate similarity. If no input is given the last image is used. :param data3d: :param voxelsize: :param seeds: :param area_weight: :param hard_constraints: :param return_unariesalt: :return:", "response": "def debug_get_reconstructed_similarity(\n self,\n data3d=None,\n voxelsize=None,\n seeds=None,\n area_weight=1,\n hard_constraints=True,\n return_unariesalt=False,\n ):\n \"\"\"\n Use actual model to calculate similarity. If no input is given the last image is used.\n :param data3d:\n :param voxelsize:\n :param seeds:\n :param area_weight:\n :param hard_constraints:\n :param return_unariesalt:\n :return:\n \"\"\"\n if data3d is None:\n data3d = self.img\n if voxelsize is None:\n voxelsize = self.voxelsize\n if seeds is None:\n seeds = self.seeds\n\n unariesalt = self.__create_tlinks(\n data3d,\n voxelsize,\n # voxels1, voxels2,\n seeds,\n area_weight,\n hard_constraints,\n )\n if return_unariesalt:\n return unariesalt\n else:\n return self._reshape_unariesalt_to_similarity(unariesalt, data3d.shape)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef debug_show_reconstructed_similarity(\n self,\n data3d=None,\n voxelsize=None,\n seeds=None,\n area_weight=1,\n hard_constraints=True,\n show=True,\n bins=20,\n slice_number=None,\n ):\n \"\"\"\n Show tlinks.\n :param data3d: ndarray with input data\n :param voxelsize:\n :param seeds:\n :param area_weight:\n :param hard_constraints:\n :param show:\n :param bins: histogram bins number\n :param slice_number:\n :return:\n \"\"\"\n\n unariesalt = self.debug_get_reconstructed_similarity(\n data3d,\n voxelsize=voxelsize,\n seeds=seeds,\n area_weight=area_weight,\n hard_constraints=hard_constraints,\n return_unariesalt=True,\n )\n\n self._debug_show_unariesalt(\n unariesalt, show=show, bins=bins, slice_number=slice_number\n )", "response": "Show the reconstructed similarity of a set of tlinks."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef debug_inspect_node(self, node_msindex):\n return inspect_node(self.nlinks, self.unariesalt2, self.msinds, node_msindex)", "response": "Get info about the node. See pycut. inspect_node for details."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef debug_interactive_inspect_node(self):\n if (\n np.sum(\n np.abs(\n np.asarray(self.msinds.shape) - np.asarray(self.segmentation.shape)\n )\n )\n == 0\n ):\n segmentation = self.segmentation\n else:\n segmentation = self.temp_msgc_resized_segmentation\n\n logger.info(\"Click to select one voxel of interest\")\n import sed3\n\n ed = sed3.sed3(self.msinds, contour=segmentation == 0)\n ed.show()\n edseeds = ed.seeds\n node_msindex = get_node_msindex(self.msinds, edseeds)\n\n node_unariesalt, node_neighboor_edges_and_weights, node_neighboor_seeds = self.debug_inspect_node(\n node_msindex\n )\n import sed3\n\n ed = sed3.sed3(\n self.msinds, contour=segmentation == 0, seeds=node_neighboor_seeds\n )\n ed.show()\n\n return (\n node_unariesalt,\n node_neighboor_edges_and_weights,\n node_neighboor_seeds,\n node_msindex,\n )", "response": "Call after segmentation to see selected node neighborhood."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\npreparing data and run computation", "response": "def _ssgc_prepare_data_and_run_computation(\n self,\n # voxels1, voxels2,\n hard_constraints=True,\n area_weight=1,\n ):\n \"\"\"\n Setting of data.\n You need set seeds if you want use hard_constraints.\n \"\"\"\n # from PyQt4.QtCore import pyqtRemoveInputHook\n # pyqtRemoveInputHook()\n # import pdb; pdb.set_trace() # BREAKPOINT\n\n unariesalt = self.__create_tlinks(\n self.img,\n self.voxelsize,\n # voxels1, voxels2,\n self.seeds,\n area_weight,\n hard_constraints,\n )\n # n\u011bkter\u00e9m testu organ semgmentation dosahuj\u00ed unaries -15. co\u017e je podin\u00e9\n # sta\u010d\u00ed vyhodit print p\u0159ed if a je to vid\u011bt\n logger.debug(\"unaries %.3g , %.3g\" % (np.max(unariesalt), np.min(unariesalt)))\n # create potts pairwise\n # pairwiseAlpha = -10\n pairwise = -(np.eye(2) - 1)\n pairwise = (self.segparams[\"pairwise_alpha\"] * pairwise).astype(np.int32)\n # pairwise = np.array([[0,30],[30,0]]).astype(np.int32)\n # print pairwise\n\n self.iparams = {}\n\n if self.segparams[\"use_boundary_penalties\"]:\n sigma = self.segparams[\"boundary_penalties_sigma\"]\n # set boundary penalties function\n # Default are penalties based on intensity differences\n boundary_penalties_fcn = lambda ax: self._boundary_penalties_array(\n axis=ax, sigma=sigma\n )\n else:\n boundary_penalties_fcn = None\n nlinks = self.__create_nlinks(\n self.img, boundary_penalties_fcn=boundary_penalties_fcn\n )\n\n self.stats[\"tlinks shape\"].append(unariesalt.reshape(-1, 2).shape)\n self.stats[\"nlinks shape\"].append(nlinks.shape)\n # we flatten the unaries\n # result_graph = cut_from_graph(nlinks, unaries.reshape(-1, 2),\n # pairwise)\n start = time.time()\n if self.debug_images:\n self._debug_show_unariesalt(unariesalt)\n result_graph = pygco.cut_from_graph(nlinks, unariesalt.reshape(-1, 2), pairwise)\n elapsed = time.time() - start\n self.stats[\"gc time\"] = elapsed\n result_labeling = result_graph.reshape(self.img.shape)\n\n return result_labeling"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nfunction resize input data to specific shape.", "response": "def resize_to_shape(data, shape, zoom=None, mode=\"nearest\", order=0):\n \"\"\"\n Function resize input data to specific shape.\n :param data: input 3d array-like data\n :param shape: shape of output data\n :param zoom: zoom is used for back compatibility\n :mode: default is 'nearest'\n \"\"\"\n # @TODO remove old code in except part\n # TODO use function from library in future\n\n try:\n # rint 'pred vyjimkou'\n # aise Exception ('test without skimage')\n # rint 'za vyjimkou'\n import skimage\n import skimage.transform\n\n # Now we need reshape seeds and segmentation to original size\n\n # with warnings.catch_warnings():\n # warnings.filterwarnings(\"ignore\", \".*'constant', will be changed to.*\")\n segm_orig_scale = skimage.transform.resize(\n data, shape, order=0, preserve_range=True, mode=\"reflect\"\n )\n\n segmentation = segm_orig_scale\n logger.debug(\"resize to orig with skimage\")\n except:\n if zoom is None:\n zoom = shape / np.asarray(data.shape).astype(np.double)\n segmentation = resize_to_shape_with_zoom(\n data, zoom=zoom, mode=mode, order=order\n )\n\n return segmentation"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef seed_zoom(seeds, zoom):\n # import scipy\n # loseeds=seeds\n labels = np.unique(seeds)\n # remove first label - 0\n labels = np.delete(labels, 0)\n # @TODO smart interpolation for seeds in one block\n # loseeds = scipy.ndimage.interpolation.zoom(\n # seeds, zoom, order=0)\n loshape = np.ceil(np.array(seeds.shape) * 1.0 / zoom).astype(np.int)\n loseeds = np.zeros(loshape, dtype=np.int8)\n loseeds = loseeds.astype(np.int8)\n for label in labels:\n a, b, c = np.where(seeds == label)\n loa = np.round(a // zoom)\n lob = np.round(b // zoom)\n loc = np.round(c // zoom)\n # loseeds = np.zeros(loshape)\n\n loseeds[loa, lob, loc] += label\n # this is to detect conflict seeds\n loseeds[loseeds > label] = 100\n\n # remove conflict seeds\n loseeds[loseeds > 99] = 0\n\n # import py3DSeedEditor\n # ped = py3DSeedEditor.py3DSeedEditor(loseeds)\n # ped.show()\n\n return loseeds", "response": "Zooms a set of seeds into a single block."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef zoom_to_shape(data, shape, dtype=None):\n import scipy\n import scipy.ndimage\n\n zoomd = np.array(shape) / np.array(data.shape, dtype=np.double)\n import warnings\n\n datares = scipy.ndimage.interpolation.zoom(data, zoomd, order=0, mode=\"reflect\")\n\n if datares.shape != shape:\n logger.warning(\"Zoom with different output shape\")\n dataout = np.zeros(shape, dtype=dtype)\n shpmin = np.minimum(dataout.shape, shape)\n\n dataout[: shpmin[0], : shpmin[1], : shpmin[2]] = datares[\n : shpmin[0], : shpmin[1], : shpmin[2]\n ]\n return datares", "response": "Zoom data to specific shape."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef crop(data, crinfo):\n crinfo = fix_crinfo(crinfo)\n return data[\n __int_or_none(crinfo[0][0]) : __int_or_none(crinfo[0][1]),\n __int_or_none(crinfo[1][0]) : __int_or_none(crinfo[1][1]),\n __int_or_none(crinfo[2][0]) : __int_or_none(crinfo[2][1]),\n ]", "response": "Crop the data.\n\n crop(data, crinfo)\n\n :param crinfo: min and max for each axis - [[minX, maxX], [minY, maxY], [minZ, maxZ]]"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\ncombine two crinfos into one crinfo.", "response": "def combinecrinfo(crinfo1, crinfo2):\n \"\"\"\n Combine two crinfos. First used is crinfo1, second used is crinfo2.\n \"\"\"\n crinfo1 = fix_crinfo(crinfo1)\n crinfo2 = fix_crinfo(crinfo2)\n\n crinfo = [\n [crinfo1[0][0] + crinfo2[0][0], crinfo1[0][0] + crinfo2[0][1]],\n [crinfo1[1][0] + crinfo2[1][0], crinfo1[1][0] + crinfo2[1][1]],\n [crinfo1[2][0] + crinfo2[2][0], crinfo1[2][0] + crinfo2[2][1]],\n ]\n\n return crinfo"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\ncreating crinfo of minimum orthogonal nonzero block in input data.", "response": "def crinfo_from_specific_data(data, margin=0):\n \"\"\"\n Create crinfo of minimum orthogonal nonzero block in input data.\n\n :param data: input data\n :param margin: add margin to minimum block\n :return:\n \"\"\"\n # hled\u00e1me automatick\u00fd o\u0159ez, nonzero d\u00e1 indexy\n logger.debug(\"crinfo\")\n logger.debug(str(margin))\n nzi = np.nonzero(data)\n logger.debug(str(nzi))\n\n if np.isscalar(margin):\n margin = [margin] * 3\n\n x1 = np.min(nzi[0]) - margin[0]\n x2 = np.max(nzi[0]) + margin[0] + 1\n y1 = np.min(nzi[1]) - margin[0]\n y2 = np.max(nzi[1]) + margin[0] + 1\n z1 = np.min(nzi[2]) - margin[0]\n z2 = np.max(nzi[2]) + margin[0] + 1\n\n # o\u0161et\u0159en\u00ed mez\u00ed pol\u00ed\n if x1 < 0:\n x1 = 0\n if y1 < 0:\n y1 = 0\n if z1 < 0:\n z1 = 0\n\n if x2 > data.shape[0]:\n x2 = data.shape[0] - 1\n if y2 > data.shape[1]:\n y2 = data.shape[1] - 1\n if z2 > data.shape[2]:\n z2 = data.shape[2] - 1\n\n # o\u0159ez\n crinfo = [[x1, x2], [y1, y2], [z1, z2]]\n return crinfo"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef uncrop(data, crinfo, orig_shape, resize=False, outside_mode=\"constant\", cval=0):\n\n if crinfo is None:\n crinfo = list(zip([0] * data.ndim, orig_shape))\n elif np.asarray(crinfo).size == data.ndim:\n crinfo = list(zip(crinfo, np.asarray(crinfo) + data.shape))\n\n crinfo = fix_crinfo(crinfo)\n data_out = np.ones(orig_shape, dtype=data.dtype) * cval\n\n # print 'uncrop ', crinfo\n # print orig_shape\n # print data.shape\n if resize:\n data = resize_to_shape(data, crinfo[:, 1] - crinfo[:, 0])\n\n startx = np.round(crinfo[0][0]).astype(int)\n starty = np.round(crinfo[1][0]).astype(int)\n startz = np.round(crinfo[2][0]).astype(int)\n\n data_out[\n # np.round(crinfo[0][0]).astype(int):np.round(crinfo[0][1]).astype(int)+1,\n # np.round(crinfo[1][0]).astype(int):np.round(crinfo[1][1]).astype(int)+1,\n # np.round(crinfo[2][0]).astype(int):np.round(crinfo[2][1]).astype(int)+1\n startx : startx + data.shape[0],\n starty : starty + data.shape[1],\n startz : startz + data.shape[2],\n ] = data\n\n if outside_mode == \"nearest\":\n # for ax in range(data.ndims):\n # ax = 0\n\n # copy border slice to pixels out of boundary - the higher part\n for ax in range(data.ndim):\n # the part under the crop\n start = np.round(crinfo[ax][0]).astype(int)\n slices = [slice(None), slice(None), slice(None)]\n slices[ax] = start\n repeated_slice = np.expand_dims(data_out[slices], ax)\n append_sz = start\n if append_sz > 0:\n tile0 = np.repeat(repeated_slice, append_sz, axis=ax)\n slices = [slice(None), slice(None), slice(None)]\n slices[ax] = slice(None, start)\n # data_out[start + data.shape[ax] : , :, :] = tile0\n data_out[slices] = tile0\n # plt.imshow(np.squeeze(repeated_slice))\n # plt.show()\n\n # the part over the crop\n start = np.round(crinfo[ax][0]).astype(int)\n slices = [slice(None), slice(None), slice(None)]\n slices[ax] = start + data.shape[ax] - 1\n repeated_slice = np.expand_dims(data_out[slices], ax)\n append_sz = data_out.shape[ax] - (start + data.shape[ax])\n if append_sz > 0:\n tile0 = np.repeat(repeated_slice, append_sz, axis=ax)\n slices = [slice(None), slice(None), slice(None)]\n slices[ax] = slice(start + data.shape[ax], None)\n # data_out[start + data.shape[ax] : , :, :] = tile0\n data_out[slices] = tile0\n # plt.imshow(np.squeeze(repeated_slice))\n # plt.show()\n\n return data_out", "response": "Unrops the image into the next image in the crinfo array."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef fix_crinfo(crinfo, to=\"axis\"):\n\n crinfo = np.asarray(crinfo)\n if crinfo.shape[0] == 2:\n crinfo = crinfo.T\n\n return crinfo", "response": "Function recognize order of crinfo and convert it to proper format."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef grid_edges(shape, inds=None, return_directions=True):\n if inds is None:\n inds = np.arange(np.prod(shape)).reshape(shape)\n # if not self.segparams['use_boundary_penalties'] and \\\n # boundary_penalties_fcn is None :\n if len(shape) == 2:\n edgx = np.c_[inds[:, :-1].ravel(), inds[:, 1:].ravel()]\n edgy = np.c_[inds[:-1, :].ravel(), inds[1:, :].ravel()]\n\n edges = [edgx, edgy]\n\n directions = [\n np.ones([edgx.shape[0]], dtype=np.int8) * 0,\n np.ones([edgy.shape[0]], dtype=np.int8) * 1,\n ]\n\n elif len(shape) == 3:\n # This is faster for some specific format\n edgx = np.c_[inds[:, :, :-1].ravel(), inds[:, :, 1:].ravel()]\n edgy = np.c_[inds[:, :-1, :].ravel(), inds[:, 1:, :].ravel()]\n edgz = np.c_[inds[:-1, :, :].ravel(), inds[1:, :, :].ravel()]\n edges = [edgx, edgy, edgz]\n else:\n logger.error(\"Expected 2D or 3D data\")\n\n # for all edges along first direction put 0, for second direction put 1, for third direction put 3\n if return_directions:\n directions = []\n for idirection in range(len(shape)):\n directions.append(\n np.ones([edges[idirection].shape[0]], dtype=np.int8) * idirection\n )\n edges = np.concatenate(edges)\n if return_directions:\n edge_dir = np.concatenate(directions)\n return edges, edge_dir\n else:\n return edges", "response": "Get list of edges for a single segment of a single segment."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\ngenerates list of edges for a base grid.", "response": "def gen_grid_2d(shape, voxelsize):\n \"\"\"\n Generate list of edges for a base grid.\n \"\"\"\n nr, nc = shape\n nrm1, ncm1 = nr - 1, nc - 1\n # sh = nm.asarray(shape)\n # calculate number of edges, in 2D: (nrows * (ncols - 1)) + ((nrows - 1) * ncols)\n nedges = 0\n for direction in range(len(shape)):\n sh = copy.copy(list(shape))\n sh[direction] += -1\n nedges += nm.prod(sh)\n\n nedges_old = ncm1 * nr + nrm1 * nc\n edges = nm.zeros((nedges, 2), dtype=nm.int16)\n edge_dir = nm.zeros((ncm1 * nr + nrm1 * nc,), dtype=nm.bool)\n nodes = nm.zeros((nm.prod(shape), 3), dtype=nm.float32)\n\n # edges\n idx = 0\n row = nm.zeros((ncm1, 2), dtype=nm.int16)\n row[:, 0] = nm.arange(ncm1)\n row[:, 1] = nm.arange(ncm1) + 1\n for ii in range(nr):\n edges[slice(idx, idx + ncm1), :] = row + nc * ii\n idx += ncm1\n\n edge_dir[slice(0, idx)] = 0 # horizontal dir\n\n idx0 = idx\n col = nm.zeros((nrm1, 2), dtype=nm.int16)\n col[:, 0] = nm.arange(nrm1) * nc\n col[:, 1] = nm.arange(nrm1) * nc + nc\n for ii in range(nc):\n edges[slice(idx, idx + nrm1), :] = col + ii\n idx += nrm1\n\n edge_dir[slice(idx0, idx)] = 1 # vertical dir\n\n # nodes\n idx = 0\n row = nm.zeros((nc, 3), dtype=nm.float32)\n row[:, 0] = voxelsize[0] * (nm.arange(nc) + 0.5)\n row[:, 1] = voxelsize[1] * 0.5\n for ii in range(nr):\n nodes[slice(idx, idx + nc), :] = row\n row[:, 1] += voxelsize[1]\n idx += nc\n\n return nodes, edges, edge_dir"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef write_grid_to_vtk(fname, nodes, edges, node_flag=None, edge_flag=None):\n\n if node_flag is None:\n node_flag = np.ones([nodes.shape[0]], dtype=np.bool)\n if edge_flag is None:\n edge_flag = np.ones([edges.shape[0]], dtype=np.bool)\n nodes = make_nodes_3d(nodes)\n f = open(fname, \"w\")\n\n f.write(\"# vtk DataFile Version 2.6\\n\")\n f.write(\"output file\\nASCII\\nDATASET UNSTRUCTURED_GRID\\n\")\n\n idxs = nm.where(node_flag > 0)[0]\n nnd = len(idxs)\n aux = -nm.ones(node_flag.shape, dtype=nm.int32)\n aux[idxs] = nm.arange(nnd, dtype=nm.int32)\n f.write(\"\\nPOINTS %d float\\n\" % nnd)\n for ndi in idxs:\n f.write(\"%.6f %.6f %.6f\\n\" % tuple(nodes[ndi, :]))\n\n idxs = nm.where(edge_flag > 0)[0]\n ned = len(idxs)\n f.write(\"\\nCELLS %d %d\\n\" % (ned, ned * 3))\n for edi in idxs:\n f.write(\"2 %d %d\\n\" % tuple(aux[edges[edi, :]]))\n\n f.write(\"\\nCELL_TYPES %d\\n\" % ned)\n for edi in idxs:\n f.write(\"3\\n\")", "response": "Writes nodes and edges to VTK file"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nadding new nodes at the end of the list.", "response": "def add_nodes(self, coors, node_low_or_high=None):\n \"\"\"\n Add new nodes at the end of the list.\n \"\"\"\n last = self.lastnode\n if type(coors) is nm.ndarray:\n if len(coors.shape) == 1:\n coors = coors.reshape((1, coors.size))\n\n nadd = coors.shape[0]\n idx = slice(last, last + nadd)\n else:\n nadd = 1\n idx = self.lastnode\n right_dimension = coors.shape[1]\n self.nodes[idx, :right_dimension] = coors\n self.node_flag[idx] = True\n self.lastnode += nadd\n self.nnodes += nadd"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nadding new edges at the end of the list.", "response": "def add_edges(self, conn, edge_direction, edge_group=None, edge_low_or_high=None):\n \"\"\"\n Add new edges at the end of the list.\n :param edge_direction: direction flag\n :param edge_group: describes group of edges from same low super node and same direction\n :param edge_low_or_high: zero for low to low resolution, one for high to high or high to low resolution.\n It is used to set weight from weight table.\n \"\"\"\n last = self.lastedge\n if type(conn) is nm.ndarray:\n nadd = conn.shape[0]\n idx = slice(last, last + nadd)\n if edge_group is None:\n edge_group = nm.arange(nadd) + last\n else:\n nadd = 1\n idx = nm.array([last])\n conn = nm.array(conn).reshape((1, 2))\n if edge_group is None:\n edge_group = idx\n\n self.edges[idx, :] = conn\n self.edge_flag[idx] = True\n # t_start0 = time.time()\n # self.edge_flag_idx.extend(list(range(idx.start, idx.stop)))\n # self.stats[\"t split 082\"] += time.time() - t_start0\n self.edge_dir[idx] = edge_direction\n self.edge_group[idx] = edge_group\n # TODO change this just to array of low_or_high_resolution\n if edge_low_or_high is not None and self._edge_weight_table is not None:\n self.edges_weights[idx] = self._edge_weight_table[\n edge_low_or_high, edge_direction\n ]\n self.lastedge += nadd\n self.nedges += nadd"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nsubstitutes low resolution edges.", "response": "def _edge_group_substitution(\n self, ndid, nsplit, idxs, sr_tab, ndoffset, ed_remove, into_or_from\n ):\n \"\"\"\n Reconnect edges.\n :param ndid: id of low resolution edges\n :param nsplit: number of split\n :param idxs: indexes of low resolution\n :param sr_tab:\n :param ndoffset:\n :param ed_remove:\n :param into_or_from: if zero, connection of input edges is done. If one, connection of output edges\n is performed.\n :return:\n \"\"\"\n # this is useful for type(idxs) == np.ndarray\n eidxs = idxs[nm.where(self.edges[idxs, 1 - into_or_from] == ndid)[0]]\n # selected_edges = self.edges[idxs, 1 - into_or_from]\n # selected_edges == ndid\n # whre = nm.where(self.edges[idxs, 1 - into_or_from] == ndid)\n # whre0 = (nm.where(self.edges[idxs, 1 - into_or_from] == ndid) == ndid)[0]\n # eidxs = [idxs[i] for i in idxs]\n for igrp in self.edges_by_group(eidxs):\n if igrp.shape[0] > 1:\n # high resolution block to high resolution block\n # all directions are the same\n directions = self.edge_dir[igrp[0]]\n edge_indexes = sr_tab[directions, :].T.flatten() + ndoffset\n # debug code\n # if len(igrp) != len(edge_indexes):\n # print(\"Problem \")\n self.edges[igrp, 1] = edge_indexes\n if self._edge_weight_table is not None:\n self.edges_weights[igrp] = self._edge_weight_table[1, directions]\n else:\n # low res block to hi res block, if into_or_from is set to 0\n # hig res block to low res block, if into_or_from is set to 1\n ed_remove.append(igrp[0])\n # number of new edges is equal to number of pixels on one side of the box (in 2D and D too)\n nnewed = np.power(nsplit, self.data.ndim - 1)\n muleidxs = nm.tile(igrp, nnewed)\n # copy the low-res edge multipletime\n newed = self.edges[muleidxs, :]\n neweddir = self.edge_dir[muleidxs]\n local_node_ids = sr_tab[\n self.edge_dir[igrp] + self.data.ndim * into_or_from, :\n ].T.flatten()\n # first or second (the actual) node id is substitued by new node indexes\n newed[:, 1 - into_or_from] = local_node_ids + ndoffset\n if self._edge_weight_table is not None:\n self.add_edges(\n newed, neweddir, self.edge_group[igrp], edge_low_or_high=1\n )\n else:\n self.add_edges(\n newed, neweddir, self.edge_group[igrp], edge_low_or_high=None\n )\n return ed_remove"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef generate_base_grid(self, vtk_filename=None):\n nd, ed, ed_dir = self.gen_grid_fcn(self.data.shape, self.voxelsize)\n self.add_nodes(nd)\n self.add_edges(ed, ed_dir, edge_low_or_high=0)\n\n if vtk_filename is not None:\n self.write_vtk(vtk_filename)", "response": "Generate base grid of the hierarchy."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nseconding step of algorithm :return:()", "response": "def split_voxels(self, vtk_filename=None):\n \"\"\"\n Second step of algorithm\n :return:()\n \"\"\"\n self.cache = {}\n self.stats[\"t graph 10\"] = time.time() - self.start_time\n self.msi = MultiscaleArray(self.data.shape, block_size=self.nsplit)\n\n # old implementation\n # idxs = nm.where(self.data)\n # nr, nc = self.data.shape\n # for k, (ir, ic) in enumerate(zip(*idxs)):\n # ndid = ic + ir * nc\n # self.split_voxel(ndid, self.nsplit)\n\n # new_implementation\n # for ndid in np.flatnonzero(self.data):\n # self.split_voxel(ndid, self.nsplit)\n\n # even newer implementation\n self.stats[\"t graph 11\"] = time.time() - self.start_time\n for ndid, val in enumerate(self.data.ravel()):\n t_split_start = time.time()\n if val == 0:\n if self.compute_msindex:\n self.msi.set_block_lowres(ndid, ndid)\n self.stats[\"t graph low\"] += time.time() - t_split_start\n else:\n self.split_voxel(ndid)\n self.stats[\"t graph high\"] += time.time() - t_split_start\n\n self.stats[\"t graph 13\"] = time.time() - self.start_time\n self.finish()\n if vtk_filename is not None:\n self.write_vtk(vtk_filename)\n self.stats[\"t graph 14\"] = time.time() - self.start_time"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nmultiply values in block", "response": "def mul_block(self, index, val):\n \"\"\"Multiply values in block\"\"\"\n self._prepare_cache_slice(index)\n self.msinds[self.cache_slice] *= val"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\ntooling to make simple feature functions take features from feature vector array by seeds.", "response": "def select_from_fv_by_seeds(fv, seeds, unique_cls):\n \"\"\"\n Tool to make simple feature functions take features from feature array by seeds.\n :param fv: ndarray with lineariezed feature. It's shape is MxN, where M is number of image pixels and N is number\n of features\n :param seeds: ndarray with seeds. Does not to be linear.\n :param unique_cls: number of used seeds clases. Like [1, 2]\n :return: fv_selection, seeds_selection - selection from feature vector and selection from seeds\n \"\"\"\n logger.debug(\"seeds\" + str(seeds))\n # fvlin = fv.reshape(-1, int(fv.size/seeds.size))\n expected_shape = [seeds.size, int(fv.size/seeds.size)]\n if fv.shape[0] != expected_shape[0] or fv.shape[1] != expected_shape[1]:\n raise AssertionError(\"Wrong shape of input feature vector array fv\")\n # sd = seeds.reshape(-1, 1)\n selection = np.in1d(seeds, unique_cls)\n fv_selection = fv[selection]\n seeds_selection = seeds.flatten()[selection]\n # sd = sd[]\n return fv_selection, seeds_selection"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nreturn features selected by seeds and unique_cls or just fv for whole image.", "response": "def return_fv_by_seeds(fv, seeds=None, unique_cls=None):\n \"\"\"\n Return features selected by seeds and unique_cls or selection from features and corresponding seed classes.\n\n :param fv: ndarray with lineariezed feature. It's shape is MxN, where M is number of image pixels and N is number\n of features\n :param seeds: ndarray with seeds. Does not to be linear.\n :param unique_cls: number of used seeds clases. Like [1, 2]\n :return: fv, sd - selection from feature vector and selection from seeds or just fv for whole image\n \"\"\"\n if seeds is not None:\n if unique_cls is not None:\n return select_from_fv_by_seeds(fv, seeds, unique_cls)\n else:\n raise AssertionError(\"Input unique_cls has to be not None if seeds is not None.\")\n else:\n return fv"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef get_gutter_client(\n alias='default',\n cache=CLIENT_CACHE,\n **kwargs\n):\n \"\"\"\n Creates gutter clients and memoizes them in a registry for future quick access.\n\n Args:\n alias (str or None): Name of the client. Used for caching.\n If name is falsy then do not use the cache.\n cache (dict): cache to store gutter managers in.\n **kwargs: kwargs to be passed the Manger class.\n\n Returns (Manager):\n A gutter client.\n\n \"\"\"\n from gutter.client.models import Manager\n\n if not alias:\n return Manager(**kwargs)\n elif alias not in cache:\n cache[alias] = Manager(**kwargs)\n\n return cache[alias]", "response": "Returns a gutter client."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef _modulo(self, decimal_argument):\n _times, remainder = self._context.divmod(decimal_argument, 100)\n\n # match the builtin % behavior by adding the N to the result if negative\n return remainder if remainder >= 0 else remainder + 100", "response": "modulo is a wrapper around the mod operator."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef enabled_for(self, inpt):\n\n signals.switch_checked.call(self)\n signal_decorated = partial(self.__signal_and_return, inpt)\n\n if self.state is self.states.GLOBAL:\n return signal_decorated(True)\n elif self.state is self.states.DISABLED:\n return signal_decorated(False)\n\n conditions_dict = ConditionsDict.from_conditions_list(self.conditions)\n conditions = conditions_dict.get_by_input(inpt)\n\n if conditions:\n result = self.__enabled_func(\n cond.call(inpt)\n for cond\n in conditions\n if cond.argument(inpt).applies\n )\n else:\n result = None\n\n return signal_decorated(result)", "response": "Returns True if the switch is enabled for the input."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nreturn if the condition applies to the inpt.", "response": "def call(self, inpt):\n \"\"\"\n Returns if the condition applies to the ``inpt``.\n\n If the class ``inpt`` is an instance of is not the same class as the\n condition's own ``argument``, then ``False`` is returned. This also\n applies to the ``NONE`` input.\n\n Otherwise, ``argument`` is called, with ``inpt`` as the instance and\n the value is compared to the ``operator`` and the Value is returned. If\n the condition is ``negative``, then then ``not`` the value is returned.\n\n Keyword Arguments:\n inpt -- An instance of the ``Input`` class.\n \"\"\"\n if inpt is Manager.NONE_INPUT:\n return False\n\n # Call (construct) the argument with the input object\n argument_instance = self.argument(inpt)\n\n if not argument_instance.applies:\n return False\n\n application = self.__apply(argument_instance, inpt)\n\n if self.negative:\n application = not application\n\n return application"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nreturn a list of all switches currently registered.", "response": "def switches(self):\n \"\"\"\n List of all switches currently registered.\n \"\"\"\n results = [\n switch for name, switch in self.storage.iteritems()\n if name.startswith(self.__joined_namespace)\n ]\n\n return results"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nreturns the switch with the provided name.", "response": "def switch(self, name):\n \"\"\"\n Returns the switch with the provided ``name``.\n\n If ``autocreate`` is set to ``True`` and no switch with that name\n exists, a ``DISABLED`` switch will be with that name.\n\n Keyword Arguments:\n name -- A name of a switch.\n \"\"\"\n try:\n switch = self.storage[self.__namespaced(name)]\n except KeyError:\n if not self.autocreate:\n raise ValueError(\"No switch named '%s' registered in '%s'\" % (name, self.namespace))\n\n switch = self.__create_and_register_disabled_switch(name)\n\n switch.manager = self\n return switch"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nregistering a switch and persist it to the storage.", "response": "def register(self, switch, signal=signals.switch_registered):\n '''\n Register a switch and persist it to the storage.\n '''\n if not switch.name:\n raise ValueError('Switch name cannot be blank')\n\n switch.manager = self\n self.__persist(switch)\n\n signal.call(switch)"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef when(obj, strict=None):\n\n if isinstance(obj, str):\n obj = get_obj(obj)\n\n if strict is None:\n strict = True\n theMock = _get_mock(obj, strict=strict)\n\n class When(object):\n def __getattr__(self, method_name):\n return invocation.StubbedInvocation(\n theMock, method_name, strict=strict)\n\n return When()", "response": "A simple interface to stub functions on a given object."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef when2(fn, *args, **kwargs):\n obj, name = get_obj_attr_tuple(fn)\n theMock = _get_mock(obj, strict=True)\n return invocation.StubbedInvocation(theMock, name)(*args, **kwargs)", "response": "Stub a function call with the given arguments returning an answer selector that exposes thenReturn thenRaise and thenAnswer."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef expect(obj, strict=None,\n times=None, atleast=None, atmost=None, between=None):\n \"\"\"Stub a function call, and set up an expected call count.\n\n Usage::\n\n # Given `dog` is an instance of a `Dog`\n expect(dog, times=1).bark('Wuff').thenReturn('Miau')\n dog.bark('Wuff')\n dog.bark('Wuff') # will throw at call time: too many invocations\n\n # maybe if you need to ensure that `dog.bark()` was called at all\n verifyNoUnwantedInteractions()\n\n .. note:: You must :func:`unstub` after stubbing, or use `with`\n statement.\n\n See :func:`when`, :func:`when2`, :func:`verifyNoUnwantedInteractions`\n\n \"\"\"\n if strict is None:\n strict = True\n theMock = _get_mock(obj, strict=strict)\n\n verification_fn = _get_wanted_verification(\n times=times, atleast=atleast, atmost=atmost, between=between)\n\n class Expect(object):\n def __getattr__(self, method_name):\n return invocation.StubbedInvocation(\n theMock, method_name, verification=verification_fn,\n strict=strict)\n\n return Expect()", "response": "Stub a function call and set up an expected call count."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nverifies that no methods have been called on given objs.", "response": "def verifyZeroInteractions(*objs):\n \"\"\"Verify that no methods have been called on given objs.\n\n Note that strict mocks usually throw early on unexpected, unstubbed\n invocations. Partial mocks ('monkeypatched' objects or modules) do not\n support this functionality at all, bc only for the stubbed invocations\n the actual usage gets recorded. So this function is of limited use,\n nowadays.\n\n \"\"\"\n for obj in objs:\n theMock = _get_mock_or_raise(obj)\n\n if len(theMock.invocations) > 0:\n raise VerificationError(\n \"\\nUnwanted interaction: %s\" % theMock.invocations[0])"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef verifyNoUnwantedInteractions(*objs):\n\n if objs:\n theMocks = map(_get_mock_or_raise, objs)\n else:\n theMocks = mock_registry.get_registered_mocks()\n\n for mock in theMocks:\n for i in mock.stubbed_invocations:\n i.verify()", "response": "Verifies that expectations set via expect are met\n\n "} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nensures that stubbed methods are actually used.", "response": "def verifyStubbedInvocationsAreUsed(*objs):\n \"\"\"Ensure stubs are actually used.\n\n This functions just ensures that stubbed methods are actually used. Its\n purpose is to detect interface changes after refactorings. It is meant\n to be invoked usually without arguments just before :func:`unstub`.\n\n \"\"\"\n if objs:\n theMocks = map(_get_mock_or_raise, objs)\n else:\n theMocks = mock_registry.get_registered_mocks()\n\n\n for mock in theMocks:\n for i in mock.stubbed_invocations:\n if not i.allow_zero_invocations and i.used < len(i.answers):\n raise VerificationError(\"\\nUnused stub: %s\" % i)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef get_obj(path):\n # Since we usually pass in mocks here; duck typing is not appropriate\n # (mocks respond to every attribute).\n if not isinstance(path, str):\n return path\n\n if path.startswith('.'):\n raise TypeError('relative imports are not supported')\n\n parts = path.split('.')\n head, tail = parts[0], parts[1:]\n\n obj = importlib.import_module(head)\n\n # Normally a simple reduce, but we go the extra mile\n # for good exception messages.\n for i, name in enumerate(tail):\n try:\n obj = getattr(obj, name)\n except AttributeError:\n # Note the [:i] instead of [:i+1], so we get the path just\n # *before* the AttributeError, t.i. the part of it that went ok.\n module = '.'.join([head] + tail[:i])\n try:\n importlib.import_module(module)\n except ImportError:\n raise AttributeError(\n \"object '%s' has no attribute '%s'\" % (module, name))\n else:\n raise AttributeError(\n \"module '%s' has no attribute '%s'\" % (module, name))\n return obj", "response": "Return obj for given dotted path."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nsplitting path into obj attribute tuple.", "response": "def get_obj_attr_tuple(path):\n \"\"\"Split path into (obj, attribute) tuple.\n\n Given `path` is 'os.path.exists' will thus return `(os.path, 'exists')`\n\n If path is not a str, delegates to `get_function_host(path)`\n\n \"\"\"\n if not isinstance(path, str):\n return get_function_host(path)\n\n if path.startswith('.'):\n raise TypeError('relative imports are not supported')\n\n try:\n leading, end = path.rsplit('.', 1)\n except ValueError:\n raise TypeError('path must have dots')\n\n return get_obj(leading), end"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nspying an object. Spying means that all functions will behave as before, so they will be side effects, but the interactions can be verified afterwards. Returns Dummy-like, almost empty object as proxy to `object`. The *returned* object must be injected and used by the code under test; after that all interactions can be verified as usual. T.i. the original object **will not be patched**, and has no further knowledge as before. E.g.:: import time time = spy(time) # inject time do_work(..., time) verify(time).time()", "response": "def spy(object):\n \"\"\"Spy an object.\n\n Spying means that all functions will behave as before, so they will\n be side effects, but the interactions can be verified afterwards.\n\n Returns Dummy-like, almost empty object as proxy to `object`.\n\n The *returned* object must be injected and used by the code under test;\n after that all interactions can be verified as usual.\n T.i. the original object **will not be patched**, and has no further\n knowledge as before.\n\n E.g.::\n\n import time\n time = spy(time)\n # inject time\n do_work(..., time)\n verify(time).time()\n\n \"\"\"\n if inspect.isclass(object) or inspect.ismodule(object):\n class_ = None\n else:\n class_ = object.__class__\n\n class Spy(_Dummy):\n if class_:\n __class__ = class_\n\n def __getattr__(self, method_name):\n return RememberedProxyInvocation(theMock, method_name)\n\n def __repr__(self):\n name = 'Spied'\n if class_:\n name += class_.__name__\n return \"<%s id=%s>\" % (name, id(self))\n\n\n obj = Spy()\n theMock = Mock(obj, strict=True, spec=object)\n\n mock_registry.register(obj, theMock)\n return obj"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef spy2(fn): # type: (...) -> None\n if isinstance(fn, str):\n answer = get_obj(fn)\n else:\n answer = fn\n\n when2(fn, Ellipsis).thenAnswer(answer)", "response": "Spy usage of given fn."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\ncreate an empty mock object.", "response": "def mock(config_or_spec=None, spec=None, strict=OMITTED):\n \"\"\"Create 'empty' objects ('Mocks').\n\n Will create an empty unconfigured object, that you can pass\n around. All interactions (method calls) will be recorded and can be\n verified using :func:`verify` et.al.\n\n A plain `mock()` will be not `strict`, and thus all methods regardless\n of the arguments will return ``None``.\n\n .. note:: Technically all attributes will return an internal interface.\n Because of that a simple ``if mock().foo:`` will surprisingly pass.\n\n If you set strict to ``True``: ``mock(strict=True)`` all unexpected\n interactions will raise an error instead.\n\n You configure a mock using :func:`when`, :func:`when2` or :func:`expect`.\n You can also very conveniently just pass in a dict here::\n\n response = mock({'text': 'ok', 'raise_for_status': lambda: None})\n\n You can also create an empty Mock which is specced against a given\n `spec`: ``mock(requests.Response)``. These mock are by default strict,\n thus they raise if you want to stub a method, the spec does not implement.\n Mockito will also match the function signature.\n\n You can pre-configure a specced mock as well::\n\n response = mock({'json': lambda: {'status': 'Ok'}},\n spec=requests.Response)\n\n Mocks are by default callable. Configure the callable behavior using\n `when`::\n\n dummy = mock()\n when(dummy).__call_(1).thenReturn(2)\n\n All other magic methods must be configured this way or they will raise an\n AttributeError.\n\n\n See :func:`verify` to verify your interactions after usage.\n\n \"\"\"\n\n if type(config_or_spec) is dict:\n config = config_or_spec\n else:\n config = {}\n spec = config_or_spec\n\n if strict is OMITTED:\n strict = False if spec is None else True\n\n\n class Dummy(_Dummy):\n if spec:\n __class__ = spec # make isinstance work\n\n def __getattr__(self, method_name):\n if strict:\n raise AttributeError(\n \"'Dummy' has no attribute %r configured\" % method_name)\n return functools.partial(\n remembered_invocation_builder, theMock, method_name)\n\n def __repr__(self):\n name = 'Dummy'\n if spec:\n name += spec.__name__\n return \"<%s id=%s>\" % (name, id(self))\n\n\n # That's a tricky one: The object we will return is an *instance* of our\n # Dummy class, but the mock we register will point and patch the class.\n # T.i. so that magic methods (`__call__` etc.) can be configured.\n obj = Dummy()\n theMock = Mock(Dummy, strict=strict, spec=spec)\n\n for n, v in config.items():\n if inspect.isfunction(v):\n invocation.StubbedInvocation(theMock, n)(Ellipsis).thenAnswer(v)\n else:\n setattr(obj, n, v)\n\n mock_registry.register(obj, theMock)\n return obj"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nfunctions importPuppetClasses Force the reload of puppet classes", "response": "def importPuppetClasses(self, smartProxyId):\n \"\"\" Function importPuppetClasses\n Force the reload of puppet classes\n\n @param smartProxyId: smartProxy Id\n @return RETURN: the API result\n \"\"\"\n return self.api.create('{}/{}/import_puppetclasses'\n .format(self.objName, smartProxyId), '{}')"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nreturn a list of templates usable by a model.", "response": "def get_templates(model):\n \"\"\" Return a list of templates usable by a model. \"\"\"\n for template_name, template in templates.items():\n if issubclass(template.model, model):\n yield (template_name, template.layout._meta.verbose_name)"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef attach(*layouts, **kwargs):\n\n def _model_admin_wrapper(layout_class):\n register(layout_class, layouts[0])\n return layout_class\n return _model_admin_wrapper", "response": "Decorator for creating admin site classes."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef enhance(self):\n self.update({'os_default_templates':\n SubDict(self.api, self.objName,\n self.payloadObj, self.key,\n SubItemOsDefaultTemplate)})\n self.update({'config_templates':\n SubDict(self.api, self.objName,\n self.payloadObj, self.key,\n SubItemConfigTemplate)})\n self.update({'ptables':\n SubDict(self.api, self.objName,\n self.payloadObj, self.key,\n SubItemPTable)})\n self.update({'media':\n SubDict(self.api, self.objName,\n self.payloadObj, self.key,\n SubItemMedia)})\n self.update({'architectures':\n SubDict(self.api, self.objName,\n self.payloadObj, self.key,\n SubItemArchitecture)})", "response": "Function enhance\n Enhance the object with new item or enhanced items\n "} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef get_api_envs():\n client_id = os.environ.get('CLIENT_ID')\n user_id = os.environ.get('USER_ID')\n if not client_id or not user_id:\n raise ValueError('API keys are not found in the environment')\n return client_id, user_id", "response": "Get required API keys from environment variables."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\ncalls given API end_point with API keys.", "response": "def api_call(method, end_point, params=None, client_id=None, access_token=None):\n \"\"\"Call given API end_point with API keys.\n :param method: HTTP method (e.g. 'get', 'delete').\n :param end_point: API endpoint (e.g. 'users/john/sets').\n :param params: Dictionary to be sent in the query string (e.g. {'myparam': 'myval'})\n :param client_id: Quizlet client ID as string.\n :param access_token: Quizlet access token as string.\n client_id and access_token are mutually exclusive but mandatory.\n \"\"\"\n if bool(client_id) == bool(access_token):\n raise ValueError('Either client_id or access_token')\n\n url = 'https://api.quizlet.com/2.0/{}'.format(end_point)\n\n if not params:\n params = {}\n if client_id:\n params['client_id'] = client_id\n\n headers = {'Authorization': 'Bearer {}'.format(access_token)} if access_token else None\n\n response = requests.request(method, url, params=params, headers=headers)\n\n if int(response.status_code / 100) != 2:\n error_title = ''\n try:\n error_title += ', ' + response.json()['error_title']\n except ValueError:\n pass\n except KeyError:\n pass\n raise ValueError(\n '{} returned {}{}'.format(url, response.status_code, error_title)\n )\n\n try:\n return response.json()\n except json.decoder.JSONDecodeError:\n pass"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef request_upload_secret(self, secret_id):\n return self._router.post_request_upload_secret(org_id=self.organizationId,\n instance_id=self.instanceId,\n secret_id=secret_id).json()", "response": "Request upload secret for a key"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef checkAndCreate(self, key, payload, domainId):\n if key not in self:\n self[key] = payload\n oid = self[key]['id']\n if not oid:\n return False\n #~ Ensure subnet contains the domain\n subnetDomainIds = []\n for domain in self[key]['domains']:\n subnetDomainIds.append(domain['id'])\n if domainId not in subnetDomainIds:\n subnetDomainIds.append(domainId)\n self[key][\"domain_ids\"] = subnetDomainIds\n if len(self[key][\"domains\"]) is not len(subnetDomainIds):\n return False\n return oid", "response": "Function checkAndCreate\n Check if a subnet exists and create it if not"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef removeDomain(self, subnetId, domainId):\n subnetDomainIds = []\n for domain in self[subnetId]['domains']:\n subnetDomainIds.append(domain['id'])\n subnetDomainIds.remove(domainId)\n self[subnetId][\"domain_ids\"] = subnetDomainIds\n return len(self[subnetId][\"domains\"]) is len(subnetDomainIds)", "response": "Function removeDomain\n Delete a domain from a subnet"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef exclusive(via=threading.Lock):\n def make_exclusive(fnc):\n fnc_guard = via()\n\n @functools.wraps(fnc)\n def exclusive_call(*args, **kwargs):\n if fnc_guard.acquire(blocking=False):\n try:\n return fnc(*args, **kwargs)\n finally:\n fnc_guard.release()\n else:\n raise RuntimeError('exclusive call to %s violated')\n return exclusive_call\n return make_exclusive", "response": "Decorator for creating a new object that is exclusive across all instances of the object."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nexecuting a payload asynchronously and provide its output", "response": "def execute(self, payload, *args, flavour: ModuleType, **kwargs):\n \"\"\"\n Synchronously run ``payload`` and provide its output\n\n If ``*args*`` and/or ``**kwargs`` are provided, pass them to ``payload`` upon execution.\n \"\"\"\n if args or kwargs:\n payload = functools.partial(payload, *args, **kwargs)\n return self._meta_runner.run_payload(payload, flavour=flavour)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef adopt(self, payload, *args, flavour: ModuleType, **kwargs):\n if args or kwargs:\n payload = functools.partial(payload, *args, **kwargs)\n self._meta_runner.register_payload(payload, flavour=flavour)", "response": "Adoptes the payload in the background."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef accept(self):\n if self._meta_runner:\n raise RuntimeError('payloads scheduled for %s before being started' % self)\n self._must_shutdown = False\n self._logger.info('%s starting', self.__class__.__name__)\n # force collecting objects so that defunct, migrated and overwritten services are destroyed now\n gc.collect()\n self._adopt_services()\n self.adopt(self._accept_services, flavour=trio)\n self._meta_runner.run()", "response": "Start accepting synchronous asynchronous and service payloads"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef shutdown(self):\n self._must_shutdown = True\n self._is_shutdown.wait()\n self._meta_runner.stop()", "response": "Shutdown the accept loop and stop running payloads"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nviews / edit / close milestones on github", "response": "def milestones(ctx, list, close):\n \"\"\"View/edit/close milestones on github\n \"\"\"\n repos = get_repos(ctx.parent.agile.get('labels'))\n if list:\n _list_milestones(repos)\n elif close:\n click.echo('Closing milestones \"%s\"' % close)\n _close_milestone(repos, close)\n else:\n click.echo(ctx.get_help())"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef start_console(local_vars={}):\n '''Starts a console; modified from code.interact'''\n transforms.CONSOLE_ACTIVE = True\n transforms.remove_not_allowed_in_console()\n sys.ps1 = prompt\n console = ExperimentalInteractiveConsole(locals=local_vars)\n console.interact(banner=banner)", "response": "Starts a console ; modified from code. interact"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\ntransforms and push a line to the interpreter.", "response": "def push(self, line):\n \"\"\"Transform and push a line to the interpreter.\n\n The line should not have a trailing newline; it may have\n internal newlines. The line is appended to a buffer and the\n interpreter's runsource() method is called with the\n concatenated contents of the buffer as source. If this\n indicates that the command was executed or invalid, the buffer\n is reset; otherwise, the command is incomplete, and the buffer\n is left as it was after the line was appended. The return\n value is 1 if more input is required, 0 if the line was dealt\n with in some way (this is the same as runsource()).\n\n \"\"\"\n if transforms.FROM_EXPERIMENTAL.match(line):\n transforms.add_transformers(line)\n self.buffer.append(\"\\n\")\n else:\n self.buffer.append(line)\n\n add_pass = False\n if line.rstrip(' ').endswith(\":\"):\n add_pass = True\n source = \"\\n\".join(self.buffer)\n if add_pass:\n source += \"pass\"\n source = transforms.transform(source)\n if add_pass:\n source = source.rstrip(' ')\n if source.endswith(\"pass\"):\n source = source[:-4]\n\n # some transformations may strip an empty line meant to end a block\n if not self.buffer[-1]:\n source += \"\\n\"\n try:\n more = self.runsource(source, self.filename)\n except SystemExit:\n os._exit(1)\n\n if not more:\n self.resetbuffer()\n return more"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef dump(obj, f, preserve=False):\n if not f.write:\n raise TypeError('You can only dump an object into a file object')\n encoder = Encoder(f, preserve=preserve)\n return encoder.write_dict(obj)", "response": "Writes a dict object into a TOML file"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nload licenses from the given directory.", "response": "def license_loader(lic_dir=LIC_DIR):\n \"\"\"Loads licenses from the given directory.\"\"\"\n lics = []\n for ln in os.listdir(lic_dir):\n lp = os.path.join(lic_dir, ln)\n with open(lp) as lf:\n txt = lf.read()\n lic = License(txt)\n lics.append(lic)\n return lics"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef get_vector(self, max_choice=3):\n vec = {}\n for dim in ['forbidden', 'required', 'permitted']:\n if self.meta[dim] is None:\n continue\n dim_vec = map(lambda x: (x, max_choice), self.meta[dim])\n vec[dim] = dict(dim_vec)\n return vec", "response": "Return pseudo - choice vectors."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef entity(ctx, debug, uncolorize, **kwargs):\n global PROVIDER_CONFIG\n\n if debug:\n log.basicConfig(level=log.DEBUG)\n log.getLogger(\"requests.packages.urllib3.connectionpool\").setLevel(log.DEBUG)\n for (k, v) in kwargs.iteritems():\n if v:\n QUBELL[k] = v\n PROVIDER_CONFIG = {\n 'configuration.provider': PROVIDER['provider_type'],\n 'configuration.legacy-regions': PROVIDER['provider_region'],\n 'configuration.endpoint-url': '',\n 'configuration.legacy-security-group': '',\n 'configuration.identity': PROVIDER['provider_identity'],\n 'configuration.credential': PROVIDER['provider_credential']\n }\n\n class UserContext(object):\n def __init__(self):\n self.platform = None\n self.unauthenticated_platform = None\n self.colorize = not (uncolorize)\n\n def get_platform(self):\n if not self.platform:\n assert QUBELL[\"tenant\"], \"No platform URL provided. Set QUBELL_TENANT or use --tenant option.\"\n if not QUBELL[\"token\"]:\n assert QUBELL[\"user\"], \"No username. Set QUBELL_USER or use --user option.\"\n assert QUBELL[\"password\"], \"No password provided. Set QUBELL_PASSWORD or use --password option.\"\n\n self.platform = QubellPlatform.connect(\n tenant=QUBELL[\"tenant\"],\n user=QUBELL[\"user\"],\n password=QUBELL[\"password\"],\n token=QUBELL[\"token\"])\n return self.platform\n\n def get_unauthenticated_platform(self):\n if not self.unauthenticated_platform:\n assert QUBELL[\"tenant\"], \"No platform URL provided. Set QUBELL_TENANT or use --tenant option.\"\n\n self.unauthenticated_platform = QubellPlatform.connect(tenant=QUBELL[\"tenant\"])\n\n return self.unauthenticated_platform\n\n ctx = click.get_current_context()\n ctx.obj = UserContext()", "response": "This function is used to create a new entity for the current user."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nuploading application from files to the current user s organization.", "response": "def import_app(files, category, overwrite, id, name):\n \"\"\" Upload application from file.\n\n By default, file name will be used as application name, with \"-vXX.YYY\" suffix stripped.\n Application is looked up by one of these classifiers, in order of priority:\n app-id, app-name, filename.\n\n If app-id is provided, looks up existing application and updates its manifest.\n If app-id is NOT specified, looks up by name, or creates new application.\n\n \"\"\"\n platform = _get_platform()\n org = platform.get_organization(QUBELL[\"organization\"])\n if category:\n category = org.categories[category]\n regex = re.compile(r\"^(.*?)(-v(\\d+)|)\\.[^.]+$\")\n if (id or name) and len(files) > 1:\n raise Exception(\"--id and --name are supported only for single-file mode\")\n\n for filename in files:\n click.echo(\"Importing \" + filename, nl=False)\n if not name:\n match = regex.match(basename(filename))\n if not match:\n click.echo(_color(\"RED\", \"FAIL\") + \" unknown filename format\")\n break\n name = regex.match(basename(filename)).group(1)\n click.echo(\" => \", nl=False)\n app = None\n try:\n app = org.get_application(id=id, name=name)\n if app and not overwrite:\n click.echo(\"%s %s already exists %s\" % (\n app.id, _color(\"BLUE\", app and app.name or name), _color(\"RED\", \"FAIL\")))\n break\n except NotFoundError:\n if id:\n click.echo(\"%s %s not found %s\" % (\n id or \"\", _color(\"BLUE\", app and app.name or name), _color(\"RED\", \"FAIL\")))\n break\n click.echo(_color(\"BLUE\", app and app.name or name) + \" \", nl=False)\n try:\n with file(filename, \"r\") as f:\n if app:\n app.update(name=app.name,\n category=category and category.id or app.category,\n manifest=Manifest(content=f.read()))\n else:\n app = org.application(id=id, name=name, manifest=Manifest(content=f.read()))\n if category:\n app.update(category=category.id)\n click.echo(app.id + _color(\"GREEN\", \" OK\"))\n except IOError as e:\n click.echo(_color(\"RED\", \" FAIL\") + \" \" + e.message)\n break"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef show_account():\n click.echo(\"# tonomi api\")\n for (key, env) in REVERSE_MAPPING.items():\n value = QUBELL.get(key, None)\n if value:\n click.echo(\"export %s='%s'\" % (env, value))\n if any(map(lambda x: PROVIDER.get(x), REVERSE_PROVIDER_MAPPING.keys())):\n click.echo(\"# cloud account\")\n for (key, env) in REVERSE_PROVIDER_MAPPING.items():\n value = PROVIDER.get(key, None)\n if value:\n click.echo(\"export %s='%s'\" % (env, value))", "response": "Shows current account in a tonomi api shell - friendly form. Takes into account\n explicit top - level flags like organization."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\ngenerate a new session token from the given refresh token.", "response": "def generate_session_token(refresh_token, verbose):\n \"\"\"\n Generates new session token from the given refresh token.\n :param refresh_token: refresh token to generate from\n :param verbose: whether expiration time should be added to output\n \"\"\"\n\n platform = _get_platform(authenticated=False)\n session_token, expires_in = platform.generate_session_token(refresh_token)\n\n if verbose:\n click.echo(\"%s\\n\\n%s\" % (session_token, _color('YELLOW', \"Expires in %d seconds\" % expires_in)))\n else:\n click.echo(session_token)"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef plotcommand(cosmology='WMAP5', plotname=None):\n\n # Plot the c-M relation as a functon of redshift\n xarray = 10**(np.arange(1, 15, 0.2))\n yval = 'c'\n\n # Specify the redshift range\n zarray = np.arange(0, 5, 0.5)\n\n xtitle = r\"Halo Mass (M$_{sol}$)\"\n ytitle = r\"Concentration\"\n linelabel = \"z=\"\n\n fig = plt.figure()\n ax = fig.add_subplot(111)\n ax.set_xlabel(xtitle)\n ax.set_ylabel(ytitle)\n plt.ylim([2, 30])\n\n colors = cm.rainbow(np.linspace(0, 1, len(zarray)))\n\n for zind, zval in enumerate(zarray):\n output = commah.run(cosmology=cosmology, zi=zval, Mi=xarray)\n\n # Access the column yval from the data file\n yarray = output[yval].flatten()\n\n # Plot each line in turn with different colour\n ax.plot(xarray, yarray, label=linelabel+str(zval), color=colors[zind])\n # Overplot the D08 predictions in black\n ax.plot(xarray, commah.commah.cduffy(zval, xarray), color=\"black\")\n\n ax.set_xscale('log')\n ax.set_yscale('log')\n\n leg = ax.legend(loc=1)\n # Make box totally transparent\n leg.get_frame().set_alpha(0)\n leg.get_frame().set_edgecolor('white')\n for label in leg.get_texts():\n label.set_fontsize('small') # the font size\n for label in leg.get_lines():\n label.set_linewidth(4) # the legend line width\n\n if plotname:\n fig.tight_layout(pad=0.2)\n print(\"Plotting to '%s_CM_relation.png'\" % (plotname))\n fig.savefig(plotname+\"_CM_relation.png\", dpi=fig.dpi*5)\n else:\n plt.show()\n\n # Plot the c-z relation as a function of mass (so always Mz=M0)\n xarray = 10**(np.arange(0, 1, 0.05)) - 1\n yval = 'c'\n\n # Specify the mass range\n zarray = 10**np.arange(6, 14, 2)\n\n xtitle = r\"Redshift\"\n ytitle = r\"NFW Concentration\"\n linelabel = r\"log$_{10}$ M$_{z}$(M$_{sol}$)=\"\n\n fig = plt.figure()\n ax = fig.add_subplot(111)\n ax.set_xlabel(xtitle)\n ax.set_ylabel(ytitle)\n colors = cm.rainbow(np.linspace(0, 1, len(zarray)))\n\n for zind, zval in enumerate(zarray):\n output = commah.run(cosmology=cosmology, zi=xarray, Mi=zval)\n\n # Access the column yval from the data file\n yarray = output[yval].flatten()\n\n # Plot each line in turn with different colours\n ax.plot(xarray, yarray,\n label=linelabel+\"{0:.1f}\".format(np.log10(zval)),\n color=colors[zind],)\n\n leg = ax.legend(loc=1)\n # Make box totally transparent\n leg.get_frame().set_alpha(0)\n leg.get_frame().set_edgecolor('white')\n for label in leg.get_texts():\n label.set_fontsize('small') # the font size\n for label in leg.get_lines():\n label.set_linewidth(4) # the legend line width\n\n if plotname:\n fig.tight_layout(pad=0.2)\n print(\"Plotting to '%s_Cz_relation.png'\" % (plotname))\n fig.savefig(plotname+\"_Cz_relation.png\", dpi=fig.dpi*5)\n else:\n plt.show()\n\n # Plot the zf-z relation for different masses (so always Mz=M0)\n xarray = 10**(np.arange(0, 1, 0.05)) - 1\n yval = 'zf'\n\n # Specify the mass range\n zarray = 10**np.arange(6, 14, 2)\n\n xtitle = r\"Redshift\"\n ytitle = r\"Formation Redshift\"\n linelabel = r\"log$_{10}$ M$_{z}$(M$_{sol}$)=\"\n\n fig = plt.figure()\n ax = fig.add_subplot(111)\n ax.set_xlabel(xtitle)\n ax.set_ylabel(ytitle)\n colors = cm.rainbow(np.linspace(0, 1, len(zarray)))\n\n for zind, zval in enumerate(zarray):\n output = commah.run(cosmology=cosmology, zi=xarray, Mi=zval)\n\n yarray = output[yval].flatten()\n\n # Plot each line in turn with different colour\n ax.plot(xarray, yarray,\n label=linelabel+\"{0:.1f}\".format(np.log10(zval)),\n color=colors[zind],)\n\n leg = ax.legend(loc=2)\n # Make box totally transparent\n leg.get_frame().set_alpha(0)\n leg.get_frame().set_edgecolor('white')\n for label in leg.get_texts():\n label.set_fontsize('small') # the font size\n for label in leg.get_lines():\n label.set_linewidth(4) # the legend line width\n\n if plotname:\n fig.tight_layout(pad=0.2)\n print(\"Plotting to '%s_zfz_relation.png'\" % (plotname))\n fig.savefig(plotname+\"_zfz_relation.png\", dpi=fig.dpi*5)\n else:\n plt.show()\n\n # Plot the dM/dt-z relation for different masses (so always Mz=M0)\n xarray = 10**(np.arange(0, 1, 0.05)) - 1\n yval = 'dMdt'\n\n # Specify the mass range\n zarray = 10**np.arange(10, 14, 0.5)\n\n xtitle = r\"log$_{10}$ (1+z)\"\n ytitle = r\"log$_{10}$ Accretion Rate M$_{sol}$ yr$^{-1}$\"\n linelabel = r\"log$_{10}$ M$_z$(M$_{sol}$)=\"\n\n fig = plt.figure()\n ax = fig.add_subplot(111)\n ax.set_xlabel(xtitle)\n ax.set_ylabel(ytitle)\n colors = cm.rainbow(np.linspace(0, 1, len(zarray)))\n\n cosmo = commah.getcosmo(cosmology)\n for zind, zval in enumerate(zarray):\n output = commah.run(cosmology=cosmology, zi=xarray, Mi=zval,\n com=False, mah=True)\n\n yarray = output[yval].flatten()\n\n # Plot each line in turn with different colour\n ax.plot(np.log10(xarray+1.), np.log10(yarray),\n label=linelabel+\"{0:.1f}\".format(np.log10(zval)),\n color=colors[zind],)\n\n # Plot the semi-analytic approximate formula from Correa et al 2015b\n semianalytic_approx = 71.6 * (zval / 1e12) * (cosmo['h'] / 0.7) *\\\n (-0.24 + 0.75 * (xarray + 1)) * np.sqrt(\n cosmo['omega_M_0'] * (xarray + 1)**3 + cosmo['omega_lambda_0'])\n\n ax.plot(np.log10(xarray + 1), np.log10(semianalytic_approx),\n color='black')\n\n leg = ax.legend(loc=2)\n # Make box totally transparent\n leg.get_frame().set_alpha(0)\n leg.get_frame().set_edgecolor('white')\n for label in leg.get_texts():\n label.set_fontsize('small') # the font size\n for label in leg.get_lines():\n label.set_linewidth(4) # the legend line width\n\n if plotname:\n fig.tight_layout(pad=0.2)\n print(\"Plotting to '%s_dMdtz_relation.png'\" % (plotname))\n fig.savefig(plotname+\"_dMdtz_relation.png\", dpi=fig.dpi*5)\n else:\n plt.show()\n\n # Plot the dMdt-M relation as a function of redshift\n xarray = 10**(np.arange(10, 14, 0.5))\n yval = 'dMdt'\n\n # Specify the redshift range\n zarray = np.arange(0, 5, 0.5)\n\n xtitle = r\"Halo Mass M$_{sol}$\"\n ytitle = r\"Accretion Rate M$_{sol}$ yr$^{-1}$\"\n linelabel = \"z=\"\n\n fig = plt.figure()\n ax = fig.add_subplot(111)\n ax.set_xlabel(xtitle)\n ax.set_ylabel(ytitle)\n colors = cm.rainbow(np.linspace(0, 1, len(zarray)))\n\n for zind, zval in enumerate(zarray):\n output = commah.run(cosmology=cosmology, zi=zval, Mi=xarray,\n com=False, mah=True)\n\n yarray = output[yval].flatten()\n\n # Plot each line in turn with different colour\n ax.plot(xarray, yarray, label=linelabel+str(zval),\n color=colors[zind],)\n\n ax.set_xscale('log')\n ax.set_yscale('log')\n\n leg = ax.legend(loc=2)\n # Make box totally transparent\n leg.get_frame().set_alpha(0)\n leg.get_frame().set_edgecolor('white')\n for label in leg.get_texts():\n label.set_fontsize('small') # the font size\n for label in leg.get_lines():\n label.set_linewidth(4) # the legend line width\n\n if plotname:\n fig.tight_layout(pad=0.2)\n print(\"Plotting to '%s_MAH_M_relation.png'\" % (plotname))\n fig.savefig(plotname+\"_MAH_M_relation.png\", dpi=fig.dpi*5)\n else:\n plt.show()\n\n # Plot the (dM/M)dt-M relation as a function of redshift\n xarray = 10**(np.arange(10, 14, 0.5))\n yval = 'dMdt'\n\n # Specify the redshift range\n zarray = np.arange(0, 5, 0.5)\n\n xtitle = r\"Halo Mass M$_{sol}$\"\n ytitle = r\"Specific Accretion Rate yr$^{-1}$\"\n linelabel = \"z=\"\n\n fig = plt.figure()\n ax = fig.add_subplot(111)\n ax.set_xlabel(xtitle)\n ax.set_ylabel(ytitle)\n colors = cm.rainbow(np.linspace(0, 1, len(zarray)))\n\n for zind, zval in enumerate(zarray):\n output = commah.run(cosmology=cosmology, zi=zval, Mi=xarray,\n mah=True, com=False)\n\n yarray = output[yval].flatten()\n\n # Plot each line in turn with different colour\n ax.plot(xarray, yarray/xarray, label=linelabel+str(zval),\n color=colors[zind],)\n\n ax.set_xscale('log')\n ax.set_yscale('log')\n\n leg = ax.legend(loc=1)\n # Make box totally transparent\n leg.get_frame().set_alpha(0)\n leg.get_frame().set_edgecolor('white')\n for label in leg.get_texts():\n label.set_fontsize('small') # the font size\n for label in leg.get_lines():\n label.set_linewidth(4) # the legend line width\n\n if plotname:\n fig.tight_layout(pad=0.2)\n print(\"Plotting to '%s_specificMAH_M_relation.png'\" % (plotname))\n fig.savefig(plotname+\"_specificMAH_M_relation.png\", dpi=fig.dpi*5)\n else:\n plt.show()\n\n # Plot the Mz-z relation as a function of mass\n # (so mass is decreasing to zero as z-> inf)\n xarray = 10**(np.arange(0, 1, 0.05)) - 1\n yval = 'Mz'\n\n # Specify the mass range\n zarray = 10**np.arange(10, 14, 0.5)\n\n xtitle = r\"Redshift\"\n ytitle = r\"M(z) (M$_{sol}$)\"\n linelabel = r\"log$_{10}$ M$_{0}$(M$_{sol}$)=\"\n\n fig = plt.figure()\n ax = fig.add_subplot(111)\n ax.set_xlabel(xtitle)\n ax.set_ylabel(ytitle)\n colors = cm.rainbow(np.linspace(0, 1, len(zarray)))\n\n for zind, zval in enumerate(zarray):\n output = commah.run(cosmology=cosmology, zi=0, Mi=zval, z=xarray)\n\n yarray = output[yval].flatten()\n\n # Plot each line in turn with different colour\n ax.plot(xarray, yarray,\n label=linelabel+\"{0:.1f}\".format(np.log10(zval)),\n color=colors[zind],)\n\n ax.set_yscale('log')\n\n leg = ax.legend(loc=1)\n # Make box totally transparent\n leg.get_frame().set_alpha(0)\n leg.get_frame().set_edgecolor('white')\n for label in leg.get_texts():\n label.set_fontsize('small') # the font size\n for label in leg.get_lines():\n label.set_linewidth(4) # the legend line width\n\n if plotname:\n fig.tight_layout(pad=0.2)\n print(\"Plotting to '%s_Mzz_relation.png'\" % (plotname))\n fig.savefig(plotname+\"_Mzz_relation.png\", dpi=fig.dpi*5)\n else:\n plt.show()\n\n # Plot the Mz/M0-z relation as a function of mass\n xarray = 10**(np.arange(0, 1, 0.02)) - 1\n yval = 'Mz'\n\n # Specify the mass range\n zarray = 10**np.arange(10, 14, 0.5)\n\n xtitle = r\"Redshift\"\n ytitle = r\"log$_{10}$ M(z)/M$_{0}$\"\n linelabel = r\"log$_{10}$ M$_{0}$(M$_{sol}$)=\"\n\n fig = plt.figure()\n ax = fig.add_subplot(111)\n ax.set_xlabel(xtitle)\n ax.set_ylabel(ytitle)\n colors = cm.rainbow(np.linspace(0, 1, len(zarray)))\n\n for zind, zval in enumerate(zarray):\n output = commah.run(cosmology=cosmology, zi=0, Mi=zval, z=xarray)\n\n yarray = output[yval].flatten()\n\n # Plot each line in turn with different colour\n ax.plot(xarray, np.log10(yarray/zval),\n label=linelabel+\"{0:.1f}\".format(np.log10(zval)),\n color=colors[zind],)\n\n leg = ax.legend(loc=3)\n # Make box totally transparent\n leg.get_frame().set_alpha(0)\n leg.get_frame().set_edgecolor('white')\n for label in leg.get_texts():\n label.set_fontsize('small') # the font size\n for label in leg.get_lines():\n label.set_linewidth(4) # the legend line width\n\n if plotname:\n fig.tight_layout(pad=0.2)\n print(\"Plotting to '%s_MzM0z_relation.png'\" % (plotname))\n fig.savefig(plotname+\"_MzM0z_relation.png\", dpi=fig.dpi*5)\n else:\n plt.show()\n\n return(\"Done\")", "response": "Command to plot the c - M relation and the D08 predictions in black - order."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nfunction enhance Enhance the object with new item or enhanced items", "response": "def enhance(self):\n \"\"\" Function enhance\n Enhance the object with new item or enhanced items\n \"\"\"\n self.update({'puppetclasses':\n SubDict(self.api, self.objName,\n self.payloadObj, self.key,\n SubItemPuppetClasses)})\n self.update({'parameters':\n SubDict(self.api, self.objName,\n self.payloadObj, self.key,\n SubItemParameter)})\n self.update({'smart_class_parameters':\n SubDict(self.api, self.objName,\n self.payloadObj, self.key,\n ItemSmartClassParameter)})"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef add_transformers(line):\n '''Extract the transformers names from a line of code of the form\n from __experimental__ import transformer1 [,...]\n and adds them to the globally known dict\n '''\n assert FROM_EXPERIMENTAL.match(line)\n\n line = FROM_EXPERIMENTAL.sub(' ', line)\n # we now have: \" transformer1 [,...]\"\n line = line.split(\"#\")[0] # remove any end of line comments\n # and insert each transformer as an item in a list\n for trans in line.replace(' ', '').split(','):\n import_transformer(trans)", "response": "Extract the transformers names from a line of code of the form\n from __experimental__ import transformer1 [... ] and add them to the globally known dict\n "} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nimport a transformer and adds it to the globally known dict ATTRIBS.", "response": "def import_transformer(name):\n '''If needed, import a transformer, and adds it to the globally known dict\n The code inside a module where a transformer is defined should be\n standard Python code, which does not need any transformation.\n So, we disable the import hook, and let the normal module import\n do its job - which is faster and likely more reliable than our\n custom method.\n '''\n if name in transformers:\n return transformers[name]\n\n # We are adding a transformer built from normal/standard Python code.\n # As we are not performing transformations, we temporarily disable\n # our import hook, both to avoid potential problems AND because we\n # found that this resulted in much faster code.\n hook = sys.meta_path[0]\n sys.meta_path = sys.meta_path[1:]\n try:\n transformers[name] = __import__(name)\n # Some transformers are not allowed in the console.\n # If an attempt is made to activate one of them in the console,\n # we replace it by a transformer that does nothing and print a\n # message specific to that transformer as written in its module.\n if CONSOLE_ACTIVE:\n if hasattr(transformers[name], \"NO_CONSOLE\"):\n print(transformers[name].NO_CONSOLE)\n transformers[name] = NullTransformer()\n except ImportError:\n sys.stderr.write(\"Warning: Import Error in add_transformers: %s not found\\n\" % name)\n transformers[name] = NullTransformer()\n except Exception as e:\n sys.stderr.write(\"Unexpected exception in transforms.import_transformer%s\\n \" %\n e.__class__.__name__)\n finally:\n sys.meta_path.insert(0, hook) # restore import hook\n\n return transformers[name]"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nscan a source file for lines of the form from __experimental__ import transformer1...", "response": "def extract_transformers_from_source(source):\n '''Scan a source for lines of the form\n from __experimental__ import transformer1 [,...]\n identifying transformers to be used. Such line is passed to the\n add_transformer function, after which it is removed from the\n code to be executed.\n '''\n lines = source.split('\\n')\n linenumbers = []\n for number, line in enumerate(lines):\n if FROM_EXPERIMENTAL.match(line):\n add_transformers(line)\n linenumbers.insert(0, number)\n\n # drop the \"fake\" import from the source code\n for number in linenumbers:\n del lines[number]\n return '\\n'.join(lines)"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef remove_not_allowed_in_console():\n '''This function should be called from the console, when it starts.\n\n Some transformers are not allowed in the console and they could have\n been loaded prior to the console being activated. We effectively remove them\n and print an information message specific to that transformer\n as written in the transformer module.\n\n '''\n not_allowed_in_console = []\n if CONSOLE_ACTIVE:\n for name in transformers:\n tr_module = import_transformer(name)\n if hasattr(tr_module, \"NO_CONSOLE\"):\n not_allowed_in_console.append((name, tr_module))\n for name, tr_module in not_allowed_in_console:\n print(tr_module.NO_CONSOLE)\n # Note: we do not remove them, so as to avoid seeing the\n # information message displayed again if an attempt is\n # made to re-import them from a console instruction.\n transformers[name] = NullTransformer()", "response": "This function is called from the console when it starts."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef transform(source):\n '''Used to convert the source code, making use of known transformers.\n\n \"transformers\" are modules which must contain a function\n\n transform_source(source)\n\n which returns a tranformed source.\n Some transformers (for example, those found in the standard library\n module lib2to3) cannot cope with non-standard syntax; as a result, they\n may fail during a first attempt. We keep track of all failing\n transformers and keep retrying them until either they all succeeded\n or a fixed set of them fails twice in a row.\n '''\n source = extract_transformers_from_source(source)\n\n # Some transformer fail when multiple non-Python constructs\n # are present. So, we loop multiple times keeping track of\n # which transformations have been unsuccessfully performed.\n not_done = transformers\n while True:\n failed = {}\n for name in not_done:\n tr_module = import_transformer(name)\n try:\n source = tr_module.transform_source(source)\n except Exception as e:\n failed[name] = tr_module\n # from traceback import print_exc\n # print(\"Unexpected exception in transforms.transform\",\n # e.__class__.__name__)\n # print_exc()\n\n if not failed:\n break\n # Insanity is doing the same Tting over and overaAgain and\n # expecting different results ...\n # If the exact same set of transformations are not performed\n # twice in a row, there is no point in trying out a third time.\n if failed == not_done:\n print(\"Warning: the following transforms could not be done:\")\n for key in failed:\n print(key)\n break\n not_done = failed # attempt another pass\n\n return source", "response": "This function transforms the source code by calling all known transformers and then returning the transformed source."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _match(self, request, response):\n is_html = 'text/html' in response.get('Content-Type', '')\n if is_html and hasattr(response, 'rendered_content'):\n correct_path = PATH_MATCHER.match(request.path) is not None\n not_included = self.include_flag not in response.rendered_content\n return correct_path and not_included\n return False", "response": "Match all requests and responses that satisfy the conditions"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _chosen_css(self):\n css = render_to_string(self.css_template, {})\n for sprite in self.chosen_sprites: # rewrite path to sprites in the css\n css = css.replace(sprite, settings.STATIC_URL + \"img/\" + sprite)\n return css", "response": "Read the minified CSS file including STATIC_URL in the references\n to the sprite images."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nembed Chosen. js directly in html of the response.", "response": "def _embed(self, request, response):\n \"\"\"Embed Chosen.js directly in html of the response.\"\"\"\n if self._match(request, response):\n # Render the and the