{"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef date_range(start_date, end_date=None, num=None, delta=None):\n if not delta:\n return []\n if end_date and start_date > end_date:\n raise Exception(\"Wait. start_date needs to be before end_date\")\n if end_date and num:\n raise Exception(\"Wait. Either specify end_date OR num\")\n if not end_date and not num:\n end_date = timezone.utcnow()\n\n delta_iscron = False\n tz = start_date.tzinfo\n if isinstance(delta, six.string_types):\n delta_iscron = True\n start_date = timezone.make_naive(start_date, tz)\n cron = croniter(delta, start_date)\n elif isinstance(delta, timedelta):\n delta = abs(delta)\n dates = []\n if end_date:\n if timezone.is_naive(start_date):\n end_date = timezone.make_naive(end_date, tz)\n while start_date <= end_date:\n if timezone.is_naive(start_date):\n dates.append(timezone.make_aware(start_date, tz))\n else:\n dates.append(start_date)\n\n if delta_iscron:\n start_date = cron.get_next(datetime)\n else:\n start_date += delta\n else:\n for _ in range(abs(num)):\n if timezone.is_naive(start_date):\n dates.append(timezone.make_aware(start_date, tz))\n else:\n dates.append(start_date)\n\n if delta_iscron:\n if num > 0:\n start_date = cron.get_next(datetime)\n else:\n start_date = cron.get_prev(datetime)\n else:\n if num > 0:\n start_date += delta\n else:\n start_date -= delta\n return sorted(dates)", "response": "Return a list of dates based on a start date and end date."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef round_time(dt, delta, start_date=timezone.make_aware(datetime.min)):\n\n if isinstance(delta, six.string_types):\n # It's cron based, so it's easy\n tz = start_date.tzinfo\n start_date = timezone.make_naive(start_date, tz)\n cron = croniter(delta, start_date)\n prev = cron.get_prev(datetime)\n if prev == start_date:\n return timezone.make_aware(start_date, tz)\n else:\n return timezone.make_aware(prev, tz)\n\n # Ignore the microseconds of dt\n dt -= timedelta(microseconds=dt.microsecond)\n\n # We are looking for a datetime in the form start_date + i * delta\n # which is as close as possible to dt. Since delta could be a relative\n # delta we don't know its exact length in seconds so we cannot rely on\n # division to find i. Instead we employ a binary search algorithm, first\n # finding an upper and lower limit and then disecting the interval until\n # we have found the closest match.\n\n # We first search an upper limit for i for which start_date + upper * delta\n # exceeds dt.\n upper = 1\n while start_date + upper * delta < dt:\n # To speed up finding an upper limit we grow this exponentially by a\n # factor of 2\n upper *= 2\n\n # Since upper is the first value for which start_date + upper * delta\n # exceeds dt, upper // 2 is below dt and therefore forms a lower limited\n # for the i we are looking for\n lower = upper // 2\n\n # We now continue to intersect the interval between\n # start_date + lower * delta and start_date + upper * delta\n # until we find the closest value\n while True:\n # Invariant: start + lower * delta < dt <= start + upper * delta\n # If start_date + (lower + 1)*delta exceeds dt, then either lower or\n # lower+1 has to be the solution we are searching for\n if start_date + (lower + 1) * delta >= dt:\n # Check if start_date + (lower + 1)*delta or\n # start_date + lower*delta is closer to dt and return the solution\n if (start_date + (lower + 1) * delta) - dt <= dt - (start_date + lower * delta):\n return start_date + (lower + 1) * delta\n else:\n return start_date + lower * delta\n\n # We intersect the interval and either replace the lower or upper\n # limit with the candidate\n candidate = lower + (upper - lower) // 2\n if start_date + candidate * delta >= dt:\n upper = candidate\n else:\n lower = candidate", "response": "Returns the datetime of the form start_date + delta which is closest to dt."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef infer_time_unit(time_seconds_arr):\n if len(time_seconds_arr) == 0:\n return 'hours'\n max_time_seconds = max(time_seconds_arr)\n if max_time_seconds <= 60 * 2:\n return 'seconds'\n elif max_time_seconds <= 60 * 60 * 2:\n return 'minutes'\n elif max_time_seconds <= 24 * 60 * 60 * 2:\n return 'hours'\n else:\n return 'days'", "response": "Infer the most appropriate time unit for an array of time durations."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nconverting an array of time durations in seconds to the specified time unit.", "response": "def scale_time_units(time_seconds_arr, unit):\n \"\"\"\n Convert an array of time durations in seconds to the specified time unit.\n \"\"\"\n if unit == 'minutes':\n return list(map(lambda x: x * 1.0 / 60, time_seconds_arr))\n elif unit == 'hours':\n return list(map(lambda x: x * 1.0 / (60 * 60), time_seconds_arr))\n elif unit == 'days':\n return list(map(lambda x: x * 1.0 / (24 * 60 * 60), time_seconds_arr))\n return time_seconds_arr"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef days_ago(n, hour=0, minute=0, second=0, microsecond=0):\n today = timezone.utcnow().replace(\n hour=hour,\n minute=minute,\n second=second,\n microsecond=microsecond)\n return today - timedelta(days=n)", "response": "Returns a datetime object representing n days ago. By default the time is midnight."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nreturns a list of Dag Runs for a specific DAG ID.", "response": "def get_dag_runs(dag_id, state=None):\n \"\"\"\n Returns a list of Dag Runs for a specific DAG ID.\n :param dag_id: String identifier of a DAG\n :param state: queued|running|success...\n :return: List of DAG runs of a DAG with requested state,\n or all runs if the state is not specified\n \"\"\"\n dagbag = DagBag()\n\n # Check DAG exists.\n if dag_id not in dagbag.dags:\n error_message = \"Dag id {} not found\".format(dag_id)\n raise AirflowException(error_message)\n\n dag_runs = list()\n state = state.lower() if state else None\n for run in DagRun.find(dag_id=dag_id, state=state):\n dag_runs.append({\n 'id': run.id,\n 'run_id': run.run_id,\n 'state': run.state,\n 'dag_id': run.dag_id,\n 'execution_date': run.execution_date.isoformat(),\n 'start_date': ((run.start_date or '') and\n run.start_date.isoformat()),\n 'dag_run_url': url_for('Airflow.graph', dag_id=run.dag_id,\n execution_date=run.execution_date)\n })\n\n return dag_runs"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\ninitializing the role with the permissions and related view - menus.", "response": "def init_role(self, role_name, role_vms, role_perms):\n \"\"\"\n Initialize the role with the permissions and related view-menus.\n\n :param role_name:\n :param role_vms:\n :param role_perms:\n :return:\n \"\"\"\n pvms = self.get_session.query(sqla_models.PermissionView).all()\n pvms = [p for p in pvms if p.permission and p.view_menu]\n\n role = self.find_role(role_name)\n if not role:\n role = self.add_role(role_name)\n\n if len(role.permissions) == 0:\n self.log.info('Initializing permissions for role:%s in the database.', role_name)\n role_pvms = set()\n for pvm in pvms:\n if pvm.view_menu.name in role_vms and pvm.permission.name in role_perms:\n role_pvms.add(pvm)\n role.permissions = list(role_pvms)\n self.get_session.merge(role)\n self.get_session.commit()\n else:\n self.log.debug('Existing permissions for the role:%s '\n 'within the database will persist.', role_name)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef delete_role(self, role_name):\n session = self.get_session\n role = session.query(sqla_models.Role)\\\n .filter(sqla_models.Role.name == role_name)\\\n .first()\n if role:\n self.log.info(\"Deleting role '%s'\", role_name)\n session.delete(role)\n session.commit()\n else:\n raise AirflowException(\"Role named '{}' does not exist\".format(\n role_name))", "response": "Delete the given role in the ab_role table."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nget all the roles associated with the user.", "response": "def get_user_roles(self, user=None):\n \"\"\"\n Get all the roles associated with the user.\n\n :param user: the ab_user in FAB model.\n :return: a list of roles associated with the user.\n \"\"\"\n if user is None:\n user = g.user\n if user.is_anonymous:\n public_role = appbuilder.config.get('AUTH_ROLE_PUBLIC')\n return [appbuilder.security_manager.find_role(public_role)] \\\n if public_role else []\n return user.roles"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nreturn a set of tuples with the perm name and view menu name", "response": "def get_all_permissions_views(self):\n \"\"\"\n Returns a set of tuples with the perm name and view menu name\n \"\"\"\n perms_views = set()\n for role in self.get_user_roles():\n perms_views.update({(perm_view.permission.name, perm_view.view_menu.name)\n for perm_view in role.permissions})\n return perms_views"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nreturning a set of all dags that the user can access to.", "response": "def get_accessible_dag_ids(self, username=None):\n \"\"\"\n Return a set of dags that user has access to(either read or write).\n\n :param username: Name of the user.\n :return: A set of dag ids that the user could access.\n \"\"\"\n if not username:\n username = g.user\n\n if username.is_anonymous or 'Public' in username.roles:\n # return an empty set if the role is public\n return set()\n\n roles = {role.name for role in username.roles}\n if {'Admin', 'Viewer', 'User', 'Op'} & roles:\n return self.DAG_VMS\n\n user_perms_views = self.get_all_permissions_views()\n # return a set of all dags that the user could access\n return set([view for perm, view in user_perms_views if perm in self.DAG_PERMS])"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef has_access(self, permission, view_name, user=None):\n if not user:\n user = g.user\n if user.is_anonymous:\n return self.is_item_public(permission, view_name)\n return self._has_view_access(user, permission, view_name)", "response": "Verify whether a given user can perform certain permission on the dag_id."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _has_role(self, role_name_or_list):\n if not isinstance(role_name_or_list, list):\n role_name_or_list = [role_name_or_list]\n return any(\n [r.name in role_name_or_list for r in self.get_user_roles()])", "response": "Returns True if the user has the role name_or_list."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef _has_perm(self, permission_name, view_menu_name):\n if hasattr(self, 'perms'):\n if (permission_name, view_menu_name) in self.perms:\n return True\n # rebuild the permissions set\n self._get_and_cache_perms()\n return (permission_name, view_menu_name) in self.perms", "response": "Returns True if the user has permission with the given name."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef clean_perms(self):\n self.log.debug('Cleaning faulty perms')\n sesh = self.get_session\n pvms = (\n sesh.query(sqla_models.PermissionView)\n .filter(or_(\n sqla_models.PermissionView.permission == None, # NOQA\n sqla_models.PermissionView.view_menu == None, # NOQA\n ))\n )\n deleted_count = pvms.delete()\n sesh.commit()\n if deleted_count:\n self.log.info('Deleted %s faulty permissions', deleted_count)", "response": "FAB leaves faulty permissions that need to be cleaned up"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nmerging the permission and view_menu into the permission_view_role.", "response": "def _merge_perm(self, permission_name, view_menu_name):\n \"\"\"\n Add the new permission , view_menu to ab_permission_view_role if not exists.\n It will add the related entry to ab_permission\n and ab_view_menu two meta tables as well.\n\n :param permission_name: Name of the permission.\n :type permission_name: str\n :param view_menu_name: Name of the view-menu\n :type view_menu_name: str\n :return:\n \"\"\"\n permission = self.find_permission(permission_name)\n view_menu = self.find_view_menu(view_menu_name)\n pv = None\n if permission and view_menu:\n pv = self.get_session.query(self.permissionview_model).filter_by(\n permission=permission, view_menu=view_menu).first()\n if not pv and permission_name and view_menu_name:\n self.add_permission_view_menu(permission_name, view_menu_name)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef create_custom_dag_permission_view(self, session=None):\n self.log.debug('Fetching a set of all permission, view_menu from FAB meta-table')\n\n def merge_pv(perm, view_menu):\n \"\"\"Create permission view menu only if it doesn't exist\"\"\"\n if view_menu and perm and (view_menu, perm) not in all_pvs:\n self._merge_perm(perm, view_menu)\n\n all_pvs = set()\n for pv in self.get_session.query(self.permissionview_model).all():\n if pv.permission and pv.view_menu:\n all_pvs.add((pv.permission.name, pv.view_menu.name))\n\n # Get all the active / paused dags and insert them into a set\n all_dags_models = session.query(models.DagModel)\\\n .filter(or_(models.DagModel.is_active, models.DagModel.is_paused))\\\n .filter(~models.DagModel.is_subdag).all()\n\n # create can_dag_edit and can_dag_read permissions for every dag(vm)\n for dag in all_dags_models:\n for perm in self.DAG_PERMS:\n merge_pv(perm, dag.dag_id)\n\n # for all the dag-level role, add the permission of viewer\n # with the dag view to ab_permission_view\n all_roles = self.get_all_roles()\n user_role = self.find_role('User')\n\n dag_role = [role for role in all_roles if role.name not in EXISTING_ROLES]\n update_perm_views = []\n\n # need to remove all_dag vm from all the existing view-menus\n dag_vm = self.find_view_menu('all_dags')\n ab_perm_view_role = sqla_models.assoc_permissionview_role\n perm_view = self.permissionview_model\n view_menu = self.viewmenu_model\n\n all_perm_view_by_user = session.query(ab_perm_view_role)\\\n .join(perm_view, perm_view.id == ab_perm_view_role\n .columns.permission_view_id)\\\n .filter(ab_perm_view_role.columns.role_id == user_role.id)\\\n .join(view_menu)\\\n .filter(perm_view.view_menu_id != dag_vm.id)\n all_perm_views = set([role.permission_view_id for role in all_perm_view_by_user])\n\n for role in dag_role:\n # Get all the perm-view of the role\n existing_perm_view_by_user = self.get_session.query(ab_perm_view_role)\\\n .filter(ab_perm_view_role.columns.role_id == role.id)\n\n existing_perms_views = set([pv.permission_view_id\n for pv in existing_perm_view_by_user])\n missing_perm_views = all_perm_views - existing_perms_views\n\n for perm_view_id in missing_perm_views:\n update_perm_views.append({'permission_view_id': perm_view_id,\n 'role_id': role.id})\n\n if update_perm_views:\n self.get_session.execute(ab_perm_view_role.insert(), update_perm_views)\n self.get_session.commit()", "response": "Create a custom permission view for all the dags."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef update_admin_perm_view(self):\n pvms = self.get_session.query(sqla_models.PermissionView).all()\n pvms = [p for p in pvms if p.permission and p.view_menu]\n\n admin = self.find_role('Admin')\n admin.permissions = list(set(admin.permissions) | set(pvms))\n\n self.get_session.commit()", "response": "Update the admin permissions table for the admin role."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nsynchronizing user roles with the default and custom roles.", "response": "def sync_roles(self):\n \"\"\"\n 1. Init the default role(Admin, Viewer, User, Op, public)\n with related permissions.\n 2. Init the custom role(dag-user) with related permissions.\n\n :return: None.\n \"\"\"\n self.log.debug('Start syncing user roles.')\n # Create global all-dag VM\n self.create_perm_vm_for_all_dag()\n\n # Create default user role.\n for config in self.ROLE_CONFIGS:\n role = config['role']\n vms = config['vms']\n perms = config['perms']\n self.init_role(role, vms, perms)\n self.create_custom_dag_permission_view()\n\n # init existing roles, the rest role could be created through UI.\n self.update_admin_perm_view()\n self.clean_perms()"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef sync_perm_for_dag(self, dag_id, access_control=None):\n for dag_perm in self.DAG_PERMS:\n perm_on_dag = self.find_permission_view_menu(dag_perm, dag_id)\n if perm_on_dag is None:\n self.add_permission_view_menu(dag_perm, dag_id)\n\n if access_control:\n self._sync_dag_view_permissions(dag_id, access_control)", "response": "Sync permissions for a dag."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef _sync_dag_view_permissions(self, dag_id, access_control):\n def _get_or_create_dag_permission(perm_name):\n dag_perm = self.find_permission_view_menu(perm_name, dag_id)\n if not dag_perm:\n self.log.info(\n \"Creating new permission '%s' on view '%s'\",\n perm_name, dag_id\n )\n dag_perm = self.add_permission_view_menu(perm_name, dag_id)\n\n return dag_perm\n\n def _revoke_stale_permissions(dag_view):\n existing_dag_perms = self.find_permissions_view_menu(dag_view)\n for perm in existing_dag_perms:\n non_admin_roles = [role for role in perm.role\n if role.name != 'Admin']\n for role in non_admin_roles:\n target_perms_for_role = access_control.get(role.name, {})\n if perm.permission.name not in target_perms_for_role:\n self.log.info(\n \"Revoking '%s' on DAG '%s' for role '%s'\",\n perm.permission, dag_id, role.name\n )\n self.del_permission_role(role, perm)\n\n dag_view = self.find_view_menu(dag_id)\n if dag_view:\n _revoke_stale_permissions(dag_view)\n\n for rolename, perms in access_control.items():\n role = self.find_role(rolename)\n if not role:\n raise AirflowException(\n \"The access_control mapping for DAG '{}' includes a role \"\n \"named '{}', but that role does not exist\".format(\n dag_id,\n rolename))\n\n perms = set(perms)\n invalid_perms = perms - self.DAG_PERMS\n if invalid_perms:\n raise AirflowException(\n \"The access_control map for DAG '{}' includes the following \"\n \"invalid permissions: {}; The set of valid permissions \"\n \"is: {}\".format(dag_id,\n (perms - self.DAG_PERMS),\n self.DAG_PERMS))\n\n for perm_name in perms:\n dag_perm = _get_or_create_dag_permission(perm_name)\n self.add_permission_role(role, dag_perm)", "response": "Synchronizes the permissions of a DAG to the view model."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\ncreates perm - vm for all - dags.", "response": "def create_perm_vm_for_all_dag(self):\n \"\"\"\n Create perm-vm if not exist and insert into FAB security model for all-dags.\n \"\"\"\n # create perm for global logical dag\n for dag_vm in self.DAG_VMS:\n for perm in self.DAG_PERMS:\n self._merge_perm(permission_name=perm,\n view_menu_name=dag_vm)"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef get_fernet():\n global _fernet\n log = LoggingMixin().log\n\n if _fernet:\n return _fernet\n try:\n from cryptography.fernet import Fernet, MultiFernet, InvalidToken\n global InvalidFernetToken\n InvalidFernetToken = InvalidToken\n\n except BuiltinImportError:\n log.warning(\n \"cryptography not found - values will not be stored encrypted.\"\n )\n _fernet = NullFernet()\n return _fernet\n\n try:\n fernet_key = configuration.conf.get('core', 'FERNET_KEY')\n if not fernet_key:\n log.warning(\n \"empty cryptography key - values will not be stored encrypted.\"\n )\n _fernet = NullFernet()\n else:\n _fernet = MultiFernet([\n Fernet(fernet_part.encode('utf-8'))\n for fernet_part in fernet_key.split(',')\n ])\n _fernet.is_encrypted = True\n except (ValueError, TypeError) as ve:\n raise AirflowException(\"Could not create Fernet object: {}\".format(ve))\n\n return _fernet", "response": "Returns a Fernet object for the given cryptography key."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef poke(self, context):\n if '.' in self.table_name:\n self.database_name, self.table_name = self.table_name.split('.')\n self.log.info(\n 'Poking for table %s. %s, expression %s', self.database_name, self.table_name, self.expression\n )\n\n return self.get_hook().check_for_partition(\n self.database_name, self.table_name, self.expression)", "response": "Checks if the partition exists in the AWS Glue Catalog table"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef get_hook(self):\n if not hasattr(self, 'hook'):\n from airflow.contrib.hooks.aws_glue_catalog_hook import AwsGlueCatalogHook\n self.hook = AwsGlueCatalogHook(\n aws_conn_id=self.aws_conn_id,\n region_name=self.region_name)\n\n return self.hook", "response": "Gets the AwsGlueCatalogHook object"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\ncheck for message on subscribed queue and write to xcom", "response": "def poke(self, context):\n \"\"\"\n Check for message on subscribed queue and write to xcom the message with key ``messages``\n\n :param context: the context object\n :type context: dict\n :return: ``True`` if message is available or ``False``\n \"\"\"\n\n sqs_hook = SQSHook(aws_conn_id=self.aws_conn_id)\n sqs_conn = sqs_hook.get_conn()\n\n self.log.info('SQSSensor checking for message on queue: %s', self.sqs_queue)\n\n messages = sqs_conn.receive_message(QueueUrl=self.sqs_queue,\n MaxNumberOfMessages=self.max_messages,\n WaitTimeSeconds=self.wait_time_seconds)\n\n self.log.info(\"reveived message %s\", str(messages))\n\n if 'Messages' in messages and len(messages['Messages']) > 0:\n\n entries = [{'Id': message['MessageId'], 'ReceiptHandle': message['ReceiptHandle']}\n for message in messages['Messages']]\n\n result = sqs_conn.delete_message_batch(QueueUrl=self.sqs_queue,\n Entries=entries)\n\n if 'Successful' in result:\n context['ti'].xcom_push(key='messages', value=messages)\n return True\n else:\n raise AirflowException(\n 'Delete SQS Messages failed ' + str(result) + ' for messages ' + str(messages))\n\n return False"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef tmp_configuration_copy(chmod=0o600):\n cfg_dict = conf.as_dict(display_sensitive=True, raw=True)\n temp_fd, cfg_path = mkstemp()\n\n with os.fdopen(temp_fd, 'w') as temp_file:\n if chmod is not None:\n os.fchmod(temp_fd, chmod)\n json.dump(cfg_dict, temp_file)\n\n return cfg_path", "response": "Returns a path to a temporary file including a full copy of the configuration\n settings."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef get_conn(self):\n # When using HAClient, proxy_user must be the same, so is ok to always\n # take the first.\n effective_user = self.proxy_user\n autoconfig = self.autoconfig\n use_sasl = configuration.conf.get('core', 'security') == 'kerberos'\n\n try:\n connections = self.get_connections(self.hdfs_conn_id)\n\n if not effective_user:\n effective_user = connections[0].login\n if not autoconfig:\n autoconfig = connections[0].extra_dejson.get('autoconfig',\n False)\n hdfs_namenode_principal = connections[0].extra_dejson.get(\n 'hdfs_namenode_principal')\n except AirflowException:\n if not autoconfig:\n raise\n\n if autoconfig:\n # will read config info from $HADOOP_HOME conf files\n client = AutoConfigClient(effective_user=effective_user,\n use_sasl=use_sasl)\n elif len(connections) == 1:\n client = Client(connections[0].host, connections[0].port,\n effective_user=effective_user, use_sasl=use_sasl,\n hdfs_namenode_principal=hdfs_namenode_principal)\n elif len(connections) > 1:\n nn = [Namenode(conn.host, conn.port) for conn in connections]\n client = HAClient(nn, effective_user=effective_user,\n use_sasl=use_sasl,\n hdfs_namenode_principal=hdfs_namenode_principal)\n else:\n raise HDFSHookException(\"conn_id doesn't exist in the repository \"\n \"and autoconfig is not specified\")\n\n return client", "response": "Returns a snakebite HDFSClient object."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nestablish a connection depending on the security mode set via config or environment variable.", "response": "def get_conn(self):\n \"\"\"\n Establishes a connection depending on the security mode set via config or environment variable.\n\n :return: a hdfscli InsecureClient or KerberosClient object.\n :rtype: hdfs.InsecureClient or hdfs.ext.kerberos.KerberosClient\n \"\"\"\n connections = self.get_connections(self.webhdfs_conn_id)\n\n for connection in connections:\n try:\n self.log.debug('Trying namenode %s', connection.host)\n client = self._get_client(connection)\n client.status('/')\n self.log.debug('Using namenode %s for hook', connection.host)\n return client\n except HdfsError as hdfs_error:\n self.log.debug('Read operation on namenode %s failed with error: %s',\n connection.host, hdfs_error)\n\n hosts = [connection.host for connection in connections]\n error_message = 'Read operations failed on the namenodes below:\\n{hosts}'.format(\n hosts='\\n'.join(hosts))\n raise AirflowWebHDFSHookException(error_message)"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef check_for_path(self, hdfs_path):\n conn = self.get_conn()\n\n status = conn.status(hdfs_path, strict=False)\n return bool(status)", "response": "Checks if a path exists in HDFS by querying FileStatus. ncache_directory"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nestablishing a connection to pinot broker through pinot dbqpi.", "response": "def get_conn(self):\n \"\"\"\n Establish a connection to pinot broker through pinot dbqpi.\n \"\"\"\n conn = self.get_connection(self.pinot_broker_conn_id)\n pinot_broker_conn = connect(\n host=conn.host,\n port=conn.port,\n path=conn.extra_dejson.get('endpoint', '/pql'),\n scheme=conn.extra_dejson.get('schema', 'http')\n )\n self.log.info('Get the connection to pinot '\n 'broker on {host}'.format(host=conn.host))\n return pinot_broker_conn"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nget the connection uri for the current object.", "response": "def get_uri(self):\n \"\"\"\n Get the connection uri for pinot broker.\n\n e.g: http://localhost:9000/pql\n \"\"\"\n conn = self.get_connection(getattr(self, self.conn_name_attr))\n host = conn.host\n if conn.port is not None:\n host += ':{port}'.format(port=conn.port)\n conn_type = 'http' if not conn.conn_type else conn.conn_type\n endpoint = conn.extra_dejson.get('endpoint', 'pql')\n return '{conn_type}://{host}/{endpoint}'.format(\n conn_type=conn_type, host=host, endpoint=endpoint)"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef get_records(self, sql):\n with self.get_conn() as cur:\n cur.execute(sql)\n return cur.fetchall()", "response": "Executes the sql and returns a set of records."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nexecuting the sql and returns the first result.", "response": "def get_first(self, sql):\n \"\"\"\n Executes the sql and returns the first resulting row.\n\n :param sql: the sql statement to be executed (str) or a list of\n sql statements to execute\n :type sql: str or list\n \"\"\"\n with self.get_conn() as cur:\n cur.execute(sql)\n return cur.fetchone()"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef smart_truncate(string, max_length=0, word_boundary=False, separator=' ', save_order=False):\n\n string = string.strip(separator)\n\n if not max_length:\n return string\n\n if len(string) < max_length:\n return string\n\n if not word_boundary:\n return string[:max_length].strip(separator)\n\n if separator not in string:\n return string[:max_length]\n\n truncated = ''\n for word in string.split(separator):\n if word:\n next_len = len(truncated) + len(word)\n if next_len < max_length:\n truncated += '{0}{1}'.format(word, separator)\n elif next_len == max_length:\n truncated += '{0}'.format(word)\n break\n else:\n if save_order:\n break\n if not truncated: # pragma: no cover\n truncated = string[:max_length]\n return truncated.strip(separator)", "response": "Truncates a string to a maximum length."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef slugify(text, entities=True, decimal=True, hexadecimal=True, max_length=0, word_boundary=False,\n separator=DEFAULT_SEPARATOR, save_order=False, stopwords=(), regex_pattern=None, lowercase=True,\n replacements=()):\n \"\"\"\n Make a slug from the given text.\n :param text (str): initial text\n :param entities (bool):\n :param decimal (bool):\n :param hexadecimal (bool):\n :param max_length (int): output string length\n :param word_boundary (bool):\n :param save_order (bool): if parameter is True and max_length > 0 return whole words in the initial order\n :param separator (str): separator between words\n :param stopwords (iterable): words to discount\n :param regex_pattern (str): regex pattern for allowed characters\n :param lowercase (bool): activate case sensitivity by setting it to False\n :param replacements (iterable): list of replacement rules e.g. [['|', 'or'], ['%', 'percent']]\n :return (str):\n \"\"\"\n\n # user-specific replacements\n if replacements:\n for old, new in replacements:\n text = text.replace(old, new)\n\n # ensure text is unicode\n if not isinstance(text, _unicode_type):\n text = _unicode(text, 'utf-8', 'ignore')\n\n # replace quotes with dashes - pre-process\n text = QUOTE_PATTERN.sub(DEFAULT_SEPARATOR, text)\n\n # decode unicode\n text = unidecode.unidecode(text)\n\n # ensure text is still in unicode\n if not isinstance(text, _unicode_type):\n text = _unicode(text, 'utf-8', 'ignore')\n\n # character entity reference\n if entities:\n text = CHAR_ENTITY_PATTERN.sub(lambda m: unichr(name2codepoint[m.group(1)]), text)\n\n # decimal character reference\n if decimal:\n try:\n text = DECIMAL_PATTERN.sub(lambda m: unichr(int(m.group(1))), text)\n except Exception:\n pass\n\n # hexadecimal character reference\n if hexadecimal:\n try:\n text = HEX_PATTERN.sub(lambda m: unichr(int(m.group(1), 16)), text)\n except Exception:\n pass\n\n # translate\n text = unicodedata.normalize('NFKD', text)\n\n # make the text lowercase (optional)\n if lowercase:\n text = text.lower()\n\n # remove generated quotes -- post-process\n text = QUOTE_PATTERN.sub('', text)\n\n # cleanup numbers\n text = NUMBERS_PATTERN.sub('', text)\n\n # replace all other unwanted characters\n if lowercase:\n pattern = regex_pattern or ALLOWED_CHARS_PATTERN\n else:\n pattern = regex_pattern or ALLOWED_CHARS_PATTERN_WITH_UPPERCASE\n text = re.sub(pattern, DEFAULT_SEPARATOR, text)\n\n # remove redundant\n text = DUPLICATE_DASH_PATTERN.sub(DEFAULT_SEPARATOR, text).strip(DEFAULT_SEPARATOR)\n\n # remove stopwords\n if stopwords:\n if lowercase:\n stopwords_lower = [s.lower() for s in stopwords]\n words = [w for w in text.split(DEFAULT_SEPARATOR) if w not in stopwords_lower]\n else:\n words = [w for w in text.split(DEFAULT_SEPARATOR) if w not in stopwords]\n text = DEFAULT_SEPARATOR.join(words)\n\n # finalize user-specific replacements\n if replacements:\n for old, new in replacements:\n text = text.replace(old, new)\n\n # smart truncate if requested\n if max_length > 0:\n text = smart_truncate(text, max_length, word_boundary, DEFAULT_SEPARATOR, save_order)\n\n if separator != DEFAULT_SEPARATOR:\n text = text.replace(DEFAULT_SEPARATOR, separator)\n\n return text", "response": "Make a slug from the given text."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nstoring an XCom value in the database.", "response": "def set(\n cls,\n key,\n value,\n execution_date,\n task_id,\n dag_id,\n session=None):\n \"\"\"\n Store an XCom value.\n TODO: \"pickling\" has been deprecated and JSON is preferred.\n \"pickling\" will be removed in Airflow 2.0.\n\n :return: None\n \"\"\"\n session.expunge_all()\n\n enable_pickling = configuration.getboolean('core', 'enable_xcom_pickling')\n if enable_pickling:\n value = pickle.dumps(value)\n else:\n try:\n value = json.dumps(value).encode('UTF-8')\n except ValueError:\n log = LoggingMixin().log\n log.error(\"Could not serialize the XCOM value into JSON. \"\n \"If you are using pickles instead of JSON \"\n \"for XCOM, then you need to enable pickle \"\n \"support for XCOM in your airflow config.\")\n raise\n\n # remove any duplicate XComs\n session.query(cls).filter(\n cls.key == key,\n cls.execution_date == execution_date,\n cls.task_id == task_id,\n cls.dag_id == dag_id).delete()\n\n session.commit()\n\n # insert new XCom\n session.add(XCom(\n key=key,\n value=value,\n execution_date=execution_date,\n task_id=task_id,\n dag_id=dag_id))\n\n session.commit()"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nretrieve an XCom value from the database optionally meeting certain criteria.", "response": "def get_one(cls,\n execution_date,\n key=None,\n task_id=None,\n dag_id=None,\n include_prior_dates=False,\n session=None):\n \"\"\"\n Retrieve an XCom value, optionally meeting certain criteria.\n TODO: \"pickling\" has been deprecated and JSON is preferred.\n \"pickling\" will be removed in Airflow 2.0.\n\n :return: XCom value\n \"\"\"\n filters = []\n if key:\n filters.append(cls.key == key)\n if task_id:\n filters.append(cls.task_id == task_id)\n if dag_id:\n filters.append(cls.dag_id == dag_id)\n if include_prior_dates:\n filters.append(cls.execution_date <= execution_date)\n else:\n filters.append(cls.execution_date == execution_date)\n\n query = (\n session.query(cls.value).filter(and_(*filters))\n .order_by(cls.execution_date.desc(), cls.timestamp.desc()))\n\n result = query.first()\n if result:\n enable_pickling = configuration.getboolean('core', 'enable_xcom_pickling')\n if enable_pickling:\n return pickle.loads(result.value)\n else:\n try:\n return json.loads(result.value.decode('UTF-8'))\n except ValueError:\n log = LoggingMixin().log\n log.error(\"Could not deserialize the XCOM value from JSON. \"\n \"If you are using pickles instead of JSON \"\n \"for XCOM, then you need to enable pickle \"\n \"support for XCOM in your airflow config.\")\n raise"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nretrieving an XCom value optionally meeting certain criteria.", "response": "def get_many(cls,\n execution_date,\n key=None,\n task_ids=None,\n dag_ids=None,\n include_prior_dates=False,\n limit=100,\n session=None):\n \"\"\"\n Retrieve an XCom value, optionally meeting certain criteria\n TODO: \"pickling\" has been deprecated and JSON is preferred.\n \"pickling\" will be removed in Airflow 2.0.\n \"\"\"\n filters = []\n if key:\n filters.append(cls.key == key)\n if task_ids:\n filters.append(cls.task_id.in_(as_tuple(task_ids)))\n if dag_ids:\n filters.append(cls.dag_id.in_(as_tuple(dag_ids)))\n if include_prior_dates:\n filters.append(cls.execution_date <= execution_date)\n else:\n filters.append(cls.execution_date == execution_date)\n\n query = (\n session.query(cls).filter(and_(*filters))\n .order_by(cls.execution_date.desc(), cls.timestamp.desc())\n .limit(limit))\n results = query.all()\n return results"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _convert_date_to_dict(field_date):\n return {DAY: field_date.day, MONTH: field_date.month, YEAR: field_date.year}", "response": "Convert native python datetime. date object to a dict"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef _convert_time_to_dict(time):\n return {HOURS: time.hour, MINUTES: time.minute, SECONDS: time.second}", "response": "Convert native python datetime. time object to a dict"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nreturn a Redis connection.", "response": "def get_conn(self):\n \"\"\"\n Returns a Redis connection.\n \"\"\"\n conn = self.get_connection(self.redis_conn_id)\n self.host = conn.host\n self.port = conn.port\n self.password = None if str(conn.password).lower() in ['none', 'false', ''] else conn.password\n self.db = conn.extra_dejson.get('db', None)\n\n if not self.redis:\n self.log.debug(\n 'Initializing redis object for conn_id \"%s\" on %s:%s:%s',\n self.redis_conn_id, self.host, self.port, self.db\n )\n self.redis = Redis(\n host=self.host,\n port=self.port,\n password=self.password,\n db=self.db)\n\n return self.redis"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nreturn a connection object for the current database entry and its class.", "response": "def get_conn(self):\n \"\"\"\n Returns a oracle connection object\n Optional parameters for using a custom DSN connection\n (instead of using a server alias from tnsnames.ora)\n The dsn (data source name) is the TNS entry\n (from the Oracle names server or tnsnames.ora file)\n or is a string like the one returned from makedsn().\n\n :param dsn: the host address for the Oracle server\n :param service_name: the db_unique_name of the database\n that you are connecting to (CONNECT_DATA part of TNS)\n\n You can set these parameters in the extra fields of your connection\n as in ``{ \"dsn\":\"some.host.address\" , \"service_name\":\"some.service.name\" }``\n see more param detail in\n `cx_Oracle.connect `_\n \"\"\"\n conn = self.get_connection(self.oracle_conn_id)\n conn_config = {\n 'user': conn.login,\n 'password': conn.password\n }\n dsn = conn.extra_dejson.get('dsn', None)\n sid = conn.extra_dejson.get('sid', None)\n mod = conn.extra_dejson.get('module', None)\n\n service_name = conn.extra_dejson.get('service_name', None)\n port = conn.port if conn.port else 1521\n if dsn and sid and not service_name:\n conn_config['dsn'] = cx_Oracle.makedsn(dsn, port, sid)\n elif dsn and service_name and not sid:\n conn_config['dsn'] = cx_Oracle.makedsn(dsn, port, service_name=service_name)\n else:\n conn_config['dsn'] = conn.host\n\n if 'encoding' in conn.extra_dejson:\n conn_config['encoding'] = conn.extra_dejson.get('encoding')\n # if `encoding` is specific but `nencoding` is not\n # `nencoding` should use same values as `encoding` to set encoding, inspired by\n # https://github.com/oracle/python-cx_Oracle/issues/157#issuecomment-371877993\n if 'nencoding' not in conn.extra_dejson:\n conn_config['nencoding'] = conn.extra_dejson.get('encoding')\n if 'nencoding' in conn.extra_dejson:\n conn_config['nencoding'] = conn.extra_dejson.get('nencoding')\n if 'threaded' in conn.extra_dejson:\n conn_config['threaded'] = conn.extra_dejson.get('threaded')\n if 'events' in conn.extra_dejson:\n conn_config['events'] = conn.extra_dejson.get('events')\n\n mode = conn.extra_dejson.get('mode', '').lower()\n if mode == 'sysdba':\n conn_config['mode'] = cx_Oracle.SYSDBA\n elif mode == 'sysasm':\n conn_config['mode'] = cx_Oracle.SYSASM\n elif mode == 'sysoper':\n conn_config['mode'] = cx_Oracle.SYSOPER\n elif mode == 'sysbkp':\n conn_config['mode'] = cx_Oracle.SYSBKP\n elif mode == 'sysdgd':\n conn_config['mode'] = cx_Oracle.SYSDGD\n elif mode == 'syskmt':\n conn_config['mode'] = cx_Oracle.SYSKMT\n elif mode == 'sysrac':\n conn_config['mode'] = cx_Oracle.SYSRAC\n\n purity = conn.extra_dejson.get('purity', '').lower()\n if purity == 'new':\n conn_config['purity'] = cx_Oracle.ATTR_PURITY_NEW\n elif purity == 'self':\n conn_config['purity'] = cx_Oracle.ATTR_PURITY_SELF\n elif purity == 'default':\n conn_config['purity'] = cx_Oracle.ATTR_PURITY_DEFAULT\n\n conn = cx_Oracle.connect(**conn_config)\n if mod is not None:\n conn.module = mod\n\n return conn"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\ninsert a set of rows into a table in Oracle.", "response": "def insert_rows(self, table, rows, target_fields=None, commit_every=1000):\n \"\"\"\n A generic way to insert a set of tuples into a table,\n the whole set of inserts is treated as one transaction\n Changes from standard DbApiHook implementation:\n\n - Oracle SQL queries in cx_Oracle can not be terminated with a semicolon (`;`)\n - Replace NaN values with NULL using `numpy.nan_to_num` (not using\n `is_nan()` because of input types error for strings)\n - Coerce datetime cells to Oracle DATETIME format during insert\n\n :param table: target Oracle table, use dot notation to target a\n specific database\n :type table: str\n :param rows: the rows to insert into the table\n :type rows: iterable of tuples\n :param target_fields: the names of the columns to fill in the table\n :type target_fields: iterable of str\n :param commit_every: the maximum number of rows to insert in one transaction\n Default 1000, Set greater than 0.\n Set 1 to insert each row in each single transaction\n :type commit_every: int\n \"\"\"\n if target_fields:\n target_fields = ', '.join(target_fields)\n target_fields = '({})'.format(target_fields)\n else:\n target_fields = ''\n conn = self.get_conn()\n cur = conn.cursor()\n if self.supports_autocommit:\n cur.execute('SET autocommit = 0')\n conn.commit()\n i = 0\n for row in rows:\n i += 1\n lst = []\n for cell in row:\n if isinstance(cell, basestring):\n lst.append(\"'\" + str(cell).replace(\"'\", \"''\") + \"'\")\n elif cell is None:\n lst.append('NULL')\n elif type(cell) == float and \\\n numpy.isnan(cell): # coerce numpy NaN to NULL\n lst.append('NULL')\n elif isinstance(cell, numpy.datetime64):\n lst.append(\"'\" + str(cell) + \"'\")\n elif isinstance(cell, datetime):\n lst.append(\"to_date('\" +\n cell.strftime('%Y-%m-%d %H:%M:%S') +\n \"','YYYY-MM-DD HH24:MI:SS')\")\n else:\n lst.append(str(cell))\n values = tuple(lst)\n sql = 'INSERT /*+ APPEND */ ' \\\n 'INTO {0} {1} VALUES ({2})'.format(table,\n target_fields,\n ','.join(values))\n cur.execute(sql)\n if i % commit_every == 0:\n conn.commit()\n self.log.info('Loaded %s into %s rows so far', i, table)\n conn.commit()\n cur.close()\n conn.close()\n self.log.info('Done loading. Loaded a total of %s rows', i)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nreturn a connection object for the current object.", "response": "def get_conn(self):\n \"\"\"Returns a connection object\n \"\"\"\n db = self.get_connection(getattr(self, self.conn_name_attr))\n return self.connector.connect(\n host=db.host,\n port=db.port,\n username=db.login,\n schema=db.schema)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nexecuting the sql and returns a pandas dataframe", "response": "def get_pandas_df(self, sql, parameters=None):\n \"\"\"\n Executes the sql and returns a pandas dataframe\n\n :param sql: the sql statement to be executed (str) or a list of\n sql statements to execute\n :type sql: str or list\n :param parameters: The parameters to render the SQL query with.\n :type parameters: mapping or iterable\n \"\"\"\n import pandas.io.sql as psql\n\n with closing(self.get_conn()) as conn:\n return psql.read_sql(sql, con=conn, params=parameters)"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nexecute the sql and returns a set of records.", "response": "def get_records(self, sql, parameters=None):\n \"\"\"\n Executes the sql and returns a set of records.\n\n :param sql: the sql statement to be executed (str) or a list of\n sql statements to execute\n :type sql: str or list\n :param parameters: The parameters to render the SQL query with.\n :type parameters: mapping or iterable\n \"\"\"\n with closing(self.get_conn()) as conn:\n with closing(conn.cursor()) as cur:\n if parameters is not None:\n cur.execute(sql, parameters)\n else:\n cur.execute(sql)\n return cur.fetchall()"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef get_first(self, sql, parameters=None):\n with closing(self.get_conn()) as conn:\n with closing(conn.cursor()) as cur:\n if parameters is not None:\n cur.execute(sql, parameters)\n else:\n cur.execute(sql)\n return cur.fetchone()", "response": "Executes the sql and returns the first result."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nexecute a SQL query or a list of SQL statements.", "response": "def run(self, sql, autocommit=False, parameters=None):\n \"\"\"\n Runs a command or a list of commands. Pass a list of sql\n statements to the sql parameter to get them to execute\n sequentially\n\n :param sql: the sql statement to be executed (str) or a list of\n sql statements to execute\n :type sql: str or list\n :param autocommit: What to set the connection's autocommit setting to\n before executing the query.\n :type autocommit: bool\n :param parameters: The parameters to render the SQL query with.\n :type parameters: mapping or iterable\n \"\"\"\n if isinstance(sql, basestring):\n sql = [sql]\n\n with closing(self.get_conn()) as conn:\n if self.supports_autocommit:\n self.set_autocommit(conn, autocommit)\n\n with closing(conn.cursor()) as cur:\n for s in sql:\n if parameters is not None:\n self.log.info(\"{} with parameters {}\".format(s, parameters))\n cur.execute(s, parameters)\n else:\n self.log.info(s)\n cur.execute(s)\n\n # If autocommit was set to False for db that supports autocommit,\n # or if db does not supports autocommit, we do a manual commit.\n if not self.get_autocommit(conn):\n conn.commit()"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nsets the autocommit flag on the connection conn.", "response": "def set_autocommit(self, conn, autocommit):\n \"\"\"\n Sets the autocommit flag on the connection\n \"\"\"\n if not self.supports_autocommit and autocommit:\n self.log.warn(\n (\"%s connection doesn't support \"\n \"autocommit but autocommit activated.\"),\n getattr(self, self.conn_name_attr))\n conn.autocommit = autocommit"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef insert_rows(self, table, rows, target_fields=None, commit_every=1000,\n replace=False):\n \"\"\"\n A generic way to insert a set of tuples into a table,\n a new transaction is created every commit_every rows\n\n :param table: Name of the target table\n :type table: str\n :param rows: The rows to insert into the table\n :type rows: iterable of tuples\n :param target_fields: The names of the columns to fill in the table\n :type target_fields: iterable of strings\n :param commit_every: The maximum number of rows to insert in one\n transaction. Set to 0 to insert all rows in one transaction.\n :type commit_every: int\n :param replace: Whether to replace instead of insert\n :type replace: bool\n \"\"\"\n if target_fields:\n target_fields = \", \".join(target_fields)\n target_fields = \"({})\".format(target_fields)\n else:\n target_fields = ''\n i = 0\n with closing(self.get_conn()) as conn:\n if self.supports_autocommit:\n self.set_autocommit(conn, False)\n\n conn.commit()\n\n with closing(conn.cursor()) as cur:\n for i, row in enumerate(rows, 1):\n lst = []\n for cell in row:\n lst.append(self._serialize_cell(cell, conn))\n values = tuple(lst)\n placeholders = [\"%s\", ] * len(values)\n if not replace:\n sql = \"INSERT INTO \"\n else:\n sql = \"REPLACE INTO \"\n sql += \"{0} {1} VALUES ({2})\".format(\n table,\n target_fields,\n \",\".join(placeholders))\n cur.execute(sql, values)\n if commit_every and i % commit_every == 0:\n conn.commit()\n self.log.info(\n \"Loaded %s into %s rows so far\", i, table\n )\n\n conn.commit()\n self.log.info(\"Done loading. Loaded a total of %s rows\", i)", "response": "A generic way to insert a set of tuples into a table."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nreturns the SQL literal of the cell as a string.", "response": "def _serialize_cell(cell, conn=None):\n \"\"\"\n Returns the SQL literal of the cell as a string.\n\n :param cell: The cell to insert into the table\n :type cell: object\n :param conn: The database connection\n :type conn: connection object\n :return: The serialized cell\n :rtype: str\n \"\"\"\n\n if cell is None:\n return None\n if isinstance(cell, datetime):\n return cell.isoformat()\n return str(cell)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef health(self, session=None):\n\n BJ = jobs.BaseJob\n payload = {}\n scheduler_health_check_threshold = timedelta(seconds=conf.getint('scheduler',\n 'scheduler_health_check_threshold'\n ))\n\n latest_scheduler_heartbeat = None\n payload['metadatabase'] = {'status': 'healthy'}\n try:\n latest_scheduler_heartbeat = session.query(func.max(BJ.latest_heartbeat)).\\\n filter(BJ.state == 'running', BJ.job_type == 'SchedulerJob').\\\n scalar()\n except Exception:\n payload['metadatabase']['status'] = 'unhealthy'\n\n if not latest_scheduler_heartbeat:\n scheduler_status = 'unhealthy'\n else:\n if timezone.utcnow() - latest_scheduler_heartbeat <= scheduler_health_check_threshold:\n scheduler_status = 'healthy'\n else:\n scheduler_status = 'unhealthy'\n\n payload['scheduler'] = {'status': scheduler_status,\n 'latest_scheduler_heartbeat': str(latest_scheduler_heartbeat)}\n\n return wwwutils.json_response(payload)", "response": "This endpoint returns the health status of the Airflow instance."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nreturning a QuerySet of all the related objects in the database.", "response": "def get_query(self):\n \"\"\"\n Default filters for model\n \"\"\"\n return (\n super().get_query()\n .filter(or_(models.DagModel.is_active,\n models.DagModel.is_paused))\n .filter(~models.DagModel.is_subdag)\n )"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef get_count_query(self):\n return (\n super().get_count_query()\n .filter(models.DagModel.is_active)\n .filter(~models.DagModel.is_subdag)\n )", "response": "Returns a QuerySet that returns the count of the active Dags."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nreturns a connection to the cloudant service and opens it automatically if used as context manager.", "response": "def get_conn(self):\n \"\"\"\n Opens a connection to the cloudant service and closes it automatically if used as context manager.\n\n .. note::\n In the connection form:\n - 'host' equals the 'Account' (optional)\n - 'login' equals the 'Username (or API Key)' (required)\n - 'password' equals the 'Password' (required)\n\n :return: an authorized cloudant session context manager object.\n :rtype: cloudant\n \"\"\"\n conn = self.get_connection(self.cloudant_conn_id)\n\n self._validate_connection(conn)\n\n cloudant_session = cloudant(user=conn.login, passwd=conn.password, account=conn.host)\n\n return cloudant_session"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef execute(self, context):\n self.hook = SlackWebhookHook(\n self.http_conn_id,\n self.webhook_token,\n self.message,\n self.attachments,\n self.channel,\n self.username,\n self.icon_emoji,\n self.link_names,\n self.proxy\n )\n self.hook.execute()", "response": "Call the SlackWebhookHook to post the provided Slack message"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _get_credentials(self):\n key_path = self._get_field('key_path', False)\n keyfile_dict = self._get_field('keyfile_dict', False)\n scope = self._get_field('scope', None)\n if scope:\n scopes = [s.strip() for s in scope.split(',')]\n else:\n scopes = _DEFAULT_SCOPES\n\n if not key_path and not keyfile_dict:\n self.log.info('Getting connection using `google.auth.default()` '\n 'since no key file is defined for hook.')\n credentials, _ = google.auth.default(scopes=scopes)\n elif key_path:\n # Get credentials from a JSON file.\n if key_path.endswith('.json'):\n self.log.debug('Getting connection using JSON key file %s' % key_path)\n credentials = (\n google.oauth2.service_account.Credentials.from_service_account_file(\n key_path, scopes=scopes)\n )\n elif key_path.endswith('.p12'):\n raise AirflowException('Legacy P12 key file are not supported, '\n 'use a JSON key file.')\n else:\n raise AirflowException('Unrecognised extension for key file.')\n else:\n # Get credentials from JSON data provided in the UI.\n try:\n keyfile_dict = json.loads(keyfile_dict)\n\n # Depending on how the JSON was formatted, it may contain\n # escaped newlines. Convert those to actual newlines.\n keyfile_dict['private_key'] = keyfile_dict['private_key'].replace(\n '\\\\n', '\\n')\n\n credentials = (\n google.oauth2.service_account.Credentials.from_service_account_info(\n keyfile_dict, scopes=scopes)\n )\n except json.decoder.JSONDecodeError:\n raise AirflowException('Invalid key JSON.')\n\n return credentials.with_subject(self.delegate_to) \\\n if self.delegate_to else credentials", "response": "Returns the Credentials object for Google API."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nreturns an authorized HTTP object to be used to build a Google Cloud service hook connection.", "response": "def _authorize(self):\n \"\"\"\n Returns an authorized HTTP object to be used to build a Google cloud\n service hook connection.\n \"\"\"\n credentials = self._get_credentials()\n http = httplib2.Http()\n authed_http = google_auth_httplib2.AuthorizedHttp(\n credentials, http=http)\n return authed_http"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _get_field(self, f, default=None):\n long_f = 'extra__google_cloud_platform__{}'.format(f)\n if hasattr(self, 'extras') and long_f in self.extras:\n return self.extras[long_f]\n else:\n return default", "response": "Gets a field from extras and returns it."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nfunction decorator that intercepts HTTP Errors and raises AirflowException with more informative message.", "response": "def catch_http_exception(func):\n \"\"\"\n Function decorator that intercepts HTTP Errors and raises AirflowException\n with more informative message.\n \"\"\"\n\n @functools.wraps(func)\n def wrapper_decorator(self, *args, **kwargs):\n try:\n return func(self, *args, **kwargs)\n except GoogleAPICallError as e:\n if isinstance(e, AlreadyExists):\n raise e\n else:\n self.log.error('The request failed:\\n%s', str(e))\n raise AirflowException(e)\n except RetryError as e:\n self.log.error('The request failed due to a retryable error and retry attempts failed.')\n raise AirflowException(e)\n except ValueError as e:\n self.log.error('The request failed, the parameters are invalid.')\n raise AirflowException(e)\n except HttpError as e:\n self.log.error('The request failed:\\n%s', str(e))\n raise AirflowException(e)\n\n return wrapper_decorator"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef unfinished(cls):\n return [\n cls.NONE,\n cls.SCHEDULED,\n cls.QUEUED,\n cls.RUNNING,\n cls.SHUTDOWN,\n cls.UP_FOR_RETRY,\n cls.UP_FOR_RESCHEDULE\n ]", "response": "A list of states indicating that a task has not yet been completed."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef delete_dag(dag_id, keep_records_in_log=True, session=None):\n DM = models.DagModel\n dag = session.query(DM).filter(DM.dag_id == dag_id).first()\n if dag is None:\n raise DagNotFound(\"Dag id {} not found\".format(dag_id))\n\n if dag.fileloc and os.path.exists(dag.fileloc):\n raise DagFileExists(\"Dag id {} is still in DagBag. \"\n \"Remove the DAG file first: {}\".format(dag_id, dag.fileloc))\n\n count = 0\n\n # noinspection PyUnresolvedReferences,PyProtectedMember\n for m in models.base.Base._decl_class_registry.values():\n if hasattr(m, \"dag_id\"):\n if keep_records_in_log and m.__name__ == 'Log':\n continue\n cond = or_(m.dag_id == dag_id, m.dag_id.like(dag_id + \".%\"))\n count += session.query(m).filter(cond).delete(synchronize_session='fetch')\n\n if dag.is_subdag:\n p, c = dag_id.rsplit(\".\", 1)\n for m in models.DagRun, TaskFail, models.TaskInstance:\n count += session.query(m).filter(m.dag_id == p, m.task_id == c).delete()\n\n return count", "response": "Delete a DAG from the database."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nconstructs the spark - sql command to execute.", "response": "def _prepare_command(self, cmd):\n \"\"\"\n Construct the spark-sql command to execute. Verbose output is enabled\n as default.\n\n :param cmd: command to append to the spark-sql command\n :type cmd: str\n :return: full command to be executed\n \"\"\"\n connection_cmd = [\"spark-sql\"]\n if self._conf:\n for conf_el in self._conf.split(\",\"):\n connection_cmd += [\"--conf\", conf_el]\n if self._total_executor_cores:\n connection_cmd += [\"--total-executor-cores\", str(self._total_executor_cores)]\n if self._executor_cores:\n connection_cmd += [\"--executor-cores\", str(self._executor_cores)]\n if self._executor_memory:\n connection_cmd += [\"--executor-memory\", self._executor_memory]\n if self._keytab:\n connection_cmd += [\"--keytab\", self._keytab]\n if self._principal:\n connection_cmd += [\"--principal\", self._principal]\n if self._num_executors:\n connection_cmd += [\"--num-executors\", str(self._num_executors)]\n if self._sql:\n sql = self._sql.strip()\n if sql.endswith(\".sql\") or sql.endswith(\".hql\"):\n connection_cmd += [\"-f\", sql]\n else:\n connection_cmd += [\"-e\", sql]\n if self._master:\n connection_cmd += [\"--master\", self._master]\n if self._name:\n connection_cmd += [\"--name\", self._name]\n if self._verbose:\n connection_cmd += [\"--verbose\"]\n if self._yarn_queue:\n connection_cmd += [\"--queue\", self._yarn_queue]\n\n connection_cmd += cmd\n self.log.debug(\"Spark-Sql cmd: %s\", connection_cmd)\n\n return connection_cmd"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef run_query(self, cmd=\"\", **kwargs):\n spark_sql_cmd = self._prepare_command(cmd)\n self._sp = subprocess.Popen(spark_sql_cmd,\n stdout=subprocess.PIPE,\n stderr=subprocess.STDOUT,\n **kwargs)\n\n for line in iter(self._sp.stdout.readline, ''):\n self.log.info(line)\n\n returncode = self._sp.wait()\n\n if returncode:\n raise AirflowException(\n \"Cannot execute {} on {}. Process exit code: {}.\".format(\n cmd, self._conn.host, returncode\n )\n )", "response": "Execute Spark - SQL query and return the result."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef densenet121(pretrained=False, **kwargs):\n model = DenseNet(num_init_features=64, growth_rate=32, block_config=(6, 12, 24, 16),\n **kwargs)\n if pretrained:\n _load_state_dict(model, model_urls['densenet121'])\n return model", "response": "r Densenet -121 model from an image net\n "} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef to_tensor(pic):\n if not(_is_pil_image(pic) or _is_numpy_image(pic)):\n raise TypeError('pic should be PIL Image or ndarray. Got {}'.format(type(pic)))\n\n if isinstance(pic, np.ndarray):\n # handle numpy array\n if pic.ndim == 2:\n pic = pic[:, :, None]\n\n img = torch.from_numpy(pic.transpose((2, 0, 1)))\n # backward compatibility\n if isinstance(img, torch.ByteTensor):\n return img.float().div(255)\n else:\n return img\n\n if accimage is not None and isinstance(pic, accimage.Image):\n nppic = np.zeros([pic.channels, pic.height, pic.width], dtype=np.float32)\n pic.copyto(nppic)\n return torch.from_numpy(nppic)\n\n # handle PIL Image\n if pic.mode == 'I':\n img = torch.from_numpy(np.array(pic, np.int32, copy=False))\n elif pic.mode == 'I;16':\n img = torch.from_numpy(np.array(pic, np.int16, copy=False))\n elif pic.mode == 'F':\n img = torch.from_numpy(np.array(pic, np.float32, copy=False))\n elif pic.mode == '1':\n img = 255 * torch.from_numpy(np.array(pic, np.uint8, copy=False))\n else:\n img = torch.ByteTensor(torch.ByteStorage.from_buffer(pic.tobytes()))\n # PIL image mode: L, LA, P, I, F, RGB, YCbCr, RGBA, CMYK\n if pic.mode == 'YCbCr':\n nchannel = 3\n elif pic.mode == 'I;16':\n nchannel = 1\n else:\n nchannel = len(pic.mode)\n img = img.view(pic.size[1], pic.size[0], nchannel)\n # put it from HWC to CHW format\n # yikes, this transpose takes 80% of the loading time/CPU\n img = img.transpose(0, 1).transpose(0, 2).contiguous()\n if isinstance(img, torch.ByteTensor):\n return img.float().div(255)\n else:\n return img", "response": "Convert a PIL Image or numpy. ndarray to a tensor."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nconverting a tensor or numpy array to PIL Image.", "response": "def to_pil_image(pic, mode=None):\n \"\"\"Convert a tensor or an ndarray to PIL Image.\n\n See :class:`~torchvision.transforms.ToPILImage` for more details.\n\n Args:\n pic (Tensor or numpy.ndarray): Image to be converted to PIL Image.\n mode (`PIL.Image mode`_): color space and pixel depth of input data (optional).\n\n .. _PIL.Image mode: https://pillow.readthedocs.io/en/latest/handbook/concepts.html#concept-modes\n\n Returns:\n PIL Image: Image converted to PIL Image.\n \"\"\"\n if not(isinstance(pic, torch.Tensor) or isinstance(pic, np.ndarray)):\n raise TypeError('pic should be Tensor or ndarray. Got {}.'.format(type(pic)))\n\n elif isinstance(pic, torch.Tensor):\n if pic.ndimension() not in {2, 3}:\n raise ValueError('pic should be 2/3 dimensional. Got {} dimensions.'.format(pic.ndimension()))\n\n elif pic.ndimension() == 2:\n # if 2D image, add channel dimension (CHW)\n pic = pic.unsqueeze(0)\n\n elif isinstance(pic, np.ndarray):\n if pic.ndim not in {2, 3}:\n raise ValueError('pic should be 2/3 dimensional. Got {} dimensions.'.format(pic.ndim))\n\n elif pic.ndim == 2:\n # if 2D image, add channel dimension (HWC)\n pic = np.expand_dims(pic, 2)\n\n npimg = pic\n if isinstance(pic, torch.FloatTensor):\n pic = pic.mul(255).byte()\n if isinstance(pic, torch.Tensor):\n npimg = np.transpose(pic.numpy(), (1, 2, 0))\n\n if not isinstance(npimg, np.ndarray):\n raise TypeError('Input pic must be a torch.Tensor or NumPy ndarray, ' +\n 'not {}'.format(type(npimg)))\n\n if npimg.shape[2] == 1:\n expected_mode = None\n npimg = npimg[:, :, 0]\n if npimg.dtype == np.uint8:\n expected_mode = 'L'\n elif npimg.dtype == np.int16:\n expected_mode = 'I;16'\n elif npimg.dtype == np.int32:\n expected_mode = 'I'\n elif npimg.dtype == np.float32:\n expected_mode = 'F'\n if mode is not None and mode != expected_mode:\n raise ValueError(\"Incorrect mode ({}) supplied for input type {}. Should be {}\"\n .format(mode, np.dtype, expected_mode))\n mode = expected_mode\n\n elif npimg.shape[2] == 2:\n permitted_2_channel_modes = ['LA']\n if mode is not None and mode not in permitted_2_channel_modes:\n raise ValueError(\"Only modes {} are supported for 2D inputs\".format(permitted_2_channel_modes))\n\n if mode is None and npimg.dtype == np.uint8:\n mode = 'LA'\n\n elif npimg.shape[2] == 4:\n permitted_4_channel_modes = ['RGBA', 'CMYK', 'RGBX']\n if mode is not None and mode not in permitted_4_channel_modes:\n raise ValueError(\"Only modes {} are supported for 4D inputs\".format(permitted_4_channel_modes))\n\n if mode is None and npimg.dtype == np.uint8:\n mode = 'RGBA'\n else:\n permitted_3_channel_modes = ['RGB', 'YCbCr', 'HSV']\n if mode is not None and mode not in permitted_3_channel_modes:\n raise ValueError(\"Only modes {} are supported for 3D inputs\".format(permitted_3_channel_modes))\n if mode is None and npimg.dtype == np.uint8:\n mode = 'RGB'\n\n if mode is None:\n raise TypeError('Input type {} is not supported'.format(npimg.dtype))\n\n return Image.fromarray(npimg, mode=mode)"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nnormalizes a tensor image with mean and standard deviation.", "response": "def normalize(tensor, mean, std, inplace=False):\n \"\"\"Normalize a tensor image with mean and standard deviation.\n\n .. note::\n This transform acts out of place by default, i.e., it does not mutates the input tensor.\n\n See :class:`~torchvision.transforms.Normalize` for more details.\n\n Args:\n tensor (Tensor): Tensor image of size (C, H, W) to be normalized.\n mean (sequence): Sequence of means for each channel.\n std (sequence): Sequence of standard deviations for each channel.\n\n Returns:\n Tensor: Normalized Tensor image.\n \"\"\"\n if not _is_tensor_image(tensor):\n raise TypeError('tensor is not a torch image.')\n\n if not inplace:\n tensor = tensor.clone()\n\n mean = torch.as_tensor(mean, dtype=torch.float32, device=tensor.device)\n std = torch.as_tensor(std, dtype=torch.float32, device=tensor.device)\n tensor.sub_(mean[:, None, None]).div_(std[:, None, None])\n return tensor"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef pad(img, padding, fill=0, padding_mode='constant'):\n if not _is_pil_image(img):\n raise TypeError('img should be PIL Image. Got {}'.format(type(img)))\n\n if not isinstance(padding, (numbers.Number, tuple)):\n raise TypeError('Got inappropriate padding arg')\n if not isinstance(fill, (numbers.Number, str, tuple)):\n raise TypeError('Got inappropriate fill arg')\n if not isinstance(padding_mode, str):\n raise TypeError('Got inappropriate padding_mode arg')\n\n if isinstance(padding, Sequence) and len(padding) not in [2, 4]:\n raise ValueError(\"Padding must be an int or a 2, or 4 element tuple, not a \" +\n \"{} element tuple\".format(len(padding)))\n\n assert padding_mode in ['constant', 'edge', 'reflect', 'symmetric'], \\\n 'Padding mode should be either constant, edge, reflect or symmetric'\n\n if padding_mode == 'constant':\n if img.mode == 'P':\n palette = img.getpalette()\n image = ImageOps.expand(img, border=padding, fill=fill)\n image.putpalette(palette)\n return image\n\n return ImageOps.expand(img, border=padding, fill=fill)\n else:\n if isinstance(padding, int):\n pad_left = pad_right = pad_top = pad_bottom = padding\n if isinstance(padding, Sequence) and len(padding) == 2:\n pad_left = pad_right = padding[0]\n pad_top = pad_bottom = padding[1]\n if isinstance(padding, Sequence) and len(padding) == 4:\n pad_left = padding[0]\n pad_top = padding[1]\n pad_right = padding[2]\n pad_bottom = padding[3]\n\n if img.mode == 'P':\n palette = img.getpalette()\n img = np.asarray(img)\n img = np.pad(img, ((pad_top, pad_bottom), (pad_left, pad_right)), padding_mode)\n img = Image.fromarray(img)\n img.putpalette(palette)\n return img\n\n img = np.asarray(img)\n # RGB image\n if len(img.shape) == 3:\n img = np.pad(img, ((pad_top, pad_bottom), (pad_left, pad_right), (0, 0)), padding_mode)\n # Grayscale image\n if len(img.shape) == 2:\n img = np.pad(img, ((pad_top, pad_bottom), (pad_left, pad_right)), padding_mode)\n\n return Image.fromarray(img)", "response": "r Pads the given PIL Image on all sides with specified padding mode and fill value."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef crop(img, i, j, h, w):\n if not _is_pil_image(img):\n raise TypeError('img should be PIL Image. Got {}'.format(type(img)))\n\n return img.crop((j, i, j + w, i + h))", "response": "Crop the given PIL Image."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\ncrop the given PIL Image and resize it to desired size.", "response": "def resized_crop(img, i, j, h, w, size, interpolation=Image.BILINEAR):\n \"\"\"Crop the given PIL Image and resize it to desired size.\n\n Notably used in :class:`~torchvision.transforms.RandomResizedCrop`.\n\n Args:\n img (PIL Image): Image to be cropped.\n i (int): i in (i,j) i.e coordinates of the upper left corner\n j (int): j in (i,j) i.e coordinates of the upper left corner\n h (int): Height of the cropped image.\n w (int): Width of the cropped image.\n size (sequence or int): Desired output size. Same semantics as ``resize``.\n interpolation (int, optional): Desired interpolation. Default is\n ``PIL.Image.BILINEAR``.\n Returns:\n PIL Image: Cropped image.\n \"\"\"\n assert _is_pil_image(img), 'img should be PIL Image'\n img = crop(img, i, j, h, w)\n img = resize(img, size, interpolation)\n return img"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _get_perspective_coeffs(startpoints, endpoints):\n matrix = []\n\n for p1, p2 in zip(endpoints, startpoints):\n matrix.append([p1[0], p1[1], 1, 0, 0, 0, -p2[0] * p1[0], -p2[0] * p1[1]])\n matrix.append([0, 0, 0, p1[0], p1[1], 1, -p2[1] * p1[0], -p2[1] * p1[1]])\n\n A = torch.tensor(matrix, dtype=torch.float)\n B = torch.tensor(startpoints, dtype=torch.float).view(8)\n res = torch.gels(B, A)[0]\n return res.squeeze_(1).tolist()", "response": "Helper function to get the coefficients for the perspective transform."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nperform a perspective transform of the given PIL Image.", "response": "def perspective(img, startpoints, endpoints, interpolation=Image.BICUBIC):\n \"\"\"Perform perspective transform of the given PIL Image.\n\n Args:\n img (PIL Image): Image to be transformed.\n coeffs (tuple) : 8-tuple (a, b, c, d, e, f, g, h) which contains the coefficients.\n for a perspective transform.\n interpolation: Default- Image.BICUBIC\n Returns:\n PIL Image: Perspectively transformed Image.\n \"\"\"\n if not _is_pil_image(img):\n raise TypeError('img should be PIL Image. Got {}'.format(type(img)))\n\n coeffs = _get_perspective_coeffs(startpoints, endpoints)\n return img.transform(img.size, Image.PERSPECTIVE, coeffs, interpolation)"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef five_crop(img, size):\n if isinstance(size, numbers.Number):\n size = (int(size), int(size))\n else:\n assert len(size) == 2, \"Please provide only two dimensions (h, w) for size.\"\n\n w, h = img.size\n crop_h, crop_w = size\n if crop_w > w or crop_h > h:\n raise ValueError(\"Requested crop size {} is bigger than input size {}\".format(size,\n (h, w)))\n tl = img.crop((0, 0, crop_w, crop_h))\n tr = img.crop((w - crop_w, 0, w, crop_h))\n bl = img.crop((0, h - crop_h, crop_w, h))\n br = img.crop((w - crop_w, h - crop_h, w, h))\n center = center_crop(img, (crop_h, crop_w))\n return (tl, tr, bl, br, center)", "response": "Crop the given PIL Image into four corners and the central crop."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef ten_crop(img, size, vertical_flip=False):\n if isinstance(size, numbers.Number):\n size = (int(size), int(size))\n else:\n assert len(size) == 2, \"Please provide only two dimensions (h, w) for size.\"\n\n first_five = five_crop(img, size)\n\n if vertical_flip:\n img = vflip(img)\n else:\n img = hflip(img)\n\n second_five = five_crop(img, size)\n return first_five + second_five", "response": "r Returns a tuple of images that are ten corners and the central crop plus the a\n flipped version of these."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nadjust the brightness of an Image.", "response": "def adjust_brightness(img, brightness_factor):\n \"\"\"Adjust brightness of an Image.\n\n Args:\n img (PIL Image): PIL Image to be adjusted.\n brightness_factor (float): How much to adjust the brightness. Can be\n any non negative number. 0 gives a black image, 1 gives the\n original image while 2 increases the brightness by a factor of 2.\n\n Returns:\n PIL Image: Brightness adjusted image.\n \"\"\"\n if not _is_pil_image(img):\n raise TypeError('img should be PIL Image. Got {}'.format(type(img)))\n\n enhancer = ImageEnhance.Brightness(img)\n img = enhancer.enhance(brightness_factor)\n return img"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef adjust_contrast(img, contrast_factor):\n if not _is_pil_image(img):\n raise TypeError('img should be PIL Image. Got {}'.format(type(img)))\n\n enhancer = ImageEnhance.Contrast(img)\n img = enhancer.enhance(contrast_factor)\n return img", "response": "Adjusts the contrast of an Image."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nadjust the saturation of an image.", "response": "def adjust_saturation(img, saturation_factor):\n \"\"\"Adjust color saturation of an image.\n\n Args:\n img (PIL Image): PIL Image to be adjusted.\n saturation_factor (float): How much to adjust the saturation. 0 will\n give a black and white image, 1 will give the original image while\n 2 will enhance the saturation by a factor of 2.\n\n Returns:\n PIL Image: Saturation adjusted image.\n \"\"\"\n if not _is_pil_image(img):\n raise TypeError('img should be PIL Image. Got {}'.format(type(img)))\n\n enhancer = ImageEnhance.Color(img)\n img = enhancer.enhance(saturation_factor)\n return img"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef adjust_hue(img, hue_factor):\n if not(-0.5 <= hue_factor <= 0.5):\n raise ValueError('hue_factor is not in [-0.5, 0.5].'.format(hue_factor))\n\n if not _is_pil_image(img):\n raise TypeError('img should be PIL Image. Got {}'.format(type(img)))\n\n input_mode = img.mode\n if input_mode in {'L', '1', 'I', 'F'}:\n return img\n\n h, s, v = img.convert('HSV').split()\n\n np_h = np.array(h, dtype=np.uint8)\n # uint8 addition take cares of rotation across boundaries\n with np.errstate(over='ignore'):\n np_h += np.uint8(hue_factor * 255)\n h = Image.fromarray(np_h, 'L')\n\n img = Image.merge('HSV', (h, s, v)).convert(input_mode)\n return img", "response": "Adjusts the hue of an image."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef rotate(img, angle, resample=False, expand=False, center=None):\n\n if not _is_pil_image(img):\n raise TypeError('img should be PIL Image. Got {}'.format(type(img)))\n\n return img.rotate(angle, resample, expand, center)", "response": "Rotate the image by angle."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\napplying an affine transformation on the image keeping image center invariant.", "response": "def affine(img, angle, translate, scale, shear, resample=0, fillcolor=None):\n \"\"\"Apply affine transformation on the image keeping image center invariant\n\n Args:\n img (PIL Image): PIL Image to be rotated.\n angle (float or int): rotation angle in degrees between -180 and 180, clockwise direction.\n translate (list or tuple of integers): horizontal and vertical translations (post-rotation translation)\n scale (float): overall scale\n shear (float): shear angle value in degrees between -180 to 180, clockwise direction.\n resample (``PIL.Image.NEAREST`` or ``PIL.Image.BILINEAR`` or ``PIL.Image.BICUBIC``, optional):\n An optional resampling filter.\n See `filters`_ for more information.\n If omitted, or if the image has mode \"1\" or \"P\", it is set to ``PIL.Image.NEAREST``.\n fillcolor (int): Optional fill color for the area outside the transform in the output image. (Pillow>=5.0.0)\n \"\"\"\n if not _is_pil_image(img):\n raise TypeError('img should be PIL Image. Got {}'.format(type(img)))\n\n assert isinstance(translate, (tuple, list)) and len(translate) == 2, \\\n \"Argument translate should be a list or tuple of length 2\"\n\n assert scale > 0.0, \"Argument scale should be positive\"\n\n output_size = img.size\n center = (img.size[0] * 0.5 + 0.5, img.size[1] * 0.5 + 0.5)\n matrix = _get_inverse_affine_matrix(center, angle, translate, scale, shear)\n kwargs = {\"fillcolor\": fillcolor} if PILLOW_VERSION[0] == '5' else {}\n return img.transform(output_size, Image.AFFINE, matrix, resample, **kwargs)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nconverting image to grayscale version of image.", "response": "def to_grayscale(img, num_output_channels=1):\n \"\"\"Convert image to grayscale version of image.\n\n Args:\n img (PIL Image): Image to be converted to grayscale.\n\n Returns:\n PIL Image: Grayscale version of the image.\n if num_output_channels = 1 : returned image is single channel\n\n if num_output_channels = 3 : returned image is 3 channel with r = g = b\n \"\"\"\n if not _is_pil_image(img):\n raise TypeError('img should be PIL Image. Got {}'.format(type(img)))\n\n if num_output_channels == 1:\n img = img.convert('L')\n elif num_output_channels == 3:\n img = img.convert('L')\n np_img = np.array(img, dtype=np.uint8)\n np_img = np.dstack([np_img, np_img, np_img])\n img = Image.fromarray(np_img, 'RGB')\n else:\n raise ValueError('num_output_channels should be either 1 or 3')\n\n return img"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nmake a grid of images.", "response": "def make_grid(tensor, nrow=8, padding=2,\n normalize=False, range=None, scale_each=False, pad_value=0):\n \"\"\"Make a grid of images.\n\n Args:\n tensor (Tensor or list): 4D mini-batch Tensor of shape (B x C x H x W)\n or a list of images all of the same size.\n nrow (int, optional): Number of images displayed in each row of the grid.\n The Final grid size is (B / nrow, nrow). Default is 8.\n padding (int, optional): amount of padding. Default is 2.\n normalize (bool, optional): If True, shift the image to the range (0, 1),\n by subtracting the minimum and dividing by the maximum pixel value.\n range (tuple, optional): tuple (min, max) where min and max are numbers,\n then these numbers are used to normalize the image. By default, min and max\n are computed from the tensor.\n scale_each (bool, optional): If True, scale each image in the batch of\n images separately rather than the (min, max) over all images.\n pad_value (float, optional): Value for the padded pixels.\n\n Example:\n See this notebook `here `_\n\n \"\"\"\n if not (torch.is_tensor(tensor) or\n (isinstance(tensor, list) and all(torch.is_tensor(t) for t in tensor))):\n raise TypeError('tensor or list of tensors expected, got {}'.format(type(tensor)))\n\n # if list of tensors, convert to a 4D mini-batch Tensor\n if isinstance(tensor, list):\n tensor = torch.stack(tensor, dim=0)\n\n if tensor.dim() == 2: # single image H x W\n tensor = tensor.unsqueeze(0)\n if tensor.dim() == 3: # single image\n if tensor.size(0) == 1: # if single-channel, convert to 3-channel\n tensor = torch.cat((tensor, tensor, tensor), 0)\n tensor = tensor.unsqueeze(0)\n\n if tensor.dim() == 4 and tensor.size(1) == 1: # single-channel images\n tensor = torch.cat((tensor, tensor, tensor), 1)\n\n if normalize is True:\n tensor = tensor.clone() # avoid modifying tensor in-place\n if range is not None:\n assert isinstance(range, tuple), \\\n \"range has to be a tuple (min, max) if specified. min and max are numbers\"\n\n def norm_ip(img, min, max):\n img.clamp_(min=min, max=max)\n img.add_(-min).div_(max - min + 1e-5)\n\n def norm_range(t, range):\n if range is not None:\n norm_ip(t, range[0], range[1])\n else:\n norm_ip(t, float(t.min()), float(t.max()))\n\n if scale_each is True:\n for t in tensor: # loop over mini-batch dimension\n norm_range(t, range)\n else:\n norm_range(tensor, range)\n\n if tensor.size(0) == 1:\n return tensor.squeeze()\n\n # make the mini-batch of images into a grid\n nmaps = tensor.size(0)\n xmaps = min(nrow, nmaps)\n ymaps = int(math.ceil(float(nmaps) / xmaps))\n height, width = int(tensor.size(2) + padding), int(tensor.size(3) + padding)\n grid = tensor.new_full((3, height * ymaps + padding, width * xmaps + padding), pad_value)\n k = 0\n for y in irange(ymaps):\n for x in irange(xmaps):\n if k >= nmaps:\n break\n grid.narrow(1, y * height + padding, height - padding)\\\n .narrow(2, x * width + padding, width - padding)\\\n .copy_(tensor[k])\n k = k + 1\n return grid"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nsave a given Tensor into an image file.", "response": "def save_image(tensor, filename, nrow=8, padding=2,\n normalize=False, range=None, scale_each=False, pad_value=0):\n \"\"\"Save a given Tensor into an image file.\n\n Args:\n tensor (Tensor or list): Image to be saved. If given a mini-batch tensor,\n saves the tensor as a grid of images by calling ``make_grid``.\n **kwargs: Other arguments are documented in ``make_grid``.\n \"\"\"\n from PIL import Image\n grid = make_grid(tensor, nrow=nrow, padding=padding, pad_value=pad_value,\n normalize=normalize, range=range, scale_each=scale_each)\n # Add 0.5 after unnormalizing to [0, 255] to round to nearest integer\n ndarr = grid.mul_(255).add_(0.5).clamp_(0, 255).permute(1, 2, 0).to('cpu', torch.uint8).numpy()\n im = Image.fromarray(ndarr)\n im.save(filename)"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nfinds the class folders in a dataset.", "response": "def _find_classes(self, dir):\n \"\"\"\n Finds the class folders in a dataset.\n\n Args:\n dir (string): Root directory path.\n\n Returns:\n tuple: (classes, class_to_idx) where classes are relative to (dir), and class_to_idx is a dictionary.\n\n Ensures:\n No class is a subdirectory of another.\n \"\"\"\n if sys.version_info >= (3, 5):\n # Faster and available in Python 3.5 and above\n classes = [d.name for d in os.scandir(dir) if d.is_dir()]\n else:\n classes = [d for d in os.listdir(dir) if os.path.isdir(os.path.join(dir, d))]\n classes.sort()\n class_to_idx = {classes[i]: i for i in range(len(classes))}\n return classes, class_to_idx"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef read_image_file(data_dir, image_ext, n):\n\n def PIL2array(_img):\n \"\"\"Convert PIL image type to numpy 2D array\n \"\"\"\n return np.array(_img.getdata(), dtype=np.uint8).reshape(64, 64)\n\n def find_files(_data_dir, _image_ext):\n \"\"\"Return a list with the file names of the images containing the patches\n \"\"\"\n files = []\n # find those files with the specified extension\n for file_dir in os.listdir(_data_dir):\n if file_dir.endswith(_image_ext):\n files.append(os.path.join(_data_dir, file_dir))\n return sorted(files) # sort files in ascend order to keep relations\n\n patches = []\n list_files = find_files(data_dir, image_ext)\n\n for fpath in list_files:\n img = Image.open(fpath)\n for y in range(0, 1024, 64):\n for x in range(0, 1024, 64):\n patch = img.crop((x, y, x + 64, y + 64))\n patches.append(PIL2array(patch))\n return torch.ByteTensor(np.array(patches[:n]))", "response": "Read the image file and return a Tensor containing the patches"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef read_info_file(data_dir, info_file):\n labels = []\n with open(os.path.join(data_dir, info_file), 'r') as f:\n labels = [int(line.split()[0]) for line in f]\n return torch.LongTensor(labels)", "response": "Read the info file and keep only the ID of the 3D point."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef read_matches_files(data_dir, matches_file):\n matches = []\n with open(os.path.join(data_dir, matches_file), 'r') as f:\n for line in f:\n line_split = line.split()\n matches.append([int(line_split[0]), int(line_split[3]),\n int(line_split[1] == line_split[4])])\n return torch.LongTensor(matches)", "response": "Read the matches file and keep only 3D point ID."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef accuracy(output, target, topk=(1,)):\n with torch.no_grad():\n maxk = max(topk)\n batch_size = target.size(0)\n\n _, pred = output.topk(maxk, 1, True, True)\n pred = pred.t()\n correct = pred.eq(target[None])\n\n res = []\n for k in topk:\n correct_k = correct[:k].flatten().sum(dtype=torch.float32)\n res.append(correct_k * (100.0 / batch_size))\n return res", "response": "Computes the accuracy over the k top predictions for the specified values of k"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef setup_for_distributed(is_master):\n import builtins as __builtin__\n builtin_print = __builtin__.print\n\n def print(*args, **kwargs):\n force = kwargs.pop('force', False)\n if is_master or force:\n builtin_print(*args, **kwargs)\n\n __builtin__.print = print", "response": "This function disables printing when not in master process\n "} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef synchronize_between_processes(self):\n if not is_dist_avail_and_initialized():\n return\n t = torch.tensor([self.count, self.total], dtype=torch.float64, device='cuda')\n dist.barrier()\n dist.all_reduce(t)\n t = t.tolist()\n self.count = int(t[0])\n self.total = t[1]", "response": "Synchronize the deque with the distribution processes."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\ncreates directories if they do not exist.", "response": "def makedir_exist_ok(dirpath):\n \"\"\"\n Python2 support for os.makedirs(.., exist_ok=True)\n \"\"\"\n try:\n os.makedirs(dirpath)\n except OSError as e:\n if e.errno == errno.EEXIST:\n pass\n else:\n raise"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\ndownloading a file from a URL and place it in a directory.", "response": "def download_url(url, root, filename=None, md5=None):\n \"\"\"Download a file from a url and place it in root.\n\n Args:\n url (str): URL to download file from\n root (str): Directory to place downloaded file in\n filename (str, optional): Name to save the file under. If None, use the basename of the URL\n md5 (str, optional): MD5 checksum of the download. If None, do not check\n \"\"\"\n from six.moves import urllib\n\n root = os.path.expanduser(root)\n if not filename:\n filename = os.path.basename(url)\n fpath = os.path.join(root, filename)\n\n makedir_exist_ok(root)\n\n # downloads file\n if os.path.isfile(fpath) and check_integrity(fpath, md5):\n print('Using downloaded and verified file: ' + fpath)\n else:\n try:\n print('Downloading ' + url + ' to ' + fpath)\n urllib.request.urlretrieve(\n url, fpath,\n reporthook=gen_bar_updater()\n )\n except OSError:\n if url[:5] == 'https':\n url = url.replace('https:', 'http:')\n print('Failed download. Trying https -> http instead.'\n ' Downloading ' + url + ' to ' + fpath)\n urllib.request.urlretrieve(\n url, fpath,\n reporthook=gen_bar_updater()\n )"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef list_dir(root, prefix=False):\n root = os.path.expanduser(root)\n directories = list(\n filter(\n lambda p: os.path.isdir(os.path.join(root, p)),\n os.listdir(root)\n )\n )\n\n if prefix is True:\n directories = [os.path.join(root, d) for d in directories]\n\n return directories", "response": "List all directories at a given root"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nlisting all files ending with a suffix at a given root", "response": "def list_files(root, suffix, prefix=False):\n \"\"\"List all files ending with a suffix at a given root\n\n Args:\n root (str): Path to directory whose folders need to be listed\n suffix (str or tuple): Suffix of the files to match, e.g. '.png' or ('.jpg', '.png').\n It uses the Python \"str.endswith\" method and is passed directly\n prefix (bool, optional): If true, prepends the path to each result, otherwise\n only returns the name of the files found\n \"\"\"\n root = os.path.expanduser(root)\n files = list(\n filter(\n lambda p: os.path.isfile(os.path.join(root, p)) and p.endswith(suffix),\n os.listdir(root)\n )\n )\n\n if prefix is True:\n files = [os.path.join(root, d) for d in files]\n\n return files"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef download_file_from_google_drive(file_id, root, filename=None, md5=None):\n # Based on https://stackoverflow.com/questions/38511444/python-download-files-from-google-drive-using-url\n import requests\n url = \"https://docs.google.com/uc?export=download\"\n\n root = os.path.expanduser(root)\n if not filename:\n filename = file_id\n fpath = os.path.join(root, filename)\n\n makedir_exist_ok(root)\n\n if os.path.isfile(fpath) and check_integrity(fpath, md5):\n print('Using downloaded and verified file: ' + fpath)\n else:\n session = requests.Session()\n\n response = session.get(url, params={'id': file_id}, stream=True)\n token = _get_confirm_token(response)\n\n if token:\n params = {'id': file_id, 'confirm': token}\n response = session.get(url, params=params, stream=True)\n\n _save_response_content(response, fpath)", "response": "Download a Google Drive file from Google Drive and place it in root."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\ngets parameters for a random crop.", "response": "def get_params(img, output_size):\n \"\"\"Get parameters for ``crop`` for a random crop.\n\n Args:\n img (PIL Image): Image to be cropped.\n output_size (tuple): Expected output size of the crop.\n\n Returns:\n tuple: params (i, j, h, w) to be passed to ``crop`` for random crop.\n \"\"\"\n w, h = img.size\n th, tw = output_size\n if w == tw and h == th:\n return 0, 0, h, w\n\n i = random.randint(0, h - th)\n j = random.randint(0, w - tw)\n return i, j, th, tw"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef get_params(width, height, distortion_scale):\n half_height = int(height / 2)\n half_width = int(width / 2)\n topleft = (random.randint(0, int(distortion_scale * half_width)),\n random.randint(0, int(distortion_scale * half_height)))\n topright = (random.randint(width - int(distortion_scale * half_width) - 1, width - 1),\n random.randint(0, int(distortion_scale * half_height)))\n botright = (random.randint(width - int(distortion_scale * half_width) - 1, width - 1),\n random.randint(height - int(distortion_scale * half_height) - 1, height - 1))\n botleft = (random.randint(0, int(distortion_scale * half_width)),\n random.randint(height - int(distortion_scale * half_height) - 1, height - 1))\n startpoints = [(0, 0), (width - 1, 0), (width - 1, height - 1), (0, height - 1)]\n endpoints = [topleft, topright, botright, botleft]\n return startpoints, endpoints", "response": "Get parameters for a random perspective transform."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef get_params(img, scale, ratio):\n area = img.size[0] * img.size[1]\n\n for attempt in range(10):\n target_area = random.uniform(*scale) * area\n log_ratio = (math.log(ratio[0]), math.log(ratio[1]))\n aspect_ratio = math.exp(random.uniform(*log_ratio))\n\n w = int(round(math.sqrt(target_area * aspect_ratio)))\n h = int(round(math.sqrt(target_area / aspect_ratio)))\n\n if w <= img.size[0] and h <= img.size[1]:\n i = random.randint(0, img.size[1] - h)\n j = random.randint(0, img.size[0] - w)\n return i, j, h, w\n\n # Fallback to central crop\n in_ratio = img.size[0] / img.size[1]\n if (in_ratio < min(ratio)):\n w = img.size[0]\n h = w / min(ratio)\n elif (in_ratio > max(ratio)):\n h = img.size[1]\n w = h * max(ratio)\n else: # whole image\n w = img.size[0]\n h = img.size[1]\n i = (img.size[1] - h) // 2\n j = (img.size[0] - w) // 2\n return i, j, h, w", "response": "Get parameters for a random sized crop."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef get_params(brightness, contrast, saturation, hue):\n transforms = []\n\n if brightness is not None:\n brightness_factor = random.uniform(brightness[0], brightness[1])\n transforms.append(Lambda(lambda img: F.adjust_brightness(img, brightness_factor)))\n\n if contrast is not None:\n contrast_factor = random.uniform(contrast[0], contrast[1])\n transforms.append(Lambda(lambda img: F.adjust_contrast(img, contrast_factor)))\n\n if saturation is not None:\n saturation_factor = random.uniform(saturation[0], saturation[1])\n transforms.append(Lambda(lambda img: F.adjust_saturation(img, saturation_factor)))\n\n if hue is not None:\n hue_factor = random.uniform(hue[0], hue[1])\n transforms.append(Lambda(lambda img: F.adjust_hue(img, hue_factor)))\n\n random.shuffle(transforms)\n transform = Compose(transforms)\n\n return transform", "response": "Get a randomized transform to be applied on image."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nget parameters for affine transformation", "response": "def get_params(degrees, translate, scale_ranges, shears, img_size):\n \"\"\"Get parameters for affine transformation\n\n Returns:\n sequence: params to be passed to the affine transformation\n \"\"\"\n angle = random.uniform(degrees[0], degrees[1])\n if translate is not None:\n max_dx = translate[0] * img_size[0]\n max_dy = translate[1] * img_size[1]\n translations = (np.round(random.uniform(-max_dx, max_dx)),\n np.round(random.uniform(-max_dy, max_dy)))\n else:\n translations = (0, 0)\n\n if scale_ranges is not None:\n scale = random.uniform(scale_ranges[0], scale_ranges[1])\n else:\n scale = 1.0\n\n if shears is not None:\n shear = random.uniform(shears[0], shears[1])\n else:\n shear = 0.0\n\n return angle, translations, scale, shear"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef inception_v3(pretrained=False, **kwargs):\n if pretrained:\n if 'transform_input' not in kwargs:\n kwargs['transform_input'] = True\n if 'aux_logits' in kwargs:\n original_aux_logits = kwargs['aux_logits']\n kwargs['aux_logits'] = True\n else:\n original_aux_logits = True\n model = Inception3(**kwargs)\n model.load_state_dict(model_zoo.load_url(model_urls['inception_v3_google']))\n if not original_aux_logits:\n model.aux_logits = False\n del model.AuxLogits\n return model\n\n return Inception3(**kwargs)", "response": "r A function to create an inception v3 model."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef download(self):\n import tarfile\n\n if self._check_integrity():\n print('Files already downloaded and verified')\n return\n\n download_url(self.url, self.root, self.filename, self.md5_checksum)\n\n # Extract file\n with tarfile.open(os.path.join(self.root, self.filename), 'r:gz') as tar:\n tar.extractall(path=self.root)\n\n # Download individual photos\n with open(os.path.join(self.root, 'dataset', 'SBU_captioned_photo_dataset_urls.txt')) as fh:\n for line in fh:\n url = line.rstrip()\n try:\n download_url(url, os.path.join(self.root, 'dataset'))\n except OSError:\n # The images point to public images on Flickr.\n # Note: Images might be removed by users at anytime.\n pass", "response": "Download and extract the tarball and download each individual photo."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef googlenet(pretrained=False, **kwargs):\n if pretrained:\n if 'transform_input' not in kwargs:\n kwargs['transform_input'] = True\n if 'aux_logits' not in kwargs:\n kwargs['aux_logits'] = False\n if kwargs['aux_logits']:\n warnings.warn('auxiliary heads in the pretrained googlenet model are NOT pretrained, '\n 'so make sure to train them')\n original_aux_logits = kwargs['aux_logits']\n kwargs['aux_logits'] = True\n kwargs['init_weights'] = False\n model = GoogLeNet(**kwargs)\n model.load_state_dict(model_zoo.load_url(model_urls['googlenet']))\n if not original_aux_logits:\n model.aux_logits = False\n del model.aux1, model.aux2\n return model\n\n return GoogLeNet(**kwargs)", "response": "r A function to create a new image net architectural version of the googlenet model."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef download(self):\n\n if self._check_exists():\n return\n\n makedir_exist_ok(self.raw_folder)\n makedir_exist_ok(self.processed_folder)\n\n # download files\n for url in self.urls:\n filename = url.rpartition('/')[2]\n file_path = os.path.join(self.raw_folder, filename)\n download_url(url, root=self.raw_folder, filename=filename, md5=None)\n self.extract_gzip(gzip_path=file_path, remove_finished=True)\n\n # process and save as torch files\n print('Processing...')\n\n training_set = (\n read_image_file(os.path.join(self.raw_folder, 'train-images-idx3-ubyte')),\n read_label_file(os.path.join(self.raw_folder, 'train-labels-idx1-ubyte'))\n )\n test_set = (\n read_image_file(os.path.join(self.raw_folder, 't10k-images-idx3-ubyte')),\n read_label_file(os.path.join(self.raw_folder, 't10k-labels-idx1-ubyte'))\n )\n with open(os.path.join(self.processed_folder, self.training_file), 'wb') as f:\n torch.save(training_set, f)\n with open(os.path.join(self.processed_folder, self.test_file), 'wb') as f:\n torch.save(test_set, f)\n\n print('Done!')", "response": "Download the MNIST data if it doesn t exist in processed_folder already."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\ndownloading the EMNIST data from the given URL and save it as torch files.", "response": "def download(self):\n \"\"\"Download the EMNIST data if it doesn't exist in processed_folder already.\"\"\"\n import shutil\n import zipfile\n\n if self._check_exists():\n return\n\n makedir_exist_ok(self.raw_folder)\n makedir_exist_ok(self.processed_folder)\n\n # download files\n filename = self.url.rpartition('/')[2]\n file_path = os.path.join(self.raw_folder, filename)\n download_url(self.url, root=self.raw_folder, filename=filename, md5=None)\n\n print('Extracting zip archive')\n with zipfile.ZipFile(file_path) as zip_f:\n zip_f.extractall(self.raw_folder)\n os.unlink(file_path)\n gzip_folder = os.path.join(self.raw_folder, 'gzip')\n for gzip_file in os.listdir(gzip_folder):\n if gzip_file.endswith('.gz'):\n self.extract_gzip(gzip_path=os.path.join(gzip_folder, gzip_file))\n\n # process and save as torch files\n for split in self.splits:\n print('Processing ' + split)\n training_set = (\n read_image_file(os.path.join(gzip_folder, 'emnist-{}-train-images-idx3-ubyte'.format(split))),\n read_label_file(os.path.join(gzip_folder, 'emnist-{}-train-labels-idx1-ubyte'.format(split)))\n )\n test_set = (\n read_image_file(os.path.join(gzip_folder, 'emnist-{}-test-images-idx3-ubyte'.format(split))),\n read_label_file(os.path.join(gzip_folder, 'emnist-{}-test-labels-idx1-ubyte'.format(split)))\n )\n with open(os.path.join(self.processed_folder, self._training_file(split)), 'wb') as f:\n torch.save(training_set, f)\n with open(os.path.join(self.processed_folder, self._test_file(split)), 'wb') as f:\n torch.save(test_set, f)\n shutil.rmtree(gzip_folder)\n\n print('Done!')"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef get_current_theme_name(override=None):\n\n if override and (override in themes or override == '__common__'):\n return override\n theme_name = request.args.get('theme', request.preferences.get_value('theme'))\n if theme_name not in themes:\n theme_name = default_theme\n return theme_name", "response": "Returns theme name.\n\n Checks in this order:\n 1. override\n 2. cookies\n 3. settings"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef index():\n\n # output_format\n output_format = request.form.get('format', 'html')\n if output_format not in ['html', 'csv', 'json', 'rss']:\n output_format = 'html'\n\n # check if there is query\n if request.form.get('q') is None:\n if output_format == 'html':\n return render(\n 'index.html',\n )\n else:\n return index_error(output_format, 'No query'), 400\n\n # search\n search_query = None\n result_container = None\n try:\n search_query = get_search_query_from_webapp(request.preferences, request.form)\n # search = Search(search_query) # without plugins\n search = SearchWithPlugins(search_query, request.user_plugins, request)\n result_container = search.search()\n except Exception as e:\n # log exception\n logger.exception('search error')\n\n # is it an invalid input parameter or something else ?\n if (issubclass(e.__class__, SearxParameterException)):\n return index_error(output_format, e.message), 400\n else:\n return index_error(output_format, gettext('search error')), 500\n\n # results\n results = result_container.get_ordered_results()\n number_of_results = result_container.results_number()\n if number_of_results < result_container.results_length():\n number_of_results = 0\n\n # UI\n advanced_search = request.form.get('advanced_search', None)\n\n # output\n for result in results:\n if output_format == 'html':\n if 'content' in result and result['content']:\n result['content'] = highlight_content(escape(result['content'][:1024]), search_query.query)\n result['title'] = highlight_content(escape(result['title'] or u''), search_query.query)\n else:\n if result.get('content'):\n result['content'] = html_to_text(result['content']).strip()\n # removing html content and whitespace duplications\n result['title'] = ' '.join(html_to_text(result['title']).strip().split())\n\n result['pretty_url'] = prettify_url(result['url'])\n\n # TODO, check if timezone is calculated right\n if 'publishedDate' in result:\n try: # test if publishedDate >= 1900 (datetime module bug)\n result['pubdate'] = result['publishedDate'].strftime('%Y-%m-%d %H:%M:%S%z')\n except ValueError:\n result['publishedDate'] = None\n else:\n if result['publishedDate'].replace(tzinfo=None) >= datetime.now() - timedelta(days=1):\n timedifference = datetime.now() - result['publishedDate'].replace(tzinfo=None)\n minutes = int((timedifference.seconds / 60) % 60)\n hours = int(timedifference.seconds / 60 / 60)\n if hours == 0:\n result['publishedDate'] = gettext(u'{minutes} minute(s) ago').format(minutes=minutes)\n else:\n result['publishedDate'] = gettext(u'{hours} hour(s), {minutes} minute(s) ago').format(hours=hours, minutes=minutes) # noqa\n else:\n result['publishedDate'] = format_date(result['publishedDate'])\n\n if output_format == 'json':\n return Response(json.dumps({'query': search_query.query.decode('utf-8'),\n 'number_of_results': number_of_results,\n 'results': results,\n 'answers': list(result_container.answers),\n 'corrections': list(result_container.corrections),\n 'infoboxes': result_container.infoboxes,\n 'suggestions': list(result_container.suggestions),\n 'unresponsive_engines': list(result_container.unresponsive_engines)},\n default=lambda item: list(item) if isinstance(item, set) else item),\n mimetype='application/json')\n elif output_format == 'csv':\n csv = UnicodeWriter(StringIO())\n keys = ('title', 'url', 'content', 'host', 'engine', 'score')\n csv.writerow(keys)\n for row in results:\n row['host'] = row['parsed_url'].netloc\n csv.writerow([row.get(key, '') for key in keys])\n csv.stream.seek(0)\n response = Response(csv.stream.read(), mimetype='application/csv')\n cont_disp = 'attachment;Filename=searx_-_{0}.csv'.format(search_query.query)\n response.headers.add('Content-Disposition', cont_disp)\n return response\n elif output_format == 'rss':\n response_rss = render(\n 'opensearch_response_rss.xml',\n results=results,\n q=request.form['q'],\n number_of_results=number_of_results,\n base_url=get_base_url(),\n override_theme='__common__',\n )\n return Response(response_rss, mimetype='text/xml')\n\n return render(\n 'results.html',\n results=results,\n q=request.form['q'],\n selected_categories=search_query.categories,\n pageno=search_query.pageno,\n time_range=search_query.time_range,\n number_of_results=format_decimal(number_of_results),\n advanced_search=advanced_search,\n suggestions=result_container.suggestions,\n answers=result_container.answers,\n corrections=result_container.corrections,\n infoboxes=result_container.infoboxes,\n paging=result_container.paging,\n unresponsive_engines=result_container.unresponsive_engines,\n current_language=match_language(search_query.lang,\n LANGUAGE_CODES,\n fallback=settings['search']['language']),\n base_url=get_base_url(),\n theme=get_current_theme_name(),\n favicons=global_favicons[themes.index(get_current_theme_name())]\n )", "response": "Render index page.\n\n Supported outputs: html, json, csv, rss."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef preferences():\n\n # save preferences\n if request.method == 'POST':\n resp = make_response(redirect(urljoin(settings['server']['base_url'], url_for('index'))))\n try:\n request.preferences.parse_form(request.form)\n except ValidationException:\n request.errors.append(gettext('Invalid settings, please edit your preferences'))\n return resp\n return request.preferences.save(resp)\n\n # render preferences\n image_proxy = request.preferences.get_value('image_proxy')\n lang = request.preferences.get_value('language')\n disabled_engines = request.preferences.engines.get_disabled()\n allowed_plugins = request.preferences.plugins.get_enabled()\n\n # stats for preferences page\n stats = {}\n\n for c in categories:\n for e in categories[c]:\n stats[e.name] = {'time': None,\n 'warn_timeout': False,\n 'warn_time': False}\n if e.timeout > settings['outgoing']['request_timeout']:\n stats[e.name]['warn_timeout'] = True\n stats[e.name]['supports_selected_language'] = _is_selected_language_supported(e, request.preferences)\n\n # get first element [0], the engine time,\n # and then the second element [1] : the time (the first one is the label)\n for engine_stat in get_engines_stats()[0][1]:\n stats[engine_stat.get('name')]['time'] = round(engine_stat.get('avg'), 3)\n if engine_stat.get('avg') > settings['outgoing']['request_timeout']:\n stats[engine_stat.get('name')]['warn_time'] = True\n # end of stats\n\n return render('preferences.html',\n locales=settings['locales'],\n current_locale=get_locale(),\n image_proxy=image_proxy,\n engines_by_category=categories,\n stats=stats,\n answerers=[{'info': a.self_info(), 'keywords': a.keywords} for a in answerers],\n disabled_engines=disabled_engines,\n autocomplete_backends=autocomplete_backends,\n shortcuts={y: x for x, y in engine_shortcuts.items()},\n themes=themes,\n plugins=plugins,\n doi_resolvers=settings['doi_resolvers'],\n current_doi_resolver=get_doi_resolver(request.args, request.preferences.get_value('doi_resolver')),\n allowed_plugins=allowed_plugins,\n theme=get_current_theme_name(),\n preferences_url_params=request.preferences.get_as_url_params(),\n base_url=get_base_url(),\n preferences=True)", "response": "Render preferences page and save user preferences"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef request(query, params):\n '''pre-request callback\n params:\n method : POST/GET\n headers : {}\n data : {} # if method == POST\n url : ''\n category: 'search category'\n pageno : 1 # number of the requested page\n '''\n\n offset = (params['pageno'] - 1)\n params['url'] = search_url.format(offset=offset, query=quote(query))\n return params", "response": "pre-request callback\n params:\n method : POST/GET\n headers : {}\n data : {} # if method == POST\n url : ''\n category: 'search category'\n pageno : 1 # number of the requested page"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nposts - response callback resp : requests response object", "response": "def response(resp):\n '''post-response callback\n resp: requests response object\n '''\n results = []\n\n dom = html.fromstring(resp.text)\n\n try:\n number_of_results_string = re.sub('[^0-9]', '', dom.xpath(\n '//a[@class=\"active\" and contains(@href,\"/suchen/dudenonline\")]/span/text()')[0]\n )\n\n results.append({'number_of_results': int(number_of_results_string)})\n\n except:\n logger.debug(\"Couldn't read number of results.\")\n pass\n\n for result in dom.xpath('//section[@class=\"wide\" and not(contains(@style,\"overflow:hidden\"))]'):\n try:\n logger.debug(\"running for %s\" % str(result))\n link = result.xpath('.//h2/a')[0]\n url = link.attrib.get('href')\n title = result.xpath('string(.//h2/a)')\n content = extract_text(result.xpath('.//p'))\n # append result\n results.append({'url': url,\n 'title': title,\n 'content': content})\n except:\n logger.debug('result parse error in:\\n%s', etree.tostring(result, pretty_print=True))\n continue\n\n return results"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef get_themes(templates_path):\n themes = os.listdir(templates_path)\n if '__common__' in themes:\n themes.remove('__common__')\n return themes", "response": "Returns available themes list."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef searx_bang(full_query):\n '''check if the searchQuery contain a bang, and create fitting autocompleter results'''\n # check if there is a query which can be parsed\n if len(full_query.getSearchQuery()) == 0:\n return []\n\n results = []\n\n # check if current query stats with !bang\n first_char = full_query.getSearchQuery()[0]\n if first_char == '!' or first_char == '?':\n if len(full_query.getSearchQuery()) == 1:\n # show some example queries\n # TODO, check if engine is not avaliable\n results.append(first_char + \"images\")\n results.append(first_char + \"wikipedia\")\n results.append(first_char + \"osm\")\n else:\n engine_query = full_query.getSearchQuery()[1:]\n\n # check if query starts with categorie name\n for categorie in categories:\n if categorie.startswith(engine_query):\n results.append(first_char + '{categorie}'.format(categorie=categorie))\n\n # check if query starts with engine name\n for engine in engines:\n if engine.startswith(engine_query.replace('_', ' ')):\n results.append(first_char + '{engine}'.format(engine=engine.replace(' ', '_')))\n\n # check if query starts with engine shortcut\n for engine_shortcut in engine_shortcuts:\n if engine_shortcut.startswith(engine_query):\n results.append(first_char + '{engine_shortcut}'.format(engine_shortcut=engine_shortcut))\n\n # check if current query stats with :bang\n elif first_char == ':':\n if len(full_query.getSearchQuery()) == 1:\n # show some example queries\n results.append(\":en\")\n results.append(\":en_us\")\n results.append(\":english\")\n results.append(\":united_kingdom\")\n else:\n engine_query = full_query.getSearchQuery()[1:]\n\n for lc in language_codes:\n lang_id, lang_name, country, english_name = map(unicode.lower, lc)\n\n # check if query starts with language-id\n if lang_id.startswith(engine_query):\n if len(engine_query) <= 2:\n results.append(u':{lang_id}'.format(lang_id=lang_id.split('-')[0]))\n else:\n results.append(u':{lang_id}'.format(lang_id=lang_id))\n\n # check if query starts with language name\n if lang_name.startswith(engine_query) or english_name.startswith(engine_query):\n results.append(u':{lang_name}'.format(lang_name=lang_name))\n\n # check if query starts with country\n if country.startswith(engine_query.replace('_', ' ')):\n results.append(u':{country}'.format(country=country.replace(' ', '_')))\n\n # remove duplicates\n result_set = set(results)\n\n # remove results which are already contained in the query\n for query_part in full_query.query_parts:\n if query_part in result_set:\n result_set.remove(query_part)\n\n # convert result_set back to list\n return list(result_set)", "response": "check if the searchQuery contains a bang and create fitting autocompleter results"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nparsing response and return list of dictionaries", "response": "def response(resp):\n \"\"\"remove first and last lines to get only json\"\"\"\n json_resp = resp.text[resp.text.find('\\n') + 1:resp.text.rfind('\\n') - 2]\n results = []\n try:\n conversion_rate = float(json.loads(json_resp)['conversion']['converted-amount'])\n except:\n return results\n answer = '{0} {1} = {2} {3}, 1 {1} ({5}) = {4} {3} ({6})'.format(\n resp.search_params['amount'],\n resp.search_params['from'],\n resp.search_params['amount'] * conversion_rate,\n resp.search_params['to'],\n conversion_rate,\n resp.search_params['from_name'],\n resp.search_params['to_name'],\n )\n\n url = 'https://duckduckgo.com/js/spice/currency/1/{0}/{1}'.format(\n resp.search_params['from'].upper(), resp.search_params['to'])\n\n results.append({'answer': answer, 'url': url})\n\n return results"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef custom_gradient(fx, gx, x, fx_gx_manually_stopped=False, name=None):\n def maybe_stop(x):\n if fx_gx_manually_stopped:\n return x\n return tf.stop_gradient(x)\n\n with tf.compat.v1.name_scope(name, 'custom_gradient', [fx, gx, x]):\n fx = tf.convert_to_tensor(value=fx, name='fx')\n # We don't want to bother eagerly computing `gx` since we may not even need\n # it.\n with tf.control_dependencies([fx]):\n if is_list_like(x):\n x = [identity(x_, name='x') for x_ in x]\n else:\n x = [identity(x, name='x')]\n\n if is_list_like(gx):\n gx = [identity(gx_, dtype=fx.dtype, name='gx')\n for gx_ in gx]\n else:\n gx = [identity(gx, dtype=fx.dtype, name='gx')]\n\n override_grad = []\n for x_, gx_ in zip(x, gx):\n # Observe: tf.gradients(f(x), x)[i].shape == x[i].shape\n # thus we check that the user is supplying correct shapes.\n equal_shape = tf.compat.v1.assert_equal(\n tf.shape(input=x_),\n tf.shape(input=gx_),\n message='Each `x` must have the same shape as each `gx`.')\n with tf.control_dependencies([equal_shape]):\n # IEEE754 ensures `(x-x)==0.` and that `0.*x==0.` so we make sure to\n # write the code this way, rather than, e.g.,\n # `sum_x * stop(gx) + stop(fx - sum_x * gx)`.\n # For more discussion regarding the relevant portions of the IEEE754\n # standard, see the StackOverflow question,\n # \"Is there a floating point value of x, for which x-x == 0 is false?\"\n # http://stackoverflow.com/q/2686644\n zeros_like_x_ = x_ - tf.stop_gradient(x_)\n override_grad.append(\n tf.reduce_sum(input_tensor=maybe_stop(gx_) * zeros_like_x_))\n override_grad = sum(override_grad)\n override_grad /= tf.cast(tf.size(input=fx), dtype=fx.dtype.base_dtype)\n\n # Proof of correctness:\n #\n # f(x) = x * stop[gx] + stop[fx - x * gx]\n # = stop[fx]\n #\n # g(x) = grad[fx]\n # = stop[gx] + grad[stop[fx - x * gx]]\n # = stop[gx] + 0\n #\n # Notice that when x is zero it still works:\n # grad[x * stop(gx) + stop(fx - x * gx)] = 1 * stop[gx] + 0 = stop[gx]\n #\n # The proof is similar for the tensor-domain case, except that we\n # `reduce_sum` the `stop[gx] * (x - stop[x])` then rescale by\n # `tf.size(fx)` since this reduced version is broadcast to `fx`.\n return maybe_stop(fx) + override_grad", "response": "Embeds a custom gradient into a tensor."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\ncompute the value of f wrt each of xs.", "response": "def value_and_gradient(f, xs, use_gradient_tape=False, name=None):\n \"\"\"Computes `f(*xs)` and its gradients wrt to `*xs`.\n\n Args:\n f: Python `callable` to be differentiated. If `f` returns a scalar, this\n scalar will be differentiated. If `f` returns a tensor or list of tensors,\n by default a scalar will be computed by adding all their values to produce\n a single scalar. If desired, the tensors can be elementwise multiplied by\n the tensors passed as the `dy` keyword argument to the returned gradient\n function.\n xs: Python list of parameters of f for which to differentiate. (Can also\n be single `Tensor`.)\n use_gradient_tape: Python `bool` indicating that `tf.GradientTape`\n should be used regardless of `tf.executing_eagerly()` status.\n Default value: `False`.\n name: Python `str` name prefixed to ops created by this function.\n Default value: `None` (i.e., `'value_and_gradient'`).\n\n Returns:\n y: `y = f(*xs)`.\n dydx: Gradient of `y` wrt each of `xs`.\n \"\"\"\n with tf.compat.v1.name_scope(name, 'value_and_gradient', [xs]):\n is_xs_list_like = isinstance(xs, (tuple, list))\n if not is_xs_list_like:\n xs = [xs]\n xs = [tf.convert_to_tensor(value=x, name='x{}'.format(i))\n for i, x in enumerate(xs)]\n if tf.executing_eagerly() or use_gradient_tape:\n with tf.GradientTape(watch_accessed_variables=False) as tape:\n for x in xs:\n tape.watch(x)\n y = f(*xs)\n dydx = tape.gradient(y, xs)\n else:\n y = f(*xs)\n dydx = tf.gradients(ys=y, xs=xs)\n if not is_xs_list_like:\n dydx = dydx[0]\n return y, dydx"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef eight_schools_joint_log_prob(\n treatment_effects, treatment_stddevs,\n avg_effect, avg_stddev, school_effects_standard):\n \"\"\"Eight-schools joint log-prob.\"\"\"\n rv_avg_effect = tfd.Normal(loc=0., scale=10.)\n rv_avg_stddev = tfd.Normal(loc=5., scale=1.)\n rv_school_effects_standard = mvn(\n loc=tf.zeros_like(school_effects_standard),\n scale=tf.ones_like(school_effects_standard))\n rv_treatment_effects = mvn(\n loc=(avg_effect + tf.exp(avg_stddev) * school_effects_standard),\n scale=treatment_stddevs)\n return (\n rv_avg_effect.log_prob(avg_effect) +\n rv_avg_stddev.log_prob(avg_stddev) +\n rv_school_effects_standard.log_prob(school_effects_standard) +\n rv_treatment_effects.log_prob(treatment_effects))", "response": "Eight - schools joint log - prob."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef benchmark_eight_schools_hmc(\n num_results=int(5e3),\n num_burnin_steps=int(3e3),\n num_leapfrog_steps=3,\n step_size=0.4):\n \"\"\"Runs HMC on the eight-schools unnormalized posterior.\"\"\"\n\n num_schools = 8\n treatment_effects = tf.constant(\n [28, 8, -3, 7, -1, 1, 18, 12],\n dtype=np.float32,\n name='treatment_effects')\n treatment_stddevs = tf.constant(\n [15, 10, 16, 11, 9, 11, 10, 18],\n dtype=np.float32,\n name='treatment_stddevs')\n\n def unnormalized_posterior_log_prob(\n avg_effect, avg_stddev, school_effects_standard):\n \"\"\"Eight-schools unnormalized log posterior.\"\"\"\n return eight_schools_joint_log_prob(\n treatment_effects, treatment_stddevs,\n avg_effect, avg_stddev, school_effects_standard)\n\n if tf.executing_eagerly():\n sample_chain = tf.function(tfp.mcmc.sample_chain)\n else:\n sample_chain = tfp.mcmc.sample_chain\n\n def computation():\n \"\"\"The benchmark computation.\"\"\"\n _, kernel_results = sample_chain(\n num_results=num_results,\n num_burnin_steps=num_burnin_steps,\n current_state=(\n tf.zeros([], name='init_avg_effect'),\n tf.zeros([], name='init_avg_stddev'),\n tf.ones([num_schools], name='init_school_effects_standard'),\n ),\n kernel=tfp.mcmc.HamiltonianMonteCarlo(\n target_log_prob_fn=unnormalized_posterior_log_prob,\n step_size=step_size,\n num_leapfrog_steps=num_leapfrog_steps))\n\n return kernel_results.is_accepted\n\n # Let's force evaluation of graph to ensure build time is not part of our time\n # trial.\n is_accepted_tensor = computation()\n if not tf.executing_eagerly():\n session = tf.compat.v1.Session()\n session.run(is_accepted_tensor)\n\n start_time = time.time()\n if tf.executing_eagerly():\n is_accepted = computation()\n else:\n is_accepted = session.run(is_accepted_tensor)\n wall_time = time.time() - start_time\n\n num_accepted = np.sum(is_accepted)\n acceptance_rate = np.float32(num_accepted) / np.float32(num_results)\n\n return dict(\n iters=(num_results + num_burnin_steps) * num_leapfrog_steps,\n extras={'acceptance_rate': acceptance_rate},\n wall_time=wall_time)", "response": "Benchmarks the eight - schools unnormalized posterior."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef expand_docstring(**kwargs):\n def _fn_wrapped(fn):\n \"\"\"Original function with modified `__doc__` attribute.\"\"\"\n doc = inspect.cleandoc(fn.__doc__)\n for k, v in six.iteritems(kwargs):\n # Capture each ${k} reference to replace with v.\n # We wrap the replacement in a function so no backslash escapes\n # are processed.\n pattern = r'\\$\\{' + str(k) + r'\\}'\n doc = re.sub(pattern, lambda match: v, doc) # pylint: disable=cell-var-from-loop\n fn.__doc__ = doc\n return fn\n return _fn_wrapped", "response": "Decorator to programmatically expand the docstring."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\ninferring the original name passed into a distribution constructor.", "response": "def _simple_name(distribution):\n \"\"\"Infer the original name passed into a distribution constructor.\n\n Distributions typically follow the pattern of\n with.name_scope(name) as name:\n super(name=name)\n so we attempt to reverse the name-scope transformation to allow\n addressing of RVs by the distribution's original, user-visible\n name kwarg.\n\n Args:\n distribution: a tfd.Distribution instance.\n Returns:\n simple_name: the original name passed into the Distribution.\n\n #### Example\n\n ```\n d1 = tfd.Normal(0., 1., name='x') # d1.name = 'x/'\n d2 = tfd.Normal(0., 1., name='x') # d2.name = 'x_2/'\n _simple_name(d2) # returns 'x'\n\n ```\n\n \"\"\"\n simple_name = distribution.name\n\n # turn 'scope/x/' into 'x'\n if simple_name.endswith('/'):\n simple_name = simple_name.split('/')[-2]\n\n # turn 'x_3' into 'x'\n parts = simple_name.split('_')\n if parts[-1].isdigit():\n simple_name = '_'.join(parts[:-1])\n\n return simple_name"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef _build_custom_rv(distribution, sample_shape, value, name):\n # Program transformations (e.g., `make_log_joint_fn`) assume that\n # the traced constructor has `name` and `value` kwargs, enabling\n # them to override the value of an RV according to its name.\n # User-defined RVs inherit their name from the provided\n # distribution; this helper method exposes the name as a dummy kwarg\n # so that it's visible to program transformations.\n del name # unused\n return RandomVariable(distribution=distribution,\n sample_shape=sample_shape,\n value=value)", "response": "Helper method to build a custom RandomVariable instance."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nwrap an existing distribution as a traceable random variable.", "response": "def as_random_variable(distribution,\n sample_shape=(),\n value=None):\n \"\"\"Wrap an existing distribution as a traceable random variable.\n\n This enables the use of custom or user-provided distributions in\n Edward models. Unlike a bare `RandomVariable` object, this method\n wraps the constructor so it is included in the Edward trace and its\n values can be properly intercepted and overridden.\n\n Where possible, you should prefer the built-in constructors\n (`ed.Normal`, etc); these simultaneously construct a Distribution\n and a RandomVariable object so that the distribution parameters\n themselves may be intercepted and overridden. RVs constructed via\n `as_random_variable()` have a fixed distribution and may not support\n program transformations (e.g, conjugate marginalization) that rely\n on overriding distribution parameters.\n\n Args:\n distribution: tfd.Distribution governing the distribution of the random\n variable, such as sampling and log-probabilities.\n sample_shape: tf.TensorShape of samples to draw from the random variable.\n Default is `()` corresponding to a single sample.\n value: Fixed tf.Tensor to associate with random variable. Must have shape\n `sample_shape + distribution.batch_shape + distribution.event_shape`.\n Default is to sample from random variable according to `sample_shape`.\n\n Returns:\n rv: a `RandomVariable` wrapping the provided distribution.\n\n #### Example\n\n ```python\n from tensorflow_probability import distributions as tfd\n from tensorflow_probability import edward2 as ed\n\n def model():\n # equivalent to ed.Normal(0., 1., name='x')\n return ed.as_random_variable(tfd.Normal(0., 1., name='x'))\n\n log_joint = ed.make_log_joint_fn(model)\n output = log_joint(x=2.)\n ```\n \"\"\"\n\n return _build_custom_rv(distribution=distribution,\n sample_shape=sample_shape,\n value=value,\n name=_simple_name(distribution))"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _make_random_variable(distribution_cls):\n\n @interceptable\n @functools.wraps(distribution_cls, assigned=('__module__', '__name__'))\n @docstring_util.expand_docstring(\n cls=distribution_cls.__name__,\n doc=inspect.cleandoc(distribution_cls.__init__.__doc__ or ''))\n def func(*args, **kwargs):\n # pylint: disable=g-doc-args\n \"\"\"Create a random variable for ${cls}.\n\n See ${cls} for more details.\n\n Returns:\n RandomVariable.\n\n #### Original Docstring for Distribution\n\n ${doc}\n \"\"\"\n # pylint: enable=g-doc-args\n sample_shape = kwargs.pop('sample_shape', ())\n value = kwargs.pop('value', None)\n return RandomVariable(distribution=distribution_cls(*args, **kwargs),\n sample_shape=sample_shape,\n value=value)\n return func", "response": "Factory function to make random variable given distribution class."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nshapes for the mode and mean Tensors.", "response": "def _mode_mean_shape(self):\n \"\"\"Shape for the mode/mean Tensors.\"\"\"\n shape = tensorshape_util.concatenate(self.batch_shape, self.event_shape)\n has_static_shape = tensorshape_util.is_fully_defined(shape)\n if not has_static_shape:\n shape = tf.concat([\n self.batch_shape_tensor(),\n self.event_shape_tensor(),\n ], 0)\n return shape"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef one_step_predictive(model, observed_time_series, parameter_samples):\n\n with tf.compat.v1.name_scope(\n 'one_step_predictive', values=[observed_time_series, parameter_samples]):\n\n [\n observed_time_series,\n is_missing\n ] = sts_util.canonicalize_observed_time_series_with_mask(\n observed_time_series)\n\n # Run filtering over the training timesteps to extract the\n # predictive means and variances.\n num_timesteps = dist_util.prefer_static_value(\n tf.shape(input=observed_time_series))[-2]\n lgssm = model.make_state_space_model(\n num_timesteps=num_timesteps, param_vals=parameter_samples)\n (_, _, _, _, _, observation_means, observation_covs\n ) = lgssm.forward_filter(observed_time_series, mask=is_missing)\n\n # Squeeze dims to convert from LGSSM's event shape `[num_timesteps, 1]`\n # to a scalar time series.\n return sts_util.mix_over_posterior_draws(\n means=observation_means[..., 0],\n variances=observation_covs[..., 0, 0])", "response": "Compute one - step predictive distributions for all timesteps."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nconstruct a predictive distribution over future observations.", "response": "def forecast(model,\n observed_time_series,\n parameter_samples,\n num_steps_forecast):\n \"\"\"Construct predictive distribution over future observations.\n\n Given samples from the posterior over parameters, return the predictive\n distribution over future observations for num_steps_forecast timesteps.\n\n Args:\n model: An instance of `StructuralTimeSeries` representing a\n time-series model. This represents a joint distribution over\n time-series and their parameters with batch shape `[b1, ..., bN]`.\n observed_time_series: `float` `Tensor` of shape\n `concat([sample_shape, model.batch_shape, [num_timesteps, 1]])` where\n `sample_shape` corresponds to i.i.d. observations, and the trailing `[1]`\n dimension may (optionally) be omitted if `num_timesteps > 1`. May\n optionally be an instance of `tfp.sts.MaskedTimeSeries` including a\n mask `Tensor` to encode the locations of missing observations.\n parameter_samples: Python `list` of `Tensors` representing posterior samples\n of model parameters, with shapes `[concat([[num_posterior_draws],\n param.prior.batch_shape, param.prior.event_shape]) for param in\n model.parameters]`. This may optionally also be a map (Python `dict`) of\n parameter names to `Tensor` values.\n num_steps_forecast: scalar `int` `Tensor` number of steps to forecast.\n\n Returns:\n forecast_dist: a `tfd.MixtureSameFamily` instance with event shape\n [num_steps_forecast, 1] and batch shape\n `concat([sample_shape, model.batch_shape])`, with `num_posterior_draws`\n mixture components.\n\n #### Examples\n\n Suppose we've built a model and fit it to data using HMC:\n\n ```python\n day_of_week = tfp.sts.Seasonal(\n num_seasons=7,\n observed_time_series=observed_time_series,\n name='day_of_week')\n local_linear_trend = tfp.sts.LocalLinearTrend(\n observed_time_series=observed_time_series,\n name='local_linear_trend')\n model = tfp.sts.Sum(components=[day_of_week, local_linear_trend],\n observed_time_series=observed_time_series)\n\n samples, kernel_results = tfp.sts.fit_with_hmc(model, observed_time_series)\n ```\n\n Passing the posterior samples into `forecast`, we construct a forecast\n distribution:\n\n ```python\n forecast_dist = tfp.sts.forecast(model, observed_time_series,\n parameter_samples=samples,\n num_steps_forecast=50)\n\n forecast_mean = forecast_dist.mean()[..., 0] # shape: [50]\n forecast_scale = forecast_dist.stddev()[..., 0] # shape: [50]\n forecast_samples = forecast_dist.sample(10)[..., 0] # shape: [10, 50]\n ```\n\n If using variational inference instead of HMC, we'd construct a forecast using\n samples from the variational posterior:\n\n ```python\n (variational_loss,\n variational_distributions) = tfp.sts.build_factored_variational_loss(\n model=model, observed_time_series=observed_time_series)\n\n # OMITTED: take steps to optimize variational loss\n\n samples = {k: q.sample(30) for (k, q) in variational_distributions.items()}\n forecast_dist = tfp.sts.forecast(model, observed_time_series,\n parameter_samples=samples,\n num_steps_forecast=50)\n ```\n\n We can visualize the forecast by plotting:\n\n ```python\n from matplotlib import pylab as plt\n def plot_forecast(observed_time_series,\n forecast_mean,\n forecast_scale,\n forecast_samples):\n plt.figure(figsize=(12, 6))\n\n num_steps = observed_time_series.shape[-1]\n num_steps_forecast = forecast_mean.shape[-1]\n num_steps_train = num_steps - num_steps_forecast\n\n c1, c2 = (0.12, 0.47, 0.71), (1.0, 0.5, 0.05)\n plt.plot(np.arange(num_steps), observed_time_series,\n lw=2, color=c1, label='ground truth')\n\n forecast_steps = np.arange(num_steps_train,\n num_steps_train+num_steps_forecast)\n plt.plot(forecast_steps, forecast_samples.T, lw=1, color=c2, alpha=0.1)\n plt.plot(forecast_steps, forecast_mean, lw=2, ls='--', color=c2,\n label='forecast')\n plt.fill_between(forecast_steps,\n forecast_mean - 2 * forecast_scale,\n forecast_mean + 2 * forecast_scale, color=c2, alpha=0.2)\n\n plt.xlim([0, num_steps])\n plt.legend()\n\n plot_forecast(observed_time_series,\n forecast_mean=forecast_mean,\n forecast_scale=forecast_scale,\n forecast_samples=forecast_samples)\n ```\n\n \"\"\"\n\n with tf.compat.v1.name_scope(\n 'forecast',\n values=[observed_time_series, parameter_samples, num_steps_forecast]):\n [\n observed_time_series,\n mask\n ] = sts_util.canonicalize_observed_time_series_with_mask(\n observed_time_series)\n\n # Run filtering over the observed timesteps to extract the\n # latent state posterior at timestep T+1 (i.e., the final\n # filtering distribution, pushed through the transition model).\n # This is the prior for the forecast model (\"today's prior\n # is yesterday's posterior\").\n num_observed_steps = dist_util.prefer_static_value(\n tf.shape(input=observed_time_series))[-2]\n observed_data_ssm = model.make_state_space_model(\n num_timesteps=num_observed_steps, param_vals=parameter_samples)\n (_, _, _, predictive_means, predictive_covs, _, _\n ) = observed_data_ssm.forward_filter(observed_time_series, mask=mask)\n\n # Build a batch of state-space models over the forecast period. Because\n # we'll use MixtureSameFamily to mix over the posterior draws, we need to\n # do some shenanigans to move the `[num_posterior_draws]` batch dimension\n # from the leftmost to the rightmost side of the model's batch shape.\n # TODO(b/120245392): enhance `MixtureSameFamily` to reduce along an\n # arbitrary axis, and eliminate `move_dimension` calls here.\n parameter_samples = model._canonicalize_param_vals_as_map(parameter_samples) # pylint: disable=protected-access\n parameter_samples_with_reordered_batch_dimension = {\n param.name: dist_util.move_dimension(\n parameter_samples[param.name],\n 0, -(1 + _prefer_static_event_ndims(param.prior)))\n for param in model.parameters}\n forecast_prior = tfd.MultivariateNormalFullCovariance(\n loc=dist_util.move_dimension(predictive_means[..., -1, :], 0, -2),\n covariance_matrix=dist_util.move_dimension(\n predictive_covs[..., -1, :, :], 0, -3))\n\n # Ugly hack: because we moved `num_posterior_draws` to the trailing (rather\n # than leading) dimension of parameters, the parameter batch shapes no\n # longer broadcast against the `constant_offset` attribute used in `sts.Sum`\n # models. We fix this by manually adding an extra broadcasting dim to\n # `constant_offset` if present.\n # The root cause of this hack is that we mucked with param dimensions above\n # and are now passing params that are 'invalid' in the sense that they don't\n # match the shapes of the model's param priors. The fix (as above) will be\n # to update MixtureSameFamily so we can avoid changing param dimensions\n # altogether.\n # TODO(b/120245392): enhance `MixtureSameFamily` to reduce along an\n # arbitrary axis, and eliminate this hack.\n kwargs = {}\n if hasattr(model, 'constant_offset'):\n kwargs['constant_offset'] = tf.convert_to_tensor(\n value=model.constant_offset,\n dtype=forecast_prior.dtype)[..., tf.newaxis]\n\n # We assume that any STS model that has a `constant_offset` attribute\n # will allow it to be overridden as a kwarg. This is currently just\n # `sts.Sum`.\n # TODO(b/120245392): when kwargs hack is removed, switch back to calling\n # the public version of `_make_state_space_model`.\n forecast_ssm = model._make_state_space_model( # pylint: disable=protected-access\n num_timesteps=num_steps_forecast,\n param_map=parameter_samples_with_reordered_batch_dimension,\n initial_state_prior=forecast_prior,\n initial_step=num_observed_steps,\n **kwargs)\n\n num_posterior_draws = dist_util.prefer_static_value(\n forecast_ssm.batch_shape_tensor())[-1]\n return tfd.MixtureSameFamily(\n mixture_distribution=tfd.Categorical(\n logits=tf.zeros([num_posterior_draws], dtype=forecast_ssm.dtype)),\n components_distribution=forecast_ssm)"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nreturn max or mask if max is not finite.", "response": "def _max_mask_non_finite(x, axis=-1, keepdims=False, mask=0):\n \"\"\"Returns `max` or `mask` if `max` is not finite.\"\"\"\n m = np.max(x, axis=_astuple(axis), keepdims=keepdims)\n needs_masking = ~np.isfinite(m)\n if needs_masking.ndim > 0:\n m[needs_masking] = mask\n elif needs_masking:\n m = mask\n return m"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\ncomputes log ( sum ( input_tensor ) along the specified axis.", "response": "def _reduce_logsumexp(input_tensor, axis=None, keepdims=False, name=None): # pylint: disable=unused-argument\n \"\"\"Computes `log(sum(exp(input_tensor))) along the specified axis.\"\"\"\n try:\n return scipy_special.logsumexp(\n input_tensor, axis=_astuple(axis), keepdims=keepdims)\n except NotImplementedError:\n # We offer a non SP version just in case SP isn't installed and this\n # because logsumexp is often used.\n m = _max_mask_non_finite(input_tensor, axis=axis, keepdims=True)\n y = input_tensor - m\n y = np.exp(y, out=y)\n return m + np.log(np.sum(y, axis=_astuple(axis), keepdims=keepdims))"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nassert all elements of `x` are finite. Args: x: Numeric `Tensor`. data: The tensors to print out if the condition is False. Defaults to error message and first few entries of `x`. summarize: Print this many entries of each tensor. message: A string to prefix to the default message. name: A name for this operation (optional). Defaults to \"assert_finite\". Returns: Op raising `InvalidArgumentError` unless `x` has specified rank or lower. If static checks determine `x` has correct rank, a `no_op` is returned. Raises: ValueError: If static checks determine `x` has wrong rank.", "response": "def assert_finite(x, data=None, summarize=None, message=None, name=None):\n \"\"\"Assert all elements of `x` are finite.\n\n Args:\n x: Numeric `Tensor`.\n data: The tensors to print out if the condition is False. Defaults to\n error message and first few entries of `x`.\n summarize: Print this many entries of each tensor.\n message: A string to prefix to the default message.\n name: A name for this operation (optional).\n Defaults to \"assert_finite\".\n\n Returns:\n Op raising `InvalidArgumentError` unless `x` has specified rank or lower.\n If static checks determine `x` has correct rank, a `no_op` is returned.\n\n Raises:\n ValueError: If static checks determine `x` has wrong rank.\n \"\"\"\n with tf.compat.v2.name_scope(name or 'assert_finite'):\n x_ = tf.get_static_value(x)\n if x_ is not None:\n if ~np.all(np.isfinite(x_)):\n raise ValueError(message)\n return x\n assertion = tf.compat.v1.assert_equal(\n tf.math.is_finite(x), tf.ones_like(x, tf.bool),\n data=data, summarize=summarize, message=message)\n with tf.control_dependencies([assertion]):\n return tf.identity(x)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef assert_rank_at_most(x, rank, data=None, summarize=None, message=None,\n name=None):\n \"\"\"Assert `x` has rank equal to `rank` or smaller.\n\n Example of adding a dependency to an operation:\n\n ```python\n with tf.control_dependencies([tf.assert_rank_at_most(x, 2)]):\n output = tf.reduce_sum(x)\n ```\n\n Args:\n x: Numeric `Tensor`.\n rank: Scalar `Tensor`.\n data: The tensors to print out if the condition is False. Defaults to\n error message and first few entries of `x`.\n summarize: Print this many entries of each tensor.\n message: A string to prefix to the default message.\n name: A name for this operation (optional).\n Defaults to \"assert_rank_at_most\".\n\n Returns:\n Op raising `InvalidArgumentError` unless `x` has specified rank or lower.\n If static checks determine `x` has correct rank, a `no_op` is returned.\n\n Raises:\n ValueError: If static checks determine `x` has wrong rank.\n \"\"\"\n with tf.compat.v2.name_scope(name or 'assert_rank_at_most'):\n return tf.compat.v1.assert_less_equal(\n tf.rank(x), rank, data=data, summarize=summarize, message=message)", "response": "A helper function to check that a tensor x has rank equal to rank or smaller."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\ncomputing the number of elements in a tensor with shape event_shape.", "response": "def _event_size(event_shape, name=None):\n \"\"\"Computes the number of elements in a tensor with shape `event_shape`.\n\n Args:\n event_shape: A tensor shape.\n name: The name to use for the tensor op to compute the number of elements\n (if such an op needs to be created).\n\n Returns:\n event_size: The number of elements in `tensor_shape`. Returns a numpy int\n when the number of elements can be computed immediately. Otherwise, returns\n a scalar tensor.\n \"\"\"\n with tf.compat.v1.name_scope(name, 'event_size', [event_shape]):\n event_shape = tf.convert_to_tensor(\n value=event_shape, dtype=tf.int32, name='event_shape')\n\n event_shape_const = tf.get_static_value(event_shape)\n if event_shape_const is not None:\n return np.prod(event_shape_const)\n else:\n return tf.reduce_prod(input_tensor=event_shape)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _eval_all_one_hot(fn, dist, name=None):\n with tf.compat.v1.name_scope(name, 'eval_all_one_hot'):\n event_size = dist.event_shape_tensor()[-1]\n batch_ndims = tf.size(input=dist.batch_shape_tensor())\n # Reshape `eye(d)` to: `[d] + [1]*batch_ndims + [d]`.\n x = tf.reshape(\n tf.eye(event_size, dtype=dist.dtype),\n shape=tf.pad(\n tensor=tf.ones(batch_ndims, tf.int32),\n paddings=[[1, 1]],\n constant_values=event_size))\n # Compute `fn(x)` then cyclically left-transpose one dim.\n perm = tf.pad(tensor=tf.range(1, batch_ndims + 1), paddings=[[0, 1]])\n return tf.transpose(a=fn(dist, x), perm=perm)", "response": "OneHotCategorical helper computing probs cdf etc over its support."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _make_kl_divergence_fn(\n distribution_b,\n use_exact_kl=False,\n test_points_reduce_axis=(), # `None` == \"all\"; () == \"none\".\n test_points_fn=tf.convert_to_tensor,\n weight=None):\n \"\"\"Creates a callable computing `KL[a,b]` from `a`, a `tfd.Distribution`.\"\"\"\n\n if use_exact_kl is None:\n kl_divergence_fn = tfd.kl_divergence\n else:\n # Closure over: test_points_fn, test_points_reduce_axis.\n def kl_divergence_fn(distribution_a, distribution_b):\n z = test_points_fn(distribution_a)\n return tf.reduce_mean(\n input_tensor=distribution_a.log_prob(z) - distribution_b.log_prob(z),\n axis=test_points_reduce_axis)\n\n # Closure over: distribution_b, kl_divergence_fn, weight.\n def _fn(distribution_a):\n \"\"\"Closure that computes KLDiv as a function of `a` as in `KL[a, b]`.\"\"\"\n with tf.compat.v1.name_scope('kldivergence_loss'):\n # TODO(b/119756336): Due to eager/graph Jacobian graph caching bug\n # we add here the capability for deferred construction of the prior.\n # This capability can probably be removed once b/119756336 is resolved.\n distribution_b_ = (distribution_b() if callable(distribution_b)\n else distribution_b)\n kl = kl_divergence_fn(distribution_a, distribution_b_)\n if weight is not None:\n kl = tf.cast(weight, dtype=kl.dtype) * kl\n # Losses appended with the model.add_loss and are expected to be a single\n # scalar, unlike model.loss, which is expected to be the loss per sample.\n # Therefore, we reduce over all dimensions, regardless of the shape.\n # We take the sum because (apparently) Keras will add this to the *post*\n # `reduce_sum` (total) loss.\n # TODO(b/126259176): Add end-to-end Keras/TFP test to ensure the API's\n # align, particularly wrt how losses are aggregated (across batch\n # members).\n return tf.reduce_sum(input_tensor=kl, name='batch_total_kl_divergence')\n\n return _fn", "response": "Creates a callable that computes KLDiv from a distribution and b."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nreturn a convert - to - tensor func given a name config callable etc.", "response": "def _get_convert_to_tensor_fn(identifier):\n \"\"\"Return a convert-to-tensor func, given a name, config, callable, etc.\"\"\"\n if identifier is None:\n return None\n\n if isinstance(identifier, six.string_types):\n identifier = str(identifier)\n return _deserialize(identifier)\n\n if isinstance(identifier, dict):\n return _deserialize(identifier)\n\n if isinstance(identifier, property):\n identifier = identifier.fget\n if callable(identifier):\n return identifier\n\n raise ValueError('Could not interpret '\n 'convert-to-tensor function identifier:', identifier)"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef get_config(self):\n config = {\n 'make_distribution_fn': _serialize_function(self._make_distribution_fn),\n 'convert_to_tensor_fn': _serialize(self._convert_to_tensor_fn),\n }\n base_config = super(DistributionLambda, self).get_config()\n return dict(list(base_config.items()) + list(config.items()))", "response": "Returns the config of this layer."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\ncreates a multivariate normal triangular distribution from a params vector.", "response": "def new(params, event_size, validate_args=False, name=None):\n \"\"\"Create the distribution instance from a `params` vector.\"\"\"\n with tf.compat.v1.name_scope(name, 'MultivariateNormalTriL',\n [params, event_size]):\n params = tf.convert_to_tensor(value=params, name='params')\n scale_tril = tfb.ScaleTriL(\n diag_shift=np.array(1e-5, params.dtype.as_numpy_dtype()),\n validate_args=validate_args)\n return tfd.MultivariateNormalTriL(\n loc=params[..., :event_size],\n scale_tril=scale_tril(params[..., event_size:]),\n validate_args=validate_args)"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef params_size(event_size, name=None):\n with tf.compat.v1.name_scope(name, 'MultivariateNormalTriL_params_size',\n [event_size]):\n return event_size + event_size * (event_size + 1) // 2", "response": "The number of params needed to create a single distribution."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\ncreating a OneHotCategorical distribution instance from a params vector.", "response": "def new(params, event_size, dtype=None, validate_args=False, name=None):\n \"\"\"Create the distribution instance from a `params` vector.\"\"\"\n with tf.compat.v1.name_scope(name, 'OneHotCategorical',\n [params, event_size]):\n return tfd.OneHotCategorical(\n logits=params,\n dtype=dtype or params.dtype.base_dtype,\n validate_args=validate_args)"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\ncreate a distribution instance from a params vector.", "response": "def new(params, event_size, num_components,\n dtype=None, validate_args=False, name=None):\n \"\"\"Create the distribution instance from a `params` vector.\"\"\"\n with tf.compat.v1.name_scope(name, 'CategoricalMixtureOfOneHotCategorical',\n [params, event_size, num_components]):\n dist = MixtureSameFamily.new(\n params,\n num_components,\n OneHotCategorical(\n event_size,\n validate_args=False, # So we can eval on simplex interior.\n name=name),\n validate_args=validate_args,\n name=name)\n # pylint: disable=protected-access\n dist._mean = functools.partial(\n _eval_all_one_hot, tfd.Distribution.prob, dist)\n dist.log_mean = functools.partial(\n _eval_all_one_hot, tfd.Distribution.log_prob, dist)\n # pylint: enable=protected-access\n return dist"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\ncreating a new IndependentBernoulli distribution instance from a params vector.", "response": "def new(params, event_shape=(), dtype=None, validate_args=False, name=None):\n \"\"\"Create the distribution instance from a `params` vector.\"\"\"\n with tf.compat.v1.name_scope(name, 'IndependentBernoulli',\n [params, event_shape]):\n params = tf.convert_to_tensor(value=params, name='params')\n event_shape = dist_util.expand_to_vector(\n tf.convert_to_tensor(\n value=event_shape, name='event_shape', dtype_hint=tf.int32),\n tensor_name='event_shape')\n new_shape = tf.concat([\n tf.shape(input=params)[:-1],\n event_shape,\n ], axis=0)\n dist = tfd.Independent(\n tfd.Bernoulli(\n logits=tf.reshape(params, new_shape),\n dtype=dtype or params.dtype.base_dtype,\n validate_args=validate_args),\n reinterpreted_batch_ndims=tf.size(input=event_shape),\n validate_args=validate_args)\n dist._logits = dist.distribution._logits # pylint: disable=protected-access\n dist._probs = dist.distribution._probs # pylint: disable=protected-access\n dist.logits = tfd.Bernoulli.logits\n dist.probs = tfd.Bernoulli.probs\n return dist"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nreturns the config of this Keras object.", "response": "def get_config(self):\n \"\"\"Returns the config of this layer.\n\n NOTE: At the moment, this configuration can only be serialized if the\n Layer's `convert_to_tensor_fn` is a serializable Keras object (i.e.,\n implements `get_config`) or one of the standard values:\n - `Distribution.sample` (or `\"sample\"`)\n - `Distribution.mean` (or `\"mean\"`)\n - `Distribution.mode` (or `\"mode\"`)\n - `Distribution.stddev` (or `\"stddev\"`)\n - `Distribution.variance` (or `\"variance\"`)\n \"\"\"\n config = {\n 'event_shape': self._event_shape,\n 'convert_to_tensor_fn': _serialize(self._convert_to_tensor_fn),\n 'sample_dtype': self._sample_dtype,\n 'validate_args': self._validate_args\n }\n base_config = super(IndependentBernoulli, self).get_config()\n return dict(list(base_config.items()) + list(config.items()))"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\ncreate a new IndependentLogistic distribution from a params vector.", "response": "def new(params, event_shape=(), validate_args=False, name=None):\n \"\"\"Create the distribution instance from a `params` vector.\"\"\"\n with tf.compat.v1.name_scope(name, 'IndependentLogistic',\n [params, event_shape]):\n params = tf.convert_to_tensor(value=params, name='params')\n event_shape = dist_util.expand_to_vector(\n tf.convert_to_tensor(\n value=event_shape, name='event_shape', dtype_hint=tf.int32),\n tensor_name='event_shape')\n output_shape = tf.concat([\n tf.shape(input=params)[:-1],\n event_shape,\n ],\n axis=0)\n loc_params, scale_params = tf.split(params, 2, axis=-1)\n return tfd.Independent(\n tfd.Logistic(\n loc=tf.reshape(loc_params, output_shape),\n scale=tf.math.softplus(tf.reshape(scale_params, output_shape)),\n validate_args=validate_args),\n reinterpreted_batch_ndims=tf.size(input=event_shape),\n validate_args=validate_args)"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\ncreates a new IndependentPoisson distribution instance from a params vector.", "response": "def new(params, event_shape=(), validate_args=False, name=None):\n \"\"\"Create the distribution instance from a `params` vector.\"\"\"\n with tf.compat.v1.name_scope(name, 'IndependentPoisson',\n [params, event_shape]):\n params = tf.convert_to_tensor(value=params, name='params')\n event_shape = dist_util.expand_to_vector(\n tf.convert_to_tensor(\n value=event_shape, name='event_shape', dtype_hint=tf.int32),\n tensor_name='event_shape')\n output_shape = tf.concat([\n tf.shape(input=params)[:-1],\n event_shape,\n ],\n axis=0)\n return tfd.Independent(\n tfd.Poisson(\n log_rate=tf.reshape(params, output_shape),\n validate_args=validate_args),\n reinterpreted_batch_ndims=tf.size(input=event_shape),\n validate_args=validate_args)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef new(params, num_components, component_layer,\n validate_args=False, name=None):\n \"\"\"Create the distribution instance from a `params` vector.\"\"\"\n with tf.compat.v1.name_scope(name, 'MixtureSameFamily',\n [params, num_components, component_layer]):\n params = tf.convert_to_tensor(value=params, name='params')\n num_components = tf.convert_to_tensor(\n value=num_components, name='num_components', dtype_hint=tf.int32)\n\n components_dist = component_layer(\n tf.reshape(\n params[..., num_components:],\n tf.concat([tf.shape(input=params)[:-1], [num_components, -1]],\n axis=0)))\n mixture_dist = tfd.Categorical(logits=params[..., :num_components])\n return tfd.MixtureSameFamily(\n mixture_dist,\n components_dist,\n # TODO(b/120154797): Change following to `validate_args=True` after\n # fixing: \"ValueError: `mixture_distribution` must have scalar\n # `event_dim`s.\" assertion in MixtureSameFamily.\n validate_args=False)", "response": "Create a mixture - same - family distribution from a params vector."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nreturning the number of parameters needed to create a mixture of the same family.", "response": "def params_size(num_components, component_params_size, name=None):\n \"\"\"Number of `params` needed to create a `MixtureSameFamily` distribution.\n\n Arguments:\n num_components: Number of component distributions in the mixture\n distribution.\n component_params_size: Number of parameters needed to create a single\n component distribution.\n name: The name to use for the op to compute the number of parameters\n (if such an op needs to be created).\n\n Returns:\n params_size: The number of parameters needed to create the mixture\n distribution.\n \"\"\"\n with tf.compat.v1.name_scope(name, 'MixtureSameFamily_params_size',\n [num_components, component_params_size]):\n num_components = tf.convert_to_tensor(\n value=num_components, name='num_components', dtype_hint=tf.int32)\n component_params_size = tf.convert_to_tensor(\n value=component_params_size, name='component_params_size')\n\n num_components = dist_util.prefer_static_value(num_components)\n component_params_size = dist_util.prefer_static_value(\n component_params_size)\n\n return num_components + num_components * component_params_size"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef params_size(num_components, event_shape=(), name=None):\n return MixtureSameFamily.params_size(\n num_components,\n IndependentNormal.params_size(event_shape, name=name),\n name=name)", "response": "The number of params needed to create a single distribution."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\ncreating the distribution instance from a params vector.", "response": "def new(params, num_components, event_shape=(),\n validate_args=False, name=None):\n \"\"\"Create the distribution instance from a `params` vector.\"\"\"\n return MixtureSameFamily.new(\n params,\n num_components,\n IndependentLogistic(\n event_shape, validate_args=validate_args, name=name),\n validate_args=validate_args,\n name=name)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef get_next_interceptor():\n try:\n interceptor = _interceptor_stack.stack.pop()\n yield interceptor\n finally:\n _interceptor_stack.stack.append(interceptor)", "response": "Yields the next interceptor on the thread - local interceptor stack."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\ngenerate synthetic data for binary classification.", "response": "def toy_logistic_data(num_examples, input_size=2, weights_prior_stddev=5.0):\n \"\"\"Generates synthetic data for binary classification.\n\n Args:\n num_examples: The number of samples to generate (scalar Python `int`).\n input_size: The input space dimension (scalar Python `int`).\n weights_prior_stddev: The prior standard deviation of the weight\n vector. (scalar Python `float`).\n\n Returns:\n random_weights: Sampled weights as a Numpy `array` of shape\n `[input_size]`.\n random_bias: Sampled bias as a scalar Python `float`.\n design_matrix: Points sampled uniformly from the cube `[-1,\n 1]^{input_size}`, as a Numpy `array` of shape `(num_examples,\n input_size)`.\n labels: Labels sampled from the logistic model `p(label=1) =\n logistic(dot(features, random_weights) + random_bias)`, as a Numpy\n `int32` `array` of shape `(num_examples, 1)`.\n \"\"\"\n random_weights = weights_prior_stddev * np.random.randn(input_size)\n random_bias = np.random.randn()\n design_matrix = np.random.rand(num_examples, input_size) * 2 - 1\n logits = np.reshape(\n np.dot(design_matrix, random_weights) + random_bias,\n (-1, 1))\n p_labels = 1. / (1 + np.exp(-logits))\n labels = np.int32(p_labels > np.random.rand(num_examples, 1))\n return random_weights, random_bias, np.float32(design_matrix), labels"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef build_input_pipeline(x, y, batch_size):\n training_dataset = tf.data.Dataset.from_tensor_slices((x, y))\n training_batches = training_dataset.repeat().batch(batch_size)\n training_iterator = tf.compat.v1.data.make_one_shot_iterator(training_batches)\n batch_features, batch_labels = training_iterator.get_next()\n return batch_features, batch_labels", "response": "Build a Dataset iterator for supervised classification."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nvalidate map_values if validate_args == True.", "response": "def _maybe_check_valid_map_values(map_values, validate_args):\n \"\"\"Validate `map_values` if `validate_args`==True.\"\"\"\n assertions = []\n\n message = 'Rank of map_values must be 1.'\n if tensorshape_util.rank(map_values.shape) is not None:\n if tensorshape_util.rank(map_values.shape) != 1:\n raise ValueError(message)\n elif validate_args:\n assertions.append(assert_util.assert_rank(map_values, 1, message=message))\n\n message = 'Size of map_values must be greater than 0.'\n if tensorshape_util.num_elements(map_values.shape) is not None:\n if tensorshape_util.num_elements(map_values.shape) == 0:\n raise ValueError(message)\n elif validate_args:\n assertions.append(\n assert_util.assert_greater(\n tf.size(input=map_values), 0, message=message))\n\n if validate_args:\n assertions.append(\n assert_util.assert_equal(\n tf.math.is_strictly_increasing(map_values),\n True,\n message='map_values is not strictly increasing.'))\n\n return assertions"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\ncalls a transition operator with args unpacking args if its a sequence.", "response": "def call_fn(fn: TransitionOperator, args: Union[Tuple[Any], Any]) -> Any:\n \"\"\"Calls a transition operator with args, unpacking args if its a sequence.\n\n Args:\n fn: A `TransitionOperator`.\n args: Arguments to `fn`\n\n Returns:\n ret: Return value of `fn`.\n \"\"\"\n\n if isinstance(args, (list, tuple)) and not mcmc_util.is_namedtuple_like(args):\n args = args # type: Tuple[Any]\n return fn(*args)\n else:\n return fn(args)"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\ncalls fn and returns the gradients with respect to args.", "response": "def call_and_grads(fn: TransitionOperator, args: Union[Tuple[Any], Any]\n ) -> Tuple[tf.Tensor, TensorNest, TensorNest]:\n \"\"\"Calls `fn` and returns the gradients with respect to `fn`'s first output.\n\n Args:\n fn: A `TransitionOperator`.\n args: Arguments to `fn`\n\n Returns:\n ret: First output of `fn`.\n extra: Second output of `fn`.\n grads: Gradients of `ret` with respect to `args`.\n \"\"\"\n with tf.GradientTape() as tape:\n tape.watch(args)\n ret, extra = call_fn(fn, args)\n grads = tape.gradient(ret, args)\n return ret, extra, grads"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\ntransforms a log - prob function using a bijector.", "response": "def transform_log_prob_fn(log_prob_fn: PotentialFn,\n bijector: BijectorNest,\n init_state: State = None\n ) -> Union[PotentialFn, Tuple[PotentialFn, State]]:\n \"\"\"Transforms a log-prob function using a bijector.\n\n This takes a log-prob function and creates a new log-prob function that now\n takes takes state in the domain of the bijector, forward transforms that state\n and calls the original log-prob function. It then returns the log-probability\n that correctly accounts for this transformation.\n\n The forward-transformed state is pre-pended to the original log-prob\n function's extra returns and returned as the new extra return.\n\n For convenience you can also pass the initial state (in the original space),\n and this function will return the inverse transformed as the 2nd return value.\n You'd use this to initialize MCMC operators that operate in the transformed\n space.\n\n Args:\n log_prob_fn: Log prob fn.\n bijector: Bijector(s), must be of the same structure as the `log_prob_fn`\n inputs.\n init_state: Initial state, in the original space.\n\n Returns:\n transformed_log_prob_fn: Transformed log prob fn.\n transformed_init_state: If `init_state` is provided. Initial state in the\n transformed space.\n \"\"\"\n\n def wrapper(*args):\n \"\"\"Transformed wrapper.\"\"\"\n bijector_ = bijector\n\n args = tf.nest.map_structure(lambda x: 0. + x, args)\n if len(args) == 1:\n args = args[0]\n elif isinstance(bijector_, list):\n bijector_ = tuple(bijector_)\n\n original_space_args = tf.nest.map_structure(lambda b, x: b.forward(x),\n bijector_, args)\n original_space_args = original_space_args # type: Tuple[Any]\n original_space_log_prob, extra = call_fn(log_prob_fn, original_space_args)\n event_ndims = tf.nest.map_structure(\n lambda x: tf.rank(x) - tf.rank(original_space_log_prob), args)\n\n return original_space_log_prob + sum(\n tf.nest.flatten(\n tf.nest.map_structure(\n lambda b, x, e: b.forward_log_det_jacobian(x, event_ndims=e),\n bijector_, args, event_ndims))), [original_space_args, extra]\n\n if init_state is None:\n return wrapper\n else:\n return wrapper, tf.nest.map_structure(lambda b, s: b.inverse(s), bijector,\n init_state)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nleapfrog `TransitionOperator`. Args: leapfrog_step_state: LeapFrogStepState. step_size: Step size, structure broadcastable to the `target_log_prob_fn` state. target_log_prob_fn: Target log prob fn. kinetic_energy_fn: Kinetic energy fn. Returns: leapfrog_step_state: LeapFrogStepState. leapfrog_step_extras: LeapFrogStepExtras.", "response": "def leapfrog_step(leapfrog_step_state: LeapFrogStepState,\n step_size: FloatTensor, target_log_prob_fn: PotentialFn,\n kinetic_energy_fn: PotentialFn\n ) -> Tuple[LeapFrogStepState, LeapFrogStepExtras]:\n \"\"\"Leapfrog `TransitionOperator`.\n\n Args:\n leapfrog_step_state: LeapFrogStepState.\n step_size: Step size, structure broadcastable to the `target_log_prob_fn`\n state.\n target_log_prob_fn: Target log prob fn.\n kinetic_energy_fn: Kinetic energy fn.\n\n Returns:\n leapfrog_step_state: LeapFrogStepState.\n leapfrog_step_extras: LeapFrogStepExtras.\n \"\"\"\n state = leapfrog_step_state.state\n state_grads = leapfrog_step_state.state_grads\n momentum = leapfrog_step_state.momentum\n step_size = maybe_broadcast_structure(step_size, state)\n\n state = tf.nest.map_structure(tf.convert_to_tensor, state)\n momentum = tf.nest.map_structure(tf.convert_to_tensor, momentum)\n state = tf.nest.map_structure(tf.convert_to_tensor, state)\n\n if state_grads is None:\n _, _, state_grads = call_and_grads(target_log_prob_fn, state)\n else:\n state_grads = tf.nest.map_structure(tf.convert_to_tensor, state_grads)\n\n momentum = tf.nest.map_structure(lambda m, sg, s: m + 0.5 * sg * s, momentum,\n state_grads, step_size)\n\n kinetic_energy, kinetic_energy_extra, momentum_grads = call_and_grads(\n kinetic_energy_fn, momentum)\n\n state = tf.nest.map_structure(lambda x, mg, s: x + mg * s, state,\n momentum_grads, step_size)\n\n target_log_prob, state_extra, state_grads = call_and_grads(\n target_log_prob_fn, state)\n\n momentum = tf.nest.map_structure(lambda m, sg, s: m + 0.5 * sg * s, momentum,\n state_grads, step_size)\n\n return LeapFrogStepState(state, state_grads, momentum), LeapFrogStepExtras(\n target_log_prob, state_extra, kinetic_energy, kinetic_energy_extra)"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef metropolis_hastings_step(current_state: State,\n proposed_state: State,\n energy_change: FloatTensor,\n seed=None) -> Tuple[State, tf.Tensor, tf.Tensor]:\n \"\"\"Metropolis-Hastings step.\n\n This probabilistically chooses between `current_state` and `proposed_state`\n based on the `energy_change` so as to preserve detailed balance.\n\n Energy change is the negative of `log_accept_ratio`.\n\n Args:\n current_state: Current state.\n proposed_state: Proposed state.\n energy_change: E(proposed_state) - E(previous_state).\n seed: For reproducibility.\n\n Returns:\n new_state: The chosen state.\n is_accepted: Whether the proposed state was accepted.\n log_uniform: The random number that was used to select between the two\n states.\n \"\"\"\n flat_current = tf.nest.flatten(current_state)\n flat_proposed = nest.flatten_up_to(current_state, proposed_state)\n # Impute the None's in the current state.\n flat_current = [\n p if c is None else c for p, c in zip(flat_proposed, flat_current)\n ]\n current_state = tf.nest.pack_sequence_as(current_state, flat_current)\n\n current_state = tf.nest.map_structure(tf.convert_to_tensor, current_state)\n proposed_state = tf.nest.map_structure(tf.convert_to_tensor, proposed_state)\n energy_change = tf.convert_to_tensor(value=energy_change)\n\n log_accept_ratio = -energy_change\n\n log_uniform = tf.math.log(\n tf.random.uniform(\n shape=tf.shape(input=log_accept_ratio),\n dtype=log_accept_ratio.dtype.base_dtype,\n seed=seed))\n is_accepted = log_uniform < log_accept_ratio\n\n next_state = mcmc_util.choose(\n is_accepted, proposed_state, current_state, name='choose_next_state')\n return next_state, is_accepted, log_uniform", "response": "Metropolis - Hastings step."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef sign_adaptation(control: FloatNest,\n output: FloatTensor,\n set_point: FloatTensor,\n adaptation_rate: FloatTensor = 0.01) -> FloatNest:\n \"\"\"A function to do simple sign-based control of a variable.\n\n ```\n control = control * (1. + adaptation_rate) ** sign(output - set_point)\n ```\n\n Args:\n control: The control variable.\n output: The output variable.\n set_point: The set point for `output`. This function will adjust `control`\n so that `output` matches `set_point`.\n adaptation_rate: Adaptation rate.\n\n Returns:\n control: New control.\n \"\"\"\n\n def _get_new_control(control, output, set_point):\n new_control = mcmc_util.choose(output > set_point,\n control * (1. + adaptation_rate),\n control / (1. + adaptation_rate))\n return new_control\n\n output = maybe_broadcast_structure(output, control)\n set_point = maybe_broadcast_structure(set_point, control)\n\n return tf.nest.map_structure(_get_new_control, control, output, set_point)", "response": "A function to do simple sign - based adaptation of a variable."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef compute_output_shape(self, input_shape):\n input_shape = tf.TensorShape(input_shape).as_list()\n if self.data_format == 'channels_last':\n space = input_shape[1:-1]\n new_space = []\n for i in range(len(space)):\n new_dim = tf_layers_util.conv_output_length(\n space[i],\n self.kernel_size[i],\n padding=self.padding,\n stride=self.strides[i],\n dilation=self.dilation_rate[i])\n new_space.append(new_dim)\n return tf.TensorShape([input_shape[0]] + new_space + [self.filters])\n else:\n space = input_shape[2:]\n new_space = []\n for i in range(len(space)):\n new_dim = tf_layers_util.conv_output_length(\n space[i],\n self.kernel_size[i],\n padding=self.padding,\n stride=self.strides[i],\n dilation=self.dilation_rate[i])\n new_space.append(new_dim)\n return tf.TensorShape([input_shape[0], self.filters] + new_space)", "response": "Computes the output shape of the layer."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nreturning the config of the current layer.", "response": "def get_config(self):\n \"\"\"Returns the config of the layer.\n\n A layer config is a Python dictionary (serializable) containing the\n configuration of a layer. The same layer can be reinstantiated later\n (without its trained weights) from this configuration.\n\n Returns:\n config: A Python dictionary of class keyword arguments and their\n serialized values.\n \"\"\"\n config = {\n 'filters': self.filters,\n 'kernel_size': self.kernel_size,\n 'strides': self.strides,\n 'padding': self.padding,\n 'data_format': self.data_format,\n 'dilation_rate': self.dilation_rate,\n 'activation': (tf.keras.activations.serialize(self.activation)\n if self.activation else None),\n 'activity_regularizer':\n tf.keras.initializers.serialize(self.activity_regularizer),\n }\n function_keys = [\n 'kernel_posterior_fn',\n 'kernel_posterior_tensor_fn',\n 'kernel_prior_fn',\n 'kernel_divergence_fn',\n 'bias_posterior_fn',\n 'bias_posterior_tensor_fn',\n 'bias_prior_fn',\n 'bias_divergence_fn',\n ]\n for function_key in function_keys:\n function = getattr(self, function_key)\n if function is None:\n function_name = None\n function_type = None\n else:\n function_name, function_type = tfp_layers_util.serialize_function(\n function)\n config[function_key] = function_name\n config[function_key + '_type'] = function_type\n base_config = super(_ConvVariational, self).get_config()\n return dict(list(base_config.items()) + list(config.items()))"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\ncreate a new node from its config dictionary.", "response": "def from_config(cls, config):\n \"\"\"Creates a layer from its config.\n\n This method is the reverse of `get_config`, capable of instantiating the\n same layer from the config dictionary.\n\n Args:\n config: A Python dictionary, typically the output of `get_config`.\n\n Returns:\n layer: A layer instance.\n \"\"\"\n config = config.copy()\n function_keys = [\n 'kernel_posterior_fn',\n 'kernel_posterior_tensor_fn',\n 'kernel_prior_fn',\n 'kernel_divergence_fn',\n 'bias_posterior_fn',\n 'bias_posterior_tensor_fn',\n 'bias_prior_fn',\n 'bias_divergence_fn',\n ]\n for function_key in function_keys:\n serial = config[function_key]\n function_type = config.pop(function_key + '_type')\n if serial is not None:\n config[function_key] = tfp_layers_util.deserialize_function(\n serial,\n function_type=function_type)\n return cls(**config)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef get_config(self):\n config = {\n 'seed': self.seed,\n }\n base_config = super(_ConvFlipout, self).get_config()\n return dict(list(base_config.items()) + list(config.items()))", "response": "Returns the config of the layer."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _create_scale_operator(self, identity_multiplier, diag, tril,\n perturb_diag, perturb_factor, shift, validate_args,\n dtype):\n \"\"\"Construct `scale` from various components.\n\n Args:\n identity_multiplier: floating point rank 0 `Tensor` representing a scaling\n done to the identity matrix.\n diag: Floating-point `Tensor` representing the diagonal matrix.`diag` has\n shape `[N1, N2, ... k]`, which represents a k x k diagonal matrix.\n tril: Floating-point `Tensor` representing the lower triangular matrix.\n `tril` has shape `[N1, N2, ... k, k]`, which represents a k x k lower\n triangular matrix.\n perturb_diag: Floating-point `Tensor` representing the diagonal matrix of\n the low rank update.\n perturb_factor: Floating-point `Tensor` representing factor matrix.\n shift: Floating-point `Tensor` representing `shift in `scale @ X + shift`.\n validate_args: Python `bool` indicating whether arguments should be\n checked for correctness.\n dtype: `DType` for arg `Tensor` conversions.\n\n Returns:\n scale. In the case of scaling by a constant, scale is a\n floating point `Tensor`. Otherwise, scale is a `LinearOperator`.\n\n Raises:\n ValueError: if all of `tril`, `diag` and `identity_multiplier` are `None`.\n \"\"\"\n identity_multiplier = _as_tensor(identity_multiplier, \"identity_multiplier\",\n dtype)\n diag = _as_tensor(diag, \"diag\", dtype)\n tril = _as_tensor(tril, \"tril\", dtype)\n perturb_diag = _as_tensor(perturb_diag, \"perturb_diag\", dtype)\n perturb_factor = _as_tensor(perturb_factor, \"perturb_factor\", dtype)\n\n # If possible, use the low rank update to infer the shape of\n # the identity matrix, when scale represents a scaled identity matrix\n # with a low rank update.\n shape_hint = None\n if perturb_factor is not None:\n shape_hint = distribution_util.dimension_size(perturb_factor, axis=-2)\n\n if self._is_only_identity_multiplier:\n if validate_args:\n return distribution_util.with_dependencies([\n assert_util.assert_none_equal(\n identity_multiplier, tf.zeros([], identity_multiplier.dtype),\n [\"identity_multiplier should be non-zero.\"])\n ], identity_multiplier)\n return identity_multiplier\n\n scale = distribution_util.make_tril_scale(\n loc=shift,\n scale_tril=tril,\n scale_diag=diag,\n scale_identity_multiplier=identity_multiplier,\n validate_args=validate_args,\n assert_positive=False,\n shape_hint=shape_hint)\n\n if perturb_factor is not None:\n return tf.linalg.LinearOperatorLowRankUpdate(\n scale,\n u=perturb_factor,\n diag_update=perturb_diag,\n is_diag_update_positive=perturb_diag is None,\n is_non_singular=True, # Implied by is_positive_definite=True.\n is_self_adjoint=True,\n is_positive_definite=True,\n is_square=True)\n\n return scale", "response": "Constructs a LinearOperator object for scaling a set of identity matrices."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef random_walk_normal_fn(scale=1., name=None):\n def _fn(state_parts, seed):\n \"\"\"Adds a normal perturbation to the input state.\n\n Args:\n state_parts: A list of `Tensor`s of any shape and real dtype representing\n the state parts of the `current_state` of the Markov chain.\n seed: `int` or None. The random seed for this `Op`. If `None`, no seed is\n applied.\n Default value: `None`.\n\n Returns:\n perturbed_state_parts: A Python `list` of The `Tensor`s. Has the same\n shape and type as the `state_parts`.\n\n Raises:\n ValueError: if `scale` does not broadcast with `state_parts`.\n \"\"\"\n with tf.compat.v1.name_scope(\n name, 'random_walk_normal_fn', values=[state_parts, scale, seed]):\n scales = scale if mcmc_util.is_list_like(scale) else [scale]\n if len(scales) == 1:\n scales *= len(state_parts)\n if len(state_parts) != len(scales):\n raise ValueError('`scale` must broadcast with `state_parts`.')\n seed_stream = distributions.SeedStream(seed, salt='RandomWalkNormalFn')\n next_state_parts = [\n tf.random.normal(\n mean=state_part,\n stddev=scale_part,\n shape=tf.shape(input=state_part),\n dtype=state_part.dtype.base_dtype,\n seed=seed_stream())\n for scale_part, state_part in zip(scales, state_parts)\n ]\n\n return next_state_parts\n return _fn", "response": "Returns a function that returns a list of the same - type list of the state parts of the Markov chain and random perturbation to the input state."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef random_walk_uniform_fn(scale=1., name=None):\n def _fn(state_parts, seed):\n \"\"\"Adds a uniform perturbation to the input state.\n\n Args:\n state_parts: A list of `Tensor`s of any shape and real dtype representing\n the state parts of the `current_state` of the Markov chain.\n seed: `int` or None. The random seed for this `Op`. If `None`, no seed is\n applied.\n Default value: `None`.\n\n Returns:\n perturbed_state_parts: A Python `list` of The `Tensor`s. Has the same\n shape and type as the `state_parts`.\n\n Raises:\n ValueError: if `scale` does not broadcast with `state_parts`.\n \"\"\"\n with tf.compat.v1.name_scope(\n name, 'random_walk_uniform_fn', values=[state_parts, scale, seed]):\n scales = scale if mcmc_util.is_list_like(scale) else [scale]\n if len(scales) == 1:\n scales *= len(state_parts)\n if len(state_parts) != len(scales):\n raise ValueError('`scale` must broadcast with `state_parts`.')\n seed_stream = distributions.SeedStream(seed, salt='RandomWalkUniformFn')\n next_state_parts = [\n tf.random.uniform(\n minval=state_part - scale_part,\n maxval=state_part + scale_part,\n shape=tf.shape(input=state_part),\n dtype=state_part.dtype.base_dtype,\n seed=seed_stream())\n for scale_part, state_part in zip(scales, state_parts)\n ]\n return next_state_parts\n return _fn", "response": "Returns a function that adds a random uniform perturbation to the input."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nbatch KL divergence `KL(a || b)` for Independent distributions. We can leverage the fact that ``` KL(Independent(a) || Independent(b)) = sum(KL(a || b)) ``` where the sum is over the `reinterpreted_batch_ndims`. Args: a: Instance of `Independent`. b: Instance of `Independent`. name: (optional) name to use for created ops. Default \"kl_independent\". Returns: Batchwise `KL(a || b)`. Raises: ValueError: If the event space for `a` and `b`, or their underlying distributions don't match.", "response": "def _kl_independent(a, b, name=\"kl_independent\"):\n \"\"\"Batched KL divergence `KL(a || b)` for Independent distributions.\n\n We can leverage the fact that\n ```\n KL(Independent(a) || Independent(b)) = sum(KL(a || b))\n ```\n where the sum is over the `reinterpreted_batch_ndims`.\n\n Args:\n a: Instance of `Independent`.\n b: Instance of `Independent`.\n name: (optional) name to use for created ops. Default \"kl_independent\".\n\n Returns:\n Batchwise `KL(a || b)`.\n\n Raises:\n ValueError: If the event space for `a` and `b`, or their underlying\n distributions don't match.\n \"\"\"\n p = a.distribution\n q = b.distribution\n\n # The KL between any two (non)-batched distributions is a scalar.\n # Given that the KL between two factored distributions is the sum, i.e.\n # KL(p1(x)p2(y) || q1(x)q2(y)) = KL(p1 || q1) + KL(q1 || q2), we compute\n # KL(p || q) and do a `reduce_sum` on the reinterpreted batch dimensions.\n if (tensorshape_util.is_fully_defined(a.event_shape) and\n tensorshape_util.is_fully_defined(b.event_shape)):\n if a.event_shape == b.event_shape:\n if p.event_shape == q.event_shape:\n num_reduce_dims = (tensorshape_util.rank(a.event_shape) -\n tensorshape_util.rank(p.event_shape))\n reduce_dims = [-i - 1 for i in range(0, num_reduce_dims)]\n\n return tf.reduce_sum(\n input_tensor=kullback_leibler.kl_divergence(p, q, name=name),\n axis=reduce_dims)\n else:\n raise NotImplementedError(\"KL between Independents with different \"\n \"event shapes not supported.\")\n else:\n raise ValueError(\"Event shapes do not match.\")\n else:\n with tf.control_dependencies(\n [\n assert_util.assert_equal(a.event_shape_tensor(),\n b.event_shape_tensor()),\n assert_util.assert_equal(p.event_shape_tensor(),\n q.event_shape_tensor())\n ]):\n num_reduce_dims = (\n prefer_static.rank_from_shape(\n a.event_shape_tensor, a.event_shape) -\n prefer_static.rank_from_shape(\n p.event_shape_tensor, a.event_shape))\n reduce_dims = prefer_static.range(-num_reduce_dims - 1, -1, 1)\n return tf.reduce_sum(\n input_tensor=kullback_leibler.kl_divergence(p, q, name=name),\n axis=reduce_dims)"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\ncomputes the default value for reinterpreted_batch_ndim __init__ arg.", "response": "def _get_default_reinterpreted_batch_ndims(self, distribution):\n \"\"\"Computes the default value for reinterpreted_batch_ndim __init__ arg.\"\"\"\n ndims = prefer_static.rank_from_shape(\n distribution.batch_shape_tensor, distribution.batch_shape)\n return prefer_static.maximum(0, ndims - 1)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _expand_to_event_rank(self, x):\n expanded_x = x\n for _ in range(tensorshape_util.rank(self.event_shape)):\n expanded_x = tf.expand_dims(expanded_x, -1)\n return expanded_x", "response": "Expand the rank of x up to static_event_rank times for broadcasting."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef entropy_lower_bound(self, name=\"entropy_lower_bound\"):\n with self._name_scope(name):\n with tf.control_dependencies(self._assertions):\n distribution_entropies = [d.entropy() for d in self.components]\n cat_probs = self._cat_probs(log_probs=False)\n partial_entropies = [\n c_p * m for (c_p, m) in zip(cat_probs, distribution_entropies)\n ]\n # These are all the same shape by virtue of matching batch_shape\n return tf.add_n(partial_entropies)", "response": "r A lower bound on the mixture s entropy."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef _cat_probs(self, log_probs):\n which_softmax = tf.nn.log_softmax if log_probs else tf.nn.softmax\n cat_probs = which_softmax(self.cat.logits)\n cat_probs = tf.unstack(cat_probs, num=self.num_components, axis=-1)\n return cat_probs", "response": "Get a list of num_components batchwise probabilities."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _maybe_validate_args(outcomes, logits, probs, validate_args):\n assertions = []\n\n def validate_equal_last_dim(tensor_a, tensor_b, message):\n if tensor_a.shape.is_fully_defined() and tensor_b.shape.is_fully_defined():\n if tensor_a.shape[-1] != tensor_b.shape[-1]:\n raise ValueError(message)\n elif validate_args:\n assertions.append(\n tf.compat.v1.assert_equal(\n tf.shape(input=tensor_a)[-1],\n tf.shape(input=tensor_b)[-1],\n message=message))\n\n if logits is not None:\n validate_equal_last_dim(\n outcomes,\n logits,\n message='Last dimension of outcomes and logits must be equal size.')\n if probs is not None:\n validate_equal_last_dim(\n outcomes,\n probs,\n message='Last dimension of outcomes and probs must be equal size.')\n\n message = 'Rank of outcomes must be 1.'\n if outcomes.shape.ndims is not None:\n if outcomes.shape.ndims != 1:\n raise ValueError(message)\n elif validate_args:\n assertions.append(tf.compat.v1.assert_rank(outcomes, 1, message=message))\n\n message = 'Size of outcomes must be greater than 0.'\n if outcomes.shape.num_elements() is not None:\n if outcomes.shape.num_elements() == 0:\n raise ValueError(message)\n elif validate_args:\n assertions.append(\n tf.compat.v1.assert_greater(\n tf.size(input=outcomes), 0, message=message))\n\n if validate_args:\n assertions.append(\n tf.compat.v1.assert_equal(\n tf.math.is_strictly_increasing(outcomes),\n True,\n message='outcomes is not strictly increasing.'))\n\n return assertions", "response": "Validate outcomes logits and probs."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nattempts to import tensorflow and ensure its version is sufficient.", "response": "def _ensure_tf_install(): # pylint: disable=g-statement-before-imports\n \"\"\"Attempt to import tensorflow, and ensure its version is sufficient.\n\n Raises:\n ImportError: if either tensorflow is not importable or its version is\n inadequate.\n \"\"\"\n try:\n import tensorflow as tf\n except ImportError:\n # Print more informative error message, then reraise.\n print(\"\\n\\nFailed to import TensorFlow. Please note that TensorFlow is not \"\n \"installed by default when you install TensorFlow Probability. This \"\n \"is so that users can decide whether to install the GPU-enabled \"\n \"TensorFlow package. To use TensorFlow Probability, please install \"\n \"the most recent version of TensorFlow, by following instructions at \"\n \"https://tensorflow.org/install.\\n\\n\")\n raise\n\n import distutils.version\n\n #\n # Update this whenever we need to depend on a newer TensorFlow release.\n #\n required_tensorflow_version = \"1.13\"\n\n if (distutils.version.LooseVersion(tf.__version__) <\n distutils.version.LooseVersion(required_tensorflow_version)):\n raise ImportError(\n \"This version of TensorFlow Probability requires TensorFlow \"\n \"version >= {required}; Detected an installation of version {present}. \"\n \"Please upgrade TensorFlow to proceed.\".format(\n required=required_tensorflow_version,\n present=tf.__version__))"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef covertype():\n import sklearn.datasets # pylint: disable=g-import-not-at-top\n data = sklearn.datasets.covtype.fetch_covtype()\n features = data.data\n labels = data.target\n\n # Normalize features and append a column of ones for the intercept.\n features -= features.mean(0)\n features /= features.std(0)\n features = np.hstack([features, np.ones([features.shape[0], 1])])\n features = tf.cast(features, dtype=tf.float32)\n\n # Binarize outcomes on whether it is a specific category.\n _, counts = np.unique(labels, return_counts=True)\n specific_category = np.argmax(counts)\n labels = (labels == specific_category)\n labels = tf.cast(labels, dtype=tf.int32)\n return features, labels", "response": "Builds the Covertype data set."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _kl_dirichlet_dirichlet(d1, d2, name=None):\n with tf.name_scope(name or \"kl_dirichlet_dirichlet\"):\n # The KL between Dirichlet distributions can be derived as follows. We have\n #\n # Dir(x; a) = 1 / B(a) * prod_i[x[i]^(a[i] - 1)]\n #\n # where B(a) is the multivariate Beta function:\n #\n # B(a) = Gamma(a[1]) * ... * Gamma(a[n]) / Gamma(a[1] + ... + a[n])\n #\n # The KL is\n #\n # KL(Dir(x; a), Dir(x; b)) = E_Dir(x; a){log(Dir(x; a) / Dir(x; b))}\n #\n # so we'll need to know the log density of the Dirichlet. This is\n #\n # log(Dir(x; a)) = sum_i[(a[i] - 1) log(x[i])] - log B(a)\n #\n # The only term that matters for the expectations is the log(x[i]). To\n # compute the expectation of this term over the Dirichlet density, we can\n # use the following facts about the Dirichlet in exponential family form:\n # 1. log(x[i]) is a sufficient statistic\n # 2. expected sufficient statistics (of any exp family distribution) are\n # equal to derivatives of the log normalizer with respect to\n # corresponding natural parameters: E{T[i](x)} = dA/d(eta[i])\n #\n # To proceed, we can rewrite the Dirichlet density in exponential family\n # form as follows:\n #\n # Dir(x; a) = exp{eta(a) . T(x) - A(a)}\n #\n # where '.' is the dot product of vectors eta and T, and A is a scalar:\n #\n # eta[i](a) = a[i] - 1\n # T[i](x) = log(x[i])\n # A(a) = log B(a)\n #\n # Now, we can use fact (2) above to write\n #\n # E_Dir(x; a)[log(x[i])]\n # = dA(a) / da[i]\n # = d/da[i] log B(a)\n # = d/da[i] (sum_j lgamma(a[j])) - lgamma(sum_j a[j])\n # = digamma(a[i])) - digamma(sum_j a[j])\n #\n # Putting it all together, we have\n #\n # KL[Dir(x; a) || Dir(x; b)]\n # = E_Dir(x; a){log(Dir(x; a) / Dir(x; b)}\n # = E_Dir(x; a){sum_i[(a[i] - b[i]) log(x[i])} - (lbeta(a) - lbeta(b))\n # = sum_i[(a[i] - b[i]) * E_Dir(x; a){log(x[i])}] - lbeta(a) + lbeta(b)\n # = sum_i[(a[i] - b[i]) * (digamma(a[i]) - digamma(sum_j a[j]))]\n # - lbeta(a) + lbeta(b))\n\n digamma_sum_d1 = tf.math.digamma(\n tf.reduce_sum(input_tensor=d1.concentration, axis=-1, keepdims=True))\n digamma_diff = tf.math.digamma(d1.concentration) - digamma_sum_d1\n concentration_diff = d1.concentration - d2.concentration\n\n return (\n tf.reduce_sum(input_tensor=concentration_diff * digamma_diff, axis=-1) -\n tf.math.lbeta(d1.concentration) + tf.math.lbeta(d2.concentration))", "response": "Batchwise KL divergence KL between two Dirichlet distributions."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nchecking the validity of the concentration parameter.", "response": "def _maybe_assert_valid_concentration(self, concentration, validate_args):\n \"\"\"Checks the validity of the concentration parameter.\"\"\"\n if not validate_args:\n return concentration\n return distribution_util.with_dependencies([\n assert_util.assert_positive(\n concentration, message=\"Concentration parameter must be positive.\"),\n assert_util.assert_rank_at_least(\n concentration,\n 1,\n message=\"Concentration parameter must have >=1 dimensions.\"),\n assert_util.assert_less(\n 1,\n tf.shape(input=concentration)[-1],\n message=\"Concentration parameter must have event_size >= 2.\"),\n ], concentration)"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\ncheck the validity of a sample.", "response": "def _maybe_assert_valid_sample(self, x):\n \"\"\"Checks the validity of a sample.\"\"\"\n if not self.validate_args:\n return x\n return distribution_util.with_dependencies([\n assert_util.assert_positive(x, message=\"samples must be positive\"),\n assert_util.assert_near(\n tf.ones([], dtype=self.dtype),\n tf.reduce_sum(input_tensor=x, axis=-1),\n message=\"sample last-dimension must sum to `1`\"),\n ], x)"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef cholesky_covariance(x, sample_axis=0, keepdims=False, name=None):\n with tf.compat.v1.name_scope(\n name, 'cholesky_covariance', values=[x, sample_axis]):\n sample_axis = tf.convert_to_tensor(value=sample_axis, dtype=tf.int32)\n cov = covariance(\n x, sample_axis=sample_axis, event_axis=-1, keepdims=keepdims)\n return tf.linalg.cholesky(cov)", "response": "Plots a Cholesky factor of the covariance matrix of vector - variate random samples."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef covariance(x,\n y=None,\n sample_axis=0,\n event_axis=-1,\n keepdims=False,\n name=None):\n \"\"\"Sample covariance between observations indexed by `event_axis`.\n\n Given `N` samples of scalar random variables `X` and `Y`, covariance may be\n estimated as\n\n ```none\n Cov[X, Y] := N^{-1} sum_{n=1}^N (X_n - Xbar) Conj{(Y_n - Ybar)}\n Xbar := N^{-1} sum_{n=1}^N X_n\n Ybar := N^{-1} sum_{n=1}^N Y_n\n ```\n\n For vector-variate random variables `X = (X1, ..., Xd)`, `Y = (Y1, ..., Yd)`,\n one is often interested in the covariance matrix, `C_{ij} := Cov[Xi, Yj]`.\n\n ```python\n x = tf.random_normal(shape=(100, 2, 3))\n y = tf.random_normal(shape=(100, 2, 3))\n\n # cov[i, j] is the sample covariance between x[:, i, j] and y[:, i, j].\n cov = tfp.stats.covariance(x, y, sample_axis=0, event_axis=None)\n\n # cov_matrix[i, m, n] is the sample covariance of x[:, i, m] and y[:, i, n]\n cov_matrix = tfp.stats.covariance(x, y, sample_axis=0, event_axis=-1)\n ```\n\n Notice we divide by `N` (the numpy default), which does not create `NaN`\n when `N = 1`, but is slightly biased.\n\n Args:\n x: A numeric `Tensor` holding samples.\n y: Optional `Tensor` with same `dtype` and `shape` as `x`.\n Default value: `None` (`y` is effectively set to `x`).\n sample_axis: Scalar or vector `Tensor` designating axis holding samples, or\n `None` (meaning all axis hold samples).\n Default value: `0` (leftmost dimension).\n event_axis: Scalar or vector `Tensor`, or `None` (scalar events).\n Axis indexing random events, whose covariance we are interested in.\n If a vector, entries must form a contiguous block of dims. `sample_axis`\n and `event_axis` should not intersect.\n Default value: `-1` (rightmost axis holds events).\n keepdims: Boolean. Whether to keep the sample axis as singletons.\n name: Python `str` name prefixed to Ops created by this function.\n Default value: `None` (i.e., `'covariance'`).\n\n Returns:\n cov: A `Tensor` of same `dtype` as the `x`, and rank equal to\n `rank(x) - len(sample_axis) + 2 * len(event_axis)`.\n\n Raises:\n AssertionError: If `x` and `y` are found to have different shape.\n ValueError: If `sample_axis` and `event_axis` are found to overlap.\n ValueError: If `event_axis` is found to not be contiguous.\n \"\"\"\n\n with tf.compat.v1.name_scope(\n name, 'covariance', values=[x, y, event_axis, sample_axis]):\n x = tf.convert_to_tensor(value=x, name='x')\n # Covariance *only* uses the centered versions of x (and y).\n x -= tf.reduce_mean(input_tensor=x, axis=sample_axis, keepdims=True)\n\n if y is None:\n y = x\n else:\n y = tf.convert_to_tensor(value=y, name='y', dtype=x.dtype)\n # If x and y have different shape, sample_axis and event_axis will likely\n # be wrong for one of them!\n x.shape.assert_is_compatible_with(y.shape)\n y -= tf.reduce_mean(input_tensor=y, axis=sample_axis, keepdims=True)\n\n if event_axis is None:\n return tf.reduce_mean(\n input_tensor=x * tf.math.conj(y), axis=sample_axis, keepdims=keepdims)\n\n if sample_axis is None:\n raise ValueError(\n 'sample_axis was None, which means all axis hold events, and this '\n 'overlaps with event_axis ({})'.format(event_axis))\n\n event_axis = _make_positive_axis(event_axis, tf.rank(x))\n sample_axis = _make_positive_axis(sample_axis, tf.rank(x))\n\n # If we get lucky and axis is statically defined, we can do some checks.\n if _is_list_like(event_axis) and _is_list_like(sample_axis):\n if set(event_axis).intersection(sample_axis):\n raise ValueError(\n 'sample_axis ({}) and event_axis ({}) overlapped'.format(\n sample_axis, event_axis))\n if (np.diff(sorted(event_axis)) > 1).any():\n raise ValueError(\n 'event_axis must be contiguous. Found: {}'.format(event_axis))\n batch_axis = list(\n sorted(\n set(range(x.shape.ndims)).difference(sample_axis + event_axis)))\n else:\n batch_axis, _ = tf.compat.v1.setdiff1d(\n tf.range(0, tf.rank(x)), tf.concat((sample_axis, event_axis), 0))\n\n event_axis = tf.convert_to_tensor(\n value=event_axis, name='event_axis', dtype=tf.int32)\n sample_axis = tf.convert_to_tensor(\n value=sample_axis, name='sample_axis', dtype=tf.int32)\n batch_axis = tf.convert_to_tensor(\n value=batch_axis, name='batch_axis', dtype=tf.int32)\n\n # Permute x/y until shape = B + E + S\n perm_for_xy = tf.concat((batch_axis, event_axis, sample_axis), 0)\n x_permed = tf.transpose(a=x, perm=perm_for_xy)\n y_permed = tf.transpose(a=y, perm=perm_for_xy)\n\n batch_ndims = tf.size(input=batch_axis)\n batch_shape = tf.shape(input=x_permed)[:batch_ndims]\n event_ndims = tf.size(input=event_axis)\n event_shape = tf.shape(input=x_permed)[batch_ndims:batch_ndims +\n event_ndims]\n sample_shape = tf.shape(input=x_permed)[batch_ndims + event_ndims:]\n sample_ndims = tf.size(input=sample_shape)\n n_samples = tf.reduce_prod(input_tensor=sample_shape)\n n_events = tf.reduce_prod(input_tensor=event_shape)\n\n # Flatten sample_axis into one long dim.\n x_permed_flat = tf.reshape(\n x_permed, tf.concat((batch_shape, event_shape, [n_samples]), 0))\n y_permed_flat = tf.reshape(\n y_permed, tf.concat((batch_shape, event_shape, [n_samples]), 0))\n # Do the same for event_axis.\n x_permed_flat = tf.reshape(\n x_permed, tf.concat((batch_shape, [n_events], [n_samples]), 0))\n y_permed_flat = tf.reshape(\n y_permed, tf.concat((batch_shape, [n_events], [n_samples]), 0))\n\n # After matmul, cov.shape = batch_shape + [n_events, n_events]\n cov = tf.matmul(\n x_permed_flat, y_permed_flat, adjoint_b=True) / tf.cast(\n n_samples, x.dtype)\n\n # Insert some singletons to make\n # cov.shape = batch_shape + event_shape**2 + [1,...,1]\n # This is just like x_permed.shape, except the sample_axis is all 1's, and\n # the [n_events] became event_shape**2.\n cov = tf.reshape(\n cov,\n tf.concat(\n (\n batch_shape,\n # event_shape**2 used here because it is the same length as\n # event_shape, and has the same number of elements as one\n # batch of covariance.\n event_shape**2,\n tf.ones([sample_ndims], tf.int32)),\n 0))\n # Permuting by the argsort inverts the permutation, making\n # cov.shape have ones in the position where there were samples, and\n # [n_events * n_events] in the event position.\n cov = tf.transpose(a=cov, perm=tf.math.invert_permutation(perm_for_xy))\n\n # Now expand event_shape**2 into event_shape + event_shape.\n # We here use (for the first time) the fact that we require event_axis to be\n # contiguous.\n e_start = event_axis[0]\n e_len = 1 + event_axis[-1] - event_axis[0]\n cov = tf.reshape(\n cov,\n tf.concat((tf.shape(input=cov)[:e_start], event_shape, event_shape,\n tf.shape(input=cov)[e_start + e_len:]), 0))\n\n # tf.squeeze requires python ints for axis, not Tensor. This is enough to\n # require our axis args to be constants.\n if not keepdims:\n squeeze_axis = tf.where(sample_axis < e_start, sample_axis,\n sample_axis + e_len)\n cov = _squeeze(cov, axis=squeeze_axis)\n\n return cov", "response": "Sample covariance between observations indexed by sample_axis and event_axis."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef correlation(x,\n y=None,\n sample_axis=0,\n event_axis=-1,\n keepdims=False,\n name=None):\n \"\"\"Sample correlation (Pearson) between observations indexed by `event_axis`.\n\n Given `N` samples of scalar random variables `X` and `Y`, correlation may be\n estimated as\n\n ```none\n Corr[X, Y] := Cov[X, Y] / Sqrt(Cov[X, X] * Cov[Y, Y]),\n where\n Cov[X, Y] := N^{-1} sum_{n=1}^N (X_n - Xbar) Conj{(Y_n - Ybar)}\n Xbar := N^{-1} sum_{n=1}^N X_n\n Ybar := N^{-1} sum_{n=1}^N Y_n\n ```\n\n Correlation is always in the interval `[-1, 1]`, and `Corr[X, X] == 1`.\n\n For vector-variate random variables `X = (X1, ..., Xd)`, `Y = (Y1, ..., Yd)`,\n one is often interested in the correlation matrix, `C_{ij} := Corr[Xi, Yj]`.\n\n ```python\n x = tf.random_normal(shape=(100, 2, 3))\n y = tf.random_normal(shape=(100, 2, 3))\n\n # corr[i, j] is the sample correlation between x[:, i, j] and y[:, i, j].\n corr = tfp.stats.correlation(x, y, sample_axis=0, event_axis=None)\n\n # corr_matrix[i, m, n] is the sample correlation of x[:, i, m] and y[:, i, n]\n corr_matrix = tfp.stats.correlation(x, y, sample_axis=0, event_axis=-1)\n ```\n\n Notice we divide by `N` (the numpy default), which does not create `NaN`\n when `N = 1`, but is slightly biased.\n\n Args:\n x: A numeric `Tensor` holding samples.\n y: Optional `Tensor` with same `dtype` and `shape` as `x`.\n Default value: `None` (`y` is effectively set to `x`).\n sample_axis: Scalar or vector `Tensor` designating axis holding samples, or\n `None` (meaning all axis hold samples).\n Default value: `0` (leftmost dimension).\n event_axis: Scalar or vector `Tensor`, or `None` (scalar events).\n Axis indexing random events, whose correlation we are interested in.\n If a vector, entries must form a contiguous block of dims. `sample_axis`\n and `event_axis` should not intersect.\n Default value: `-1` (rightmost axis holds events).\n keepdims: Boolean. Whether to keep the sample axis as singletons.\n name: Python `str` name prefixed to Ops created by this function.\n Default value: `None` (i.e., `'correlation'`).\n\n Returns:\n corr: A `Tensor` of same `dtype` as the `x`, and rank equal to\n `rank(x) - len(sample_axis) + 2 * len(event_axis)`.\n\n Raises:\n AssertionError: If `x` and `y` are found to have different shape.\n ValueError: If `sample_axis` and `event_axis` are found to overlap.\n ValueError: If `event_axis` is found to not be contiguous.\n \"\"\"\n\n with tf.compat.v1.name_scope(\n name, 'correlation', values=[x, y, event_axis, sample_axis]):\n # Corr[X, Y] = Cov[X, Y] / (Stddev[X] * Stddev[Y])\n # = Cov[X / Stddev[X], Y / Stddev[Y]]\n # So we could compute covariance first then divide by stddev, or\n # divide by stddev and compute covariance.\n # Dividing by stddev then computing covariance is potentially more stable.\n # But... computing covariance first then dividing involves 2 fewer large\n # broadcasts. We choose to divide first, largely because it avoids\n # difficulties with the various options for sample/event axis kwargs.\n\n x /= stddev(x, sample_axis=sample_axis, keepdims=True)\n if y is not None:\n y /= stddev(y, sample_axis=sample_axis, keepdims=True)\n\n return covariance(\n x=x,\n y=y,\n event_axis=event_axis,\n sample_axis=sample_axis,\n keepdims=keepdims)", "response": "Sample correlation between observations X and Y."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef stddev(x, sample_axis=0, keepdims=False, name=None):\n with tf.compat.v1.name_scope(name, 'stddev', values=[x, sample_axis]):\n return tf.sqrt(variance(x, sample_axis=sample_axis, keepdims=keepdims))", "response": "Estimate standard deviation of samples of scalar valued random variable X."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef variance(x, sample_axis=0, keepdims=False, name=None):\n with tf.compat.v1.name_scope(name, 'variance', values=[x, sample_axis]):\n return covariance(\n x, y=None, sample_axis=sample_axis, event_axis=None, keepdims=keepdims)", "response": "Estimate variance using samples."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _make_list_or_1d_tensor(values):\n values = tf.convert_to_tensor(value=values, name='values')\n values_ = tf.get_static_value(values)\n\n # Static didn't work.\n if values_ is None:\n # Cheap way to bring to at least 1d.\n return values + tf.zeros([1], dtype=values.dtype)\n\n # Static worked!\n if values_.ndim > 1:\n raise ValueError('values had > 1 dim: {}'.format(values_.shape))\n # Cheap way to bring to at least 1d.\n values_ = values_ + np.zeros([1], dtype=values_.dtype)\n return list(values_)", "response": "Return a list or 1d Tensor from values."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef _make_positive_axis(axis, ndims):\n axis = _make_list_or_1d_tensor(axis)\n\n ndims = tf.convert_to_tensor(value=ndims, name='ndims', dtype=tf.int32)\n ndims_ = tf.get_static_value(ndims)\n\n if _is_list_like(axis) and ndims_ is not None:\n # Static case\n positive_axis = []\n for a in axis:\n if a < 0:\n a = ndims_ + a\n positive_axis.append(a)\n else:\n # Dynamic case\n axis = tf.convert_to_tensor(value=axis, name='axis', dtype=tf.int32)\n positive_axis = tf.where(axis >= 0, axis, axis + ndims)\n\n return positive_axis", "response": "Rectify possibly negatively axis. Prefer return Python list."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef _squeeze(x, axis):\n x = tf.convert_to_tensor(value=x, name='x')\n if axis is None:\n return tf.squeeze(x, axis=None)\n axis = tf.convert_to_tensor(value=axis, name='axis', dtype=tf.int32)\n axis += tf.zeros([1], dtype=axis.dtype) # Make axis at least 1d.\n keep_axis, _ = tf.compat.v1.setdiff1d(tf.range(0, tf.rank(x)), axis)\n return tf.reshape(x, tf.gather(tf.shape(input=x), keep_axis))", "response": "A version of squeeze that works with dynamic axis."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _kl_normal_normal(n_a, n_b, name=None):\n with tf.name_scope(name or \"kl_normal_normal\"):\n one = tf.constant(1, dtype=n_a.dtype)\n two = tf.constant(2, dtype=n_a.dtype)\n half = tf.constant(0.5, dtype=n_a.dtype)\n s_a_squared = tf.square(n_a.scale)\n s_b_squared = tf.square(n_b.scale)\n ratio = s_a_squared / s_b_squared\n return (tf.square(n_a.loc - n_b.loc) / (two * s_b_squared) + half *\n (ratio - one - tf.math.log(ratio)))", "response": "Calculate the batched KL divergence of n_a and n_b Normal."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nstandardize input x to a unit normal.", "response": "def _z(self, x):\n \"\"\"Standardize input `x` to a unit normal.\"\"\"\n with tf.name_scope(\"standardize\"):\n return (x - self.loc) / self.scale"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _inv_z(self, z):\n with tf.name_scope(\"reconstruct\"):\n return z * self.scale + self.loc", "response": "Reconstruct input x from a its normalized version."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nbuilding the transition matrix for a semi - local linear trend model.", "response": "def semilocal_linear_trend_transition_matrix(autoregressive_coef):\n \"\"\"Build the transition matrix for a semi-local linear trend model.\"\"\"\n # We want to write the following 2 x 2 matrix:\n # [[1., 1., ], # level(t+1) = level(t) + slope(t)\n # [0., ar_coef], # slope(t+1) = ar_coef * slope(t)\n # but it's slightly tricky to properly incorporate the batch shape of\n # autoregressive_coef. E.g., if autoregressive_coef has shape [4,6], we want\n # to return shape [4, 6, 2, 2]. We do this by breaking the matrix into its\n # fixed entries, written explicitly, and then the autoregressive_coef part\n # which we add in after using a mask to broadcast to the correct matrix shape.\n\n fixed_entries = tf.constant(\n [[1., 1.],\n [0., 0.]],\n dtype=autoregressive_coef.dtype)\n\n autoregressive_coef_mask = tf.constant([[0., 0.],\n [0., 1.]],\n dtype=autoregressive_coef.dtype)\n bottom_right_entry = (autoregressive_coef[..., tf.newaxis, tf.newaxis] *\n autoregressive_coef_mask)\n return tf.linalg.LinearOperatorFullMatrix(\n fixed_entries + bottom_right_entry)"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nbuilds a transition noise model for a semi - local linear trend model.", "response": "def semilocal_linear_trend_transition_noise(level_scale,\n slope_mean,\n slope_scale,\n autoregressive_coef):\n \"\"\"Build the transition noise model for a semi-local linear trend model.\"\"\"\n\n # At each timestep, the stochasticity of `level` and `slope` are given\n # by `level_scale` and `slope_scale` respectively.\n broadcast_batch_shape = dist_util.get_broadcast_shape(\n level_scale, slope_mean, slope_scale, autoregressive_coef)\n broadcast_ones = tf.ones(broadcast_batch_shape, dtype=level_scale.dtype)\n scale_diag = tf.stack([level_scale * broadcast_ones,\n slope_scale * broadcast_ones],\n axis=-1)\n\n # We additionally fold in a bias term implementing the nonzero `slope_mean`.\n # The overall `slope` update is (from `SemiLocalLinearTrend` docstring)\n # slope[t] = (slope_mean +\n # autoregressive_coef * (slope[t-1] - slope_mean) +\n # Normal(0., slope_scale))\n # which we rewrite as\n # slope[t] = (\n # autoregressive_coef * slope[t-1] + # linear transition\n # Normal(loc=slope_mean - autoregressive_coef * slope_mean, # noise bias\n # scale=slope_scale)) # noise scale\n bias = tf.stack([tf.zeros_like(broadcast_ones),\n slope_mean * (1 - autoregressive_coef) * broadcast_ones],\n axis=-1)\n return tfd.MultivariateNormalDiag(\n loc=bias,\n scale_diag=scale_diag)"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef sample_halton_sequence(dim,\n num_results=None,\n sequence_indices=None,\n dtype=tf.float32,\n randomized=True,\n seed=None,\n name=None):\n r\"\"\"Returns a sample from the `dim` dimensional Halton sequence.\n\n Warning: The sequence elements take values only between 0 and 1. Care must be\n taken to appropriately transform the domain of a function if it differs from\n the unit cube before evaluating integrals using Halton samples. It is also\n important to remember that quasi-random numbers without randomization are not\n a replacement for pseudo-random numbers in every context. Quasi random numbers\n are completely deterministic and typically have significant negative\n autocorrelation unless randomization is used.\n\n Computes the members of the low discrepancy Halton sequence in dimension\n `dim`. The `dim`-dimensional sequence takes values in the unit hypercube in\n `dim` dimensions. Currently, only dimensions up to 1000 are supported. The\n prime base for the k-th axes is the k-th prime starting from 2. For example,\n if `dim` = 3, then the bases will be [2, 3, 5] respectively and the first\n element of the non-randomized sequence will be: [0.5, 0.333, 0.2]. For a more\n complete description of the Halton sequences see\n [here](https://en.wikipedia.org/wiki/Halton_sequence). For low discrepancy\n sequences and their applications see\n [here](https://en.wikipedia.org/wiki/Low-discrepancy_sequence).\n\n If `randomized` is true, this function produces a scrambled version of the\n Halton sequence introduced by [Owen (2017)][1]. For the advantages of\n randomization of low discrepancy sequences see [here](\n https://en.wikipedia.org/wiki/Quasi-Monte_Carlo_method#Randomization_of_quasi-Monte_Carlo).\n\n The number of samples produced is controlled by the `num_results` and\n `sequence_indices` parameters. The user must supply either `num_results` or\n `sequence_indices` but not both.\n The former is the number of samples to produce starting from the first\n element. If `sequence_indices` is given instead, the specified elements of\n the sequence are generated. For example, sequence_indices=tf.range(10) is\n equivalent to specifying n=10.\n\n #### Examples\n\n ```python\n import tensorflow as tf\n import tensorflow_probability as tfp\n\n # Produce the first 1000 members of the Halton sequence in 3 dimensions.\n num_results = 1000\n dim = 3\n sample = tfp.mcmc.sample_halton_sequence(\n dim,\n num_results=num_results,\n seed=127)\n\n # Evaluate the integral of x_1 * x_2^2 * x_3^3 over the three dimensional\n # hypercube.\n powers = tf.range(1.0, limit=dim + 1)\n integral = tf.reduce_mean(tf.reduce_prod(sample ** powers, axis=-1))\n true_value = 1.0 / tf.reduce_prod(powers + 1.0)\n with tf.Session() as session:\n values = session.run((integral, true_value))\n\n # Produces a relative absolute error of 1.7%.\n print (\"Estimated: %f, True Value: %f\" % values)\n\n # Now skip the first 1000 samples and recompute the integral with the next\n # thousand samples. The sequence_indices argument can be used to do this.\n\n\n sequence_indices = tf.range(start=1000, limit=1000 + num_results,\n dtype=tf.int32)\n sample_leaped = tfp.mcmc.sample_halton_sequence(\n dim,\n sequence_indices=sequence_indices,\n seed=111217)\n\n integral_leaped = tf.reduce_mean(tf.reduce_prod(sample_leaped ** powers,\n axis=-1))\n with tf.Session() as session:\n values = session.run((integral_leaped, true_value))\n # Now produces a relative absolute error of 0.05%.\n print (\"Leaped Estimated: %f, True Value: %f\" % values)\n ```\n\n Args:\n dim: Positive Python `int` representing each sample's `event_size.` Must\n not be greater than 1000.\n num_results: (Optional) Positive scalar `Tensor` of dtype int32. The number\n of samples to generate. Either this parameter or sequence_indices must\n be specified but not both. If this parameter is None, then the behaviour\n is determined by the `sequence_indices`.\n Default value: `None`.\n sequence_indices: (Optional) `Tensor` of dtype int32 and rank 1. The\n elements of the sequence to compute specified by their position in the\n sequence. The entries index into the Halton sequence starting with 0 and\n hence, must be whole numbers. For example, sequence_indices=[0, 5, 6] will\n produce the first, sixth and seventh elements of the sequence. If this\n parameter is None, then the `num_results` parameter must be specified\n which gives the number of desired samples starting from the first sample.\n Default value: `None`.\n dtype: (Optional) The dtype of the sample. One of: `float16`, `float32` or\n `float64`.\n Default value: `tf.float32`.\n randomized: (Optional) bool indicating whether to produce a randomized\n Halton sequence. If True, applies the randomization described in\n [Owen (2017)][1].\n Default value: `True`.\n seed: (Optional) Python integer to seed the random number generator. Only\n used if `randomized` is True. If not supplied and `randomized` is True,\n no seed is set.\n Default value: `None`.\n name: (Optional) Python `str` describing ops managed by this function. If\n not supplied the name of this function is used.\n Default value: \"sample_halton_sequence\".\n\n Returns:\n halton_elements: Elements of the Halton sequence. `Tensor` of supplied dtype\n and `shape` `[num_results, dim]` if `num_results` was specified or shape\n `[s, dim]` where s is the size of `sequence_indices` if `sequence_indices`\n were specified.\n\n Raises:\n ValueError: if both `sequence_indices` and `num_results` were specified or\n if dimension `dim` is less than 1 or greater than 1000.\n\n #### References\n\n [1]: Art B. Owen. A randomized Halton algorithm in R. _arXiv preprint\n arXiv:1706.02808_, 2017. https://arxiv.org/abs/1706.02808\n \"\"\"\n if dim < 1 or dim > _MAX_DIMENSION:\n raise ValueError(\n 'Dimension must be between 1 and {}. Supplied {}'.format(_MAX_DIMENSION,\n dim))\n if (num_results is None) == (sequence_indices is None):\n raise ValueError('Either `num_results` or `sequence_indices` must be'\n ' specified but not both.')\n\n if not dtype.is_floating:\n raise ValueError('dtype must be of `float`-type')\n\n with tf.compat.v1.name_scope(\n name, 'sample', values=[num_results, sequence_indices]):\n # Here and in the following, the shape layout is as follows:\n # [sample dimension, event dimension, coefficient dimension].\n # The coefficient dimension is an intermediate axes which will hold the\n # weights of the starting integer when expressed in the (prime) base for\n # an event dimension.\n if num_results is not None:\n num_results = tf.convert_to_tensor(value=num_results)\n if sequence_indices is not None:\n sequence_indices = tf.convert_to_tensor(value=sequence_indices)\n indices = _get_indices(num_results, sequence_indices, dtype)\n radixes = tf.constant(_PRIMES[0:dim], dtype=dtype, shape=[dim, 1])\n\n max_sizes_by_axes = _base_expansion_size(\n tf.reduce_max(input_tensor=indices), radixes)\n\n max_size = tf.reduce_max(input_tensor=max_sizes_by_axes)\n\n # The powers of the radixes that we will need. Note that there is a bit\n # of an excess here. Suppose we need the place value coefficients of 7\n # in base 2 and 3. For 2, we will have 3 digits but we only need 2 digits\n # for base 3. However, we can only create rectangular tensors so we\n # store both expansions in a [2, 3] tensor. This leads to the problem that\n # we might end up attempting to raise large numbers to large powers. For\n # example, base 2 expansion of 1024 has 10 digits. If we were in 10\n # dimensions, then the 10th prime (29) we will end up computing 29^10 even\n # though we don't need it. We avoid this by setting the exponents for each\n # axes to 0 beyond the maximum value needed for that dimension.\n exponents_by_axes = tf.tile([tf.range(max_size)], [dim, 1])\n\n # The mask is true for those coefficients that are irrelevant.\n weight_mask = exponents_by_axes >= max_sizes_by_axes\n capped_exponents = tf.where(\n weight_mask,\n tf.zeros_like(exponents_by_axes),\n exponents_by_axes)\n weights = radixes ** capped_exponents\n # The following computes the base b expansion of the indices. Suppose,\n # x = a0 + a1*b + a2*b^2 + ... Then, performing a floor div of x with\n # the vector (1, b, b^2, b^3, ...) will produce\n # (a0 + s1 * b, a1 + s2 * b, ...) where s_i are coefficients we don't care\n # about. Noting that all a_i < b by definition of place value expansion,\n # we see that taking the elements mod b of the above vector produces the\n # place value expansion coefficients.\n coeffs = tf.math.floordiv(indices, weights)\n coeffs *= 1. - tf.cast(weight_mask, dtype)\n coeffs %= radixes\n if not randomized:\n coeffs /= radixes\n return tf.reduce_sum(input_tensor=coeffs / weights, axis=-1)\n stream = distributions.SeedStream(seed, salt='MCMCSampleHaltonSequence')\n coeffs = _randomize(coeffs, radixes, seed=stream())\n # Remove the contribution from randomizing the trailing zero for the\n # axes where max_size_by_axes < max_size. This will be accounted\n # for separately below (using zero_correction).\n coeffs *= 1. - tf.cast(weight_mask, dtype)\n coeffs /= radixes\n base_values = tf.reduce_sum(input_tensor=coeffs / weights, axis=-1)\n\n # The randomization used in Owen (2017) does not leave 0 invariant. While\n # we have accounted for the randomization of the first `max_size_by_axes`\n # coefficients, we still need to correct for the trailing zeros. Luckily,\n # this is equivalent to adding a uniform random value scaled so the first\n # `max_size_by_axes` coefficients are zero. The following statements perform\n # this correction.\n zero_correction = tf.random.uniform([dim, 1], seed=stream(), dtype=dtype)\n zero_correction /= radixes ** max_sizes_by_axes\n return base_values + tf.reshape(zero_correction, [-1])", "response": "r Returns a sample from the Halton sequence."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _randomize(coeffs, radixes, seed=None):\n given_dtype = coeffs.dtype\n coeffs = tf.cast(coeffs, dtype=tf.int32)\n num_coeffs = tf.shape(input=coeffs)[-1]\n radixes = tf.reshape(tf.cast(radixes, dtype=tf.int32), shape=[-1])\n stream = distributions.SeedStream(seed, salt='MCMCSampleHaltonSequence2')\n perms = _get_permutations(num_coeffs, radixes, seed=stream())\n perms = tf.reshape(perms, shape=[-1])\n radix_sum = tf.reduce_sum(input_tensor=radixes)\n radix_offsets = tf.reshape(tf.cumsum(radixes, exclusive=True),\n shape=[-1, 1])\n offsets = radix_offsets + tf.range(num_coeffs) * radix_sum\n permuted_coeffs = tf.gather(perms, coeffs + offsets)\n return tf.cast(permuted_coeffs, dtype=given_dtype)", "response": "Applies the Owen ( 2017 ) randomization to the coefficients."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nreturns a tensor of shape num_results and the same dtype as the num_results.", "response": "def _get_permutations(num_results, dims, seed=None):\n \"\"\"Uniform iid sample from the space of permutations.\n\n Draws a sample of size `num_results` from the group of permutations of degrees\n specified by the `dims` tensor. These are packed together into one tensor\n such that each row is one sample from each of the dimensions in `dims`. For\n example, if dims = [2,3] and num_results = 2, the result is a tensor of shape\n [2, 2 + 3] and the first row of the result might look like:\n [1, 0, 2, 0, 1]. The first two elements are a permutation over 2 elements\n while the next three are a permutation over 3 elements.\n\n Args:\n num_results: A positive scalar `Tensor` of integral type. The number of\n draws from the discrete uniform distribution over the permutation groups.\n dims: A 1D `Tensor` of the same dtype as `num_results`. The degree of the\n permutation groups from which to sample.\n seed: (Optional) Python integer to seed the random number generator.\n\n Returns:\n permutations: A `Tensor` of shape `[num_results, sum(dims)]` and the same\n dtype as `dims`.\n \"\"\"\n sample_range = tf.range(num_results)\n stream = distributions.SeedStream(seed, salt='MCMCSampleHaltonSequence3')\n def generate_one(d):\n seed = stream()\n fn = lambda _: tf.random.shuffle(tf.range(d), seed=seed)\n return tf.map_fn(\n fn,\n sample_range,\n parallel_iterations=1 if seed is not None else 10)\n return tf.concat([generate_one(d) for d in tf.unstack(dims)],\n axis=-1)"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\ngenerating the indices for the Halton sequence procedure.", "response": "def _get_indices(num_results, sequence_indices, dtype, name=None):\n \"\"\"Generates starting points for the Halton sequence procedure.\n\n The k'th element of the sequence is generated starting from a positive integer\n which must be distinct for each `k`. It is conventional to choose the starting\n point as `k` itself (or `k+1` if k is zero based). This function generates\n the starting integers for the required elements and reshapes the result for\n later use.\n\n Args:\n num_results: Positive scalar `Tensor` of dtype int32. The number of samples\n to generate. If this parameter is supplied, then `sequence_indices`\n should be None.\n sequence_indices: `Tensor` of dtype int32 and rank 1. The entries\n index into the Halton sequence starting with 0 and hence, must be whole\n numbers. For example, sequence_indices=[0, 5, 6] will produce the first,\n sixth and seventh elements of the sequence. If this parameter is not None\n then `n` must be None.\n dtype: The dtype of the sample. One of `float32` or `float64`.\n Default is `float32`.\n name: Python `str` name which describes ops created by this function.\n\n Returns:\n indices: `Tensor` of dtype `dtype` and shape = `[n, 1, 1]`.\n \"\"\"\n with tf.compat.v1.name_scope(name, '_get_indices',\n [num_results, sequence_indices]):\n if sequence_indices is None:\n num_results = tf.cast(num_results, dtype=dtype)\n sequence_indices = tf.range(num_results, dtype=dtype)\n else:\n sequence_indices = tf.cast(sequence_indices, dtype)\n\n # Shift the indices so they are 1 based.\n indices = sequence_indices + 1\n\n # Reshape to make space for the event dimension and the place value\n # coefficients.\n return tf.reshape(indices, [-1, 1, 1])"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _base_expansion_size(num, bases):\n return tf.floor(tf.math.log(num) / tf.math.log(bases)) + 1", "response": "Computes the number of terms in the place value expansion of num in base bases."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _primes_less_than(n):\n # Based on\n # https://stackoverflow.com/questions/2068372/fastest-way-to-list-all-primes-below-n-in-python/3035188#3035188\n \"\"\"Returns sorted array of primes such that `2 <= prime < n`.\"\"\"\n small_primes = np.array((2, 3, 5))\n if n <= 6:\n return small_primes[small_primes < n]\n sieve = np.ones(n // 3 + (n % 6 == 2), dtype=np.bool)\n sieve[0] = False\n m = int(n ** 0.5) // 3 + 1\n for i in range(m):\n if not sieve[i]:\n continue\n k = 3 * i + 1 | 1\n sieve[k ** 2 // 3::2 * k] = False\n sieve[(k ** 2 + 4 * k - 2 * k * (i & 1)) // 3::2 * k] = False\n return np.r_[2, 3, 3 * np.nonzero(sieve)[0] + 1 | 1]", "response": "Returns sorted array of primes such that 2 <= prime < n."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nreturning the machine epsilon for the supplied dtype.", "response": "def _machine_eps(dtype):\n \"\"\"Returns the machine epsilon for the supplied dtype.\"\"\"\n if isinstance(dtype, tf.DType):\n dtype = dtype.as_numpy_dtype()\n return np.finfo(dtype).eps"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nshrink the input step size until the value and grad become finite.", "response": "def _fix_step_size(value_and_gradients_function,\n val_c_input,\n active,\n step_size_shrink_param):\n \"\"\"Shrinks the input step size until the value and grad become finite.\"\"\"\n # The maximum iterations permitted are determined as the number of halvings\n # it takes to reduce 1 to 0 in the given dtype.\n iter_max = np.ceil(-np.log2(_machine_eps(val_c_input.x.dtype)))\n\n def _cond(i, val_c, to_fix):\n del val_c # Unused.\n return (i < iter_max) & tf.reduce_any(input_tensor=to_fix)\n\n def _body(i, val_c, to_fix):\n next_c = tf.where(to_fix, val_c.x * step_size_shrink_param, val_c.x)\n next_val_c = value_and_gradients_function(next_c)\n still_to_fix = to_fix & ~hzl.is_finite(next_val_c)\n return (i + 1, next_val_c, still_to_fix)\n\n to_fix = active & ~hzl.is_finite(val_c_input)\n return tf.while_loop(\n cond=_cond, body=_body, loop_vars=(0, val_c_input, to_fix))"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nbracket the minimum and performs a line search. Args: value_and_gradients_function: A Python callable that accepts a real scalar tensor and returns a namedtuple with the fields 'x', 'f', and 'df' that correspond to scalar tensors of real dtype containing the point at which the function was evaluated, the value of the function, and its derivative at that point. The other namedtuple fields, if present, should be tensors or sequences (possibly nested) of tensors. In usual optimization application, this function would be generated by projecting the multivariate objective function along some specific direction. The direction is determined by some other procedure but should be a descent direction (i.e. the derivative of the projected univariate function must be negative at 0.). Alternatively, the function may represent the batching of `n` such line functions (e.g. projecting a single multivariate objective function along `n` distinct directions at once) accepting n points as input, i.e. a tensor of shape [n], and the fields 'x', 'f' and 'df' in the returned namedtuple should each be a tensor of shape [n], with the corresponding input points, function values, and derivatives at those input points. init_interval: Instance of `HagerZhangLineSearchResults` containing the initial line search interval. The gradient of init_interval.left must be negative (i.e. must be a descent direction), while init_interval.right must be positive and finite. f_lim: Scalar `Tensor` of float dtype. max_iterations: Positive scalar `Tensor` of integral dtype. The maximum number of iterations to perform in the line search. The number of iterations used to bracket the minimum are also counted against this parameter. shrinkage_param: Scalar positive Tensor of real dtype. Must be less than `1.`. Corresponds to the parameter `gamma` in [Hager and Zhang (2006)][2]. expansion_param: Scalar positive `Tensor` of real dtype. Must be greater than `1.`. Used to expand the initial interval in case it does not bracket a minimum. Corresponds to `rho` in [Hager and Zhang (2006)][2]. sufficient_decrease_param: Positive scalar `Tensor` of real dtype. Bounded above by the curvature param. Corresponds to `delta` in the terminology of [Hager and Zhang (2006)][2]. curvature_param: Positive scalar `Tensor` of real dtype. Bounded above by `1.`. Corresponds to 'sigma' in the terminology of [Hager and Zhang (2006)][2]. Returns: A namedtuple containing the following fields. converged: Boolean `Tensor` of shape [n]. Whether a point satisfying Wolfe/Approx wolfe was found. failed: Boolean `Tensor` of shape [n]. Whether line search failed e.g. if either the objective function or the gradient are not finite at an evaluation point. iterations: Scalar int32 `Tensor`. Number of line search iterations made. func_evals: Scalar int32 `Tensor`. Number of function evaluations made. left: A namedtuple, as returned by value_and_gradients_function, of the left end point of the updated bracketing interval. right: A namedtuple, as returned by value_and_gradients_function, of the right end point of the updated bracketing interval.", "response": "def _bracket_and_search(\n value_and_gradients_function,\n init_interval,\n f_lim,\n max_iterations,\n shrinkage_param,\n expansion_param,\n sufficient_decrease_param,\n curvature_param):\n \"\"\"Brackets the minimum and performs a line search.\n\n Args:\n value_and_gradients_function: A Python callable that accepts a real scalar\n tensor and returns a namedtuple with the fields 'x', 'f', and 'df' that\n correspond to scalar tensors of real dtype containing the point at which\n the function was evaluated, the value of the function, and its\n derivative at that point. The other namedtuple fields, if present,\n should be tensors or sequences (possibly nested) of tensors.\n In usual optimization application, this function would be generated by\n projecting the multivariate objective function along some specific\n direction. The direction is determined by some other procedure but should\n be a descent direction (i.e. the derivative of the projected univariate\n function must be negative at 0.).\n Alternatively, the function may represent the batching of `n` such line\n functions (e.g. projecting a single multivariate objective function along\n `n` distinct directions at once) accepting n points as input, i.e. a\n tensor of shape [n], and the fields 'x', 'f' and 'df' in the returned\n namedtuple should each be a tensor of shape [n], with the corresponding\n input points, function values, and derivatives at those input points.\n init_interval: Instance of `HagerZhangLineSearchResults` containing\n the initial line search interval. The gradient of init_interval.left must\n be negative (i.e. must be a descent direction), while init_interval.right\n must be positive and finite.\n f_lim: Scalar `Tensor` of float dtype.\n max_iterations: Positive scalar `Tensor` of integral dtype. The maximum\n number of iterations to perform in the line search. The number of\n iterations used to bracket the minimum are also counted against this\n parameter.\n shrinkage_param: Scalar positive Tensor of real dtype. Must be less than\n `1.`. Corresponds to the parameter `gamma` in [Hager and Zhang (2006)][2].\n expansion_param: Scalar positive `Tensor` of real dtype. Must be greater\n than `1.`. Used to expand the initial interval in case it does not bracket\n a minimum. Corresponds to `rho` in [Hager and Zhang (2006)][2].\n sufficient_decrease_param: Positive scalar `Tensor` of real dtype.\n Bounded above by the curvature param. Corresponds to `delta` in the\n terminology of [Hager and Zhang (2006)][2].\n curvature_param: Positive scalar `Tensor` of real dtype. Bounded above\n by `1.`. Corresponds to 'sigma' in the terminology of\n [Hager and Zhang (2006)][2].\n\n Returns:\n A namedtuple containing the following fields.\n converged: Boolean `Tensor` of shape [n]. Whether a point satisfying\n Wolfe/Approx wolfe was found.\n failed: Boolean `Tensor` of shape [n]. Whether line search failed e.g.\n if either the objective function or the gradient are not finite at\n an evaluation point.\n iterations: Scalar int32 `Tensor`. Number of line search iterations made.\n func_evals: Scalar int32 `Tensor`. Number of function evaluations made.\n left: A namedtuple, as returned by value_and_gradients_function,\n of the left end point of the updated bracketing interval.\n right: A namedtuple, as returned by value_and_gradients_function,\n of the right end point of the updated bracketing interval.\n \"\"\"\n bracket_result = hzl.bracket(value_and_gradients_function, init_interval,\n f_lim, max_iterations, expansion_param)\n\n converged = init_interval.converged | _very_close(\n bracket_result.left.x, bracket_result.right.x)\n\n # We fail if we have not yet converged but already exhausted all iterations.\n exhausted_iterations = ~converged & tf.greater_equal(\n bracket_result.iteration, max_iterations)\n\n line_search_args = HagerZhangLineSearchResult(\n converged=converged,\n failed=bracket_result.failed | exhausted_iterations,\n iterations=bracket_result.iteration,\n func_evals=bracket_result.num_evals,\n left=bracket_result.left,\n right=bracket_result.right)\n\n return _line_search_after_bracketing(\n value_and_gradients_function, line_search_args, init_interval.left,\n f_lim, max_iterations, sufficient_decrease_param, curvature_param,\n shrinkage_param)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _line_search_after_bracketing(\n value_and_gradients_function,\n search_interval,\n val_0,\n f_lim,\n max_iterations,\n sufficient_decrease_param,\n curvature_param,\n shrinkage_param):\n \"\"\"The main loop of line search after the minimum has been bracketed.\n\n Args:\n value_and_gradients_function: A Python callable that accepts a real scalar\n tensor and returns a namedtuple with the fields 'x', 'f', and 'df' that\n correspond to scalar tensors of real dtype containing the point at which\n the function was evaluated, the value of the function, and its\n derivative at that point. The other namedtuple fields, if present,\n should be tensors or sequences (possibly nested) of tensors.\n In usual optimization application, this function would be generated by\n projecting the multivariate objective function along some specific\n direction. The direction is determined by some other procedure but should\n be a descent direction (i.e. the derivative of the projected univariate\n function must be negative at 0.).\n Alternatively, the function may represent the batching of `n` such line\n functions (e.g. projecting a single multivariate objective function along\n `n` distinct directions at once) accepting n points as input, i.e. a\n tensor of shape [n], and the fields 'x', 'f' and 'df' in the returned\n namedtuple should each be a tensor of shape [n], with the corresponding\n input points, function values, and derivatives at those input points.\n search_interval: Instance of `HagerZhangLineSearchResults` containing\n the current line search interval.\n val_0: A namedtuple as returned by value_and_gradients_function evaluated\n at `0.`. The gradient must be negative (i.e. must be a descent direction).\n f_lim: Scalar `Tensor` of float dtype.\n max_iterations: Positive scalar `Tensor` of integral dtype. The maximum\n number of iterations to perform in the line search. The number of\n iterations used to bracket the minimum are also counted against this\n parameter.\n sufficient_decrease_param: Positive scalar `Tensor` of real dtype.\n Bounded above by the curvature param. Corresponds to `delta` in the\n terminology of [Hager and Zhang (2006)][2].\n curvature_param: Positive scalar `Tensor` of real dtype. Bounded above\n by `1.`. Corresponds to 'sigma' in the terminology of\n [Hager and Zhang (2006)][2].\n shrinkage_param: Scalar positive Tensor of real dtype. Must be less than\n `1.`. Corresponds to the parameter `gamma` in [Hager and Zhang (2006)][2].\n\n Returns:\n A namedtuple containing the following fields.\n converged: Boolean `Tensor` of shape [n]. Whether a point satisfying\n Wolfe/Approx wolfe was found.\n failed: Boolean `Tensor` of shape [n]. Whether line search failed e.g.\n if either the objective function or the gradient are not finite at\n an evaluation point.\n iterations: Scalar int32 `Tensor`. Number of line search iterations made.\n func_evals: Scalar int32 `Tensor`. Number of function evaluations made.\n left: A namedtuple, as returned by value_and_gradients_function,\n of the left end point of the updated bracketing interval.\n right: A namedtuple, as returned by value_and_gradients_function,\n of the right end point of the updated bracketing interval.\n \"\"\"\n\n def _loop_cond(curr_interval):\n \"\"\"Loop condition.\"\"\"\n active = ~(curr_interval.converged | curr_interval.failed)\n return (curr_interval.iterations <\n max_iterations) & tf.reduce_any(input_tensor=active)\n\n def _loop_body(curr_interval):\n \"\"\"The loop body.\"\"\"\n secant2_raw_result = hzl.secant2(\n value_and_gradients_function, val_0, curr_interval, f_lim,\n sufficient_decrease_param, curvature_param)\n secant2_result = HagerZhangLineSearchResult(\n converged=secant2_raw_result.converged,\n failed=secant2_raw_result.failed,\n iterations=curr_interval.iterations + 1,\n func_evals=secant2_raw_result.num_evals,\n left=secant2_raw_result.left,\n right=secant2_raw_result.right)\n\n should_check_shrinkage = ~(secant2_result.converged | secant2_result.failed)\n\n def _do_check_shrinkage():\n \"\"\"Check if interval has shrinked enough.\"\"\"\n old_width = curr_interval.right.x - curr_interval.left.x\n new_width = secant2_result.right.x - secant2_result.left.x\n sufficient_shrinkage = new_width < old_width * shrinkage_param\n func_is_flat = (\n _very_close(curr_interval.left.f, curr_interval.right.f) &\n _very_close(secant2_result.left.f, secant2_result.right.f))\n\n new_converged = (\n should_check_shrinkage & sufficient_shrinkage & func_is_flat)\n needs_inner_bisect = should_check_shrinkage & ~sufficient_shrinkage\n\n inner_bisect_args = secant2_result._replace(\n converged=secant2_result.converged | new_converged)\n\n def _apply_inner_bisect():\n return _line_search_inner_bisection(\n value_and_gradients_function, inner_bisect_args,\n needs_inner_bisect, f_lim)\n\n return prefer_static.cond(\n tf.reduce_any(input_tensor=needs_inner_bisect),\n _apply_inner_bisect,\n lambda: inner_bisect_args)\n\n next_args = prefer_static.cond(\n tf.reduce_any(input_tensor=should_check_shrinkage),\n _do_check_shrinkage,\n lambda: secant2_result)\n\n interval_shrunk = (\n ~next_args.failed & _very_close(next_args.left.x, next_args.right.x))\n return [next_args._replace(converged=next_args.converged | interval_shrunk)]\n\n return tf.while_loop(\n cond=_loop_cond,\n body=_loop_body,\n loop_vars=[search_interval],\n parallel_iterations=1)[0]", "response": "This is the main loop of the HagerZhangLineSearchResults function that is called after the minimum has been bracketed."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef _line_search_inner_bisection(\n value_and_gradients_function,\n search_interval,\n active,\n f_lim):\n \"\"\"Performs bisection and updates the interval.\"\"\"\n midpoint = (search_interval.left.x + search_interval.right.x) / 2\n val_mid = value_and_gradients_function(midpoint)\n is_valid_mid = hzl.is_finite(val_mid)\n\n still_active = active & is_valid_mid\n new_failed = active & ~is_valid_mid\n next_inteval = search_interval._replace(\n failed=search_interval.failed | new_failed,\n func_evals=search_interval.func_evals + 1)\n\n def _apply_update():\n update_result = hzl.update(\n value_and_gradients_function, next_inteval.left, next_inteval.right,\n val_mid, f_lim, active=still_active)\n return HagerZhangLineSearchResult(\n converged=next_inteval.converged,\n failed=next_inteval.failed | update_result.failed,\n iterations=next_inteval.iterations + update_result.iteration,\n func_evals=next_inteval.func_evals + update_result.num_evals,\n left=update_result.left,\n right=update_result.right)\n\n return prefer_static.cond(\n tf.reduce_any(input_tensor=still_active),\n _apply_update,\n lambda: next_inteval)", "response": "Performs a bisection and updates the interval."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\npreparing the arguments for the line search.", "response": "def _prepare_args(value_and_gradients_function,\n initial_step_size,\n val_initial,\n val_0,\n approximate_wolfe_threshold):\n \"\"\"Prepares the arguments for the line search initialization.\n\n Args:\n value_and_gradients_function: A Python callable that accepts a real scalar\n tensor and returns a namedtuple with the fields 'x', 'f', and 'df' that\n correspond to scalar tensors of real dtype containing the point at which\n the function was evaluated, the value of the function, and its\n derivative at that point. The other namedtuple fields, if present,\n should be tensors or sequences (possibly nested) of tensors.\n In usual optimization application, this function would be generated by\n projecting the multivariate objective function along some specific\n direction. The direction is determined by some other procedure but should\n be a descent direction (i.e. the derivative of the projected univariate\n function must be negative at 0.).\n Alternatively, the function may represent the batching of `n` such line\n functions (e.g. projecting a single multivariate objective function along\n `n` distinct directions at once) accepting n points as input, i.e. a\n tensor of shape [n], and the fields 'x', 'f' and 'df' in the returned\n namedtuple should each be a tensor of shape [n], with the corresponding\n input points, function values, and derivatives at those input points.\n initial_step_size: Scalar positive `Tensor` of real dtype, or a tensor of\n shape [n] in batching mode. The initial value (or values) to try to\n bracket the minimum. Default is `1.` as a float32.\n Note that this point need not necessarily bracket the minimum for the line\n search to work correctly but the supplied value must be greater than 0.\n A good initial value will make the search converge faster.\n val_initial: The full return value of evaluating\n value_and_gradients_function at initial_step_size, i.e. a namedtuple with\n 'x', 'f', 'df', if already known by the caller. If not None the value of\n `initial_step_size` will be ignored, otherwise the tuple will be computed\n by evaluating value_and_gradients_function.\n val_0: The full return value of value_and_gradients_function at `0.`, i.e.\n a namedtuple with 'x', 'f', 'df', if already known by the caller. If None\n the tuple will be computed by evaluating value_and_gradients_function.\n approximate_wolfe_threshold: Scalar positive `Tensor` of\n real dtype. Corresponds to the parameter 'epsilon' in\n [Hager and Zhang (2006)][2]. Used to estimate the\n threshold at which the line search switches to approximate Wolfe\n conditions.\n\n Returns:\n left: A namedtuple, as returned by value_and_gradients_function,\n containing the value and derivative of the function at `0.`.\n val_initial: A namedtuple, as returned by value_and_gradients_function,\n containing the value and derivative of the function at\n `initial_step_size`.\n f_lim: Real `Tensor` of shape [n]. The function value threshold for\n the approximate Wolfe conditions to be checked.\n eval_count: Scalar int32 `Tensor`. The number of target function\n evaluations made by this function.\n \"\"\"\n eval_count = 0\n if val_initial is None:\n if initial_step_size is not None:\n initial_step_size = tf.convert_to_tensor(value=initial_step_size)\n else:\n initial_step_size = tf.convert_to_tensor(value=1.0, dtype=tf.float32)\n val_initial = value_and_gradients_function(initial_step_size)\n eval_count += 1\n\n if val_0 is None:\n x_0 = tf.zeros_like(val_initial.x)\n val_0 = value_and_gradients_function(x_0)\n eval_count += 1\n\n f_lim = val_0.f + (approximate_wolfe_threshold * tf.abs(val_0.f))\n return val_0, val_initial, f_lim, tf.convert_to_tensor(value=eval_count)"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _to_str(x):\n x = tf.convert_to_tensor(value=x)\n if x.dtype == tf.bool:\n return tf.where(x, tf.fill(x.shape, 'True'), tf.fill(x.shape, 'False'))\n return x", "response": "Converts a bool tensor to a string with True / False values."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nwraps for tf. Print which supports lists and namedtuples for printing.", "response": "def _print(pass_through_tensor, values):\n \"\"\"Wrapper for tf.Print which supports lists and namedtuples for printing.\"\"\"\n flat_values = []\n for value in values:\n # Checks if it is a namedtuple.\n if hasattr(value, '_fields'):\n for field in value._fields:\n flat_values.extend([field, _to_str(getattr(value, field))])\n continue\n if isinstance(value, (list, tuple)):\n for v in value:\n flat_values.append(_to_str(v))\n continue\n flat_values.append(_to_str(value))\n return tf.compat.v1.Print(pass_through_tensor, flat_values)"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _kl_brute_force(a, b, name=None):\n\n def squared_frobenius_norm(x):\n \"\"\"Helper to make KL calculation slightly more readable.\"\"\"\n # http://mathworld.wolfram.com/FrobeniusNorm.html\n # The gradient of KL[p,q] is not defined when p==q. The culprit is\n # tf.norm, i.e., we cannot use the commented out code.\n # return tf.square(tf.norm(x, ord=\"fro\", axis=[-2, -1]))\n return tf.reduce_sum(input_tensor=tf.square(x), axis=[-2, -1])\n\n # TODO(b/35041439): See also b/35040945. Remove this function once LinOp\n # supports something like:\n # A.inverse().solve(B).norm(order='fro', axis=[-1, -2])\n def is_diagonal(x):\n \"\"\"Helper to identify if `LinearOperator` has only a diagonal component.\"\"\"\n return (isinstance(x, tf.linalg.LinearOperatorIdentity) or\n isinstance(x, tf.linalg.LinearOperatorScaledIdentity) or\n isinstance(x, tf.linalg.LinearOperatorDiag))\n\n with tf.name_scope(name or \"kl_mvn\"):\n # Calculation is based on:\n # http://stats.stackexchange.com/questions/60680/kl-divergence-between-two-multivariate-gaussians\n # and,\n # https://en.wikipedia.org/wiki/Matrix_norm#Frobenius_norm\n # i.e.,\n # If Ca = AA', Cb = BB', then\n # tr[inv(Cb) Ca] = tr[inv(B)' inv(B) A A']\n # = tr[inv(B) A A' inv(B)']\n # = tr[(inv(B) A) (inv(B) A)']\n # = sum_{ij} (inv(B) A)_{ij}**2\n # = ||inv(B) A||_F**2\n # where ||.||_F is the Frobenius norm and the second equality follows from\n # the cyclic permutation property.\n if is_diagonal(a.scale) and is_diagonal(b.scale):\n # Using `stddev` because it handles expansion of Identity cases.\n b_inv_a = (a.stddev() / b.stddev())[..., tf.newaxis]\n else:\n b_inv_a = b.scale.solve(a.scale.to_dense())\n kl_div = (\n b.scale.log_abs_determinant() - a.scale.log_abs_determinant() +\n 0.5 * (-tf.cast(a.scale.domain_dimension_tensor(), a.dtype) +\n squared_frobenius_norm(b_inv_a) + squared_frobenius_norm(\n b.scale.solve((b.mean() - a.mean())[..., tf.newaxis]))))\n tensorshape_util.set_shape(\n kl_div, tf.broadcast_static_shape(a.batch_shape, b.batch_shape))\n return kl_div", "response": "Batchwise KL divergence for a batch of multivariate normals."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef quadrature_scheme_softmaxnormal_gauss_hermite(\n normal_loc, normal_scale, quadrature_size,\n validate_args=False, name=None):\n \"\"\"Use Gauss-Hermite quadrature to form quadrature on `K - 1` simplex.\n\n A `SoftmaxNormal` random variable `Y` may be generated via\n\n ```\n Y = SoftmaxCentered(X),\n X = Normal(normal_loc, normal_scale)\n ```\n\n Note: for a given `quadrature_size`, this method is generally less accurate\n than `quadrature_scheme_softmaxnormal_quantiles`.\n\n Args:\n normal_loc: `float`-like `Tensor` with shape `[b1, ..., bB, K-1]`, B>=0.\n The location parameter of the Normal used to construct the SoftmaxNormal.\n normal_scale: `float`-like `Tensor`. Broadcastable with `normal_loc`.\n The scale parameter of the Normal used to construct the SoftmaxNormal.\n quadrature_size: Python `int` scalar representing the number of quadrature\n points.\n validate_args: Python `bool`, default `False`. When `True` distribution\n parameters are checked for validity despite possibly degrading runtime\n performance. When `False` invalid inputs may silently render incorrect\n outputs.\n name: Python `str` name prefixed to Ops created by this class.\n\n Returns:\n grid: Shape `[b1, ..., bB, K, quadrature_size]` `Tensor` representing the\n convex combination of affine parameters for `K` components.\n `grid[..., :, n]` is the `n`-th grid point, living in the `K - 1` simplex.\n probs: Shape `[b1, ..., bB, K, quadrature_size]` `Tensor` representing the\n associated with each grid point.\n \"\"\"\n with tf.name_scope(\n name or \"quadrature_scheme_softmaxnormal_gauss_hermite\"):\n normal_loc = tf.convert_to_tensor(value=normal_loc, name=\"normal_loc\")\n npdt = dtype_util.as_numpy_dtype(normal_loc.dtype)\n normal_scale = tf.convert_to_tensor(\n value=normal_scale, dtype=npdt, name=\"normal_scale\")\n\n normal_scale = maybe_check_quadrature_param(\n normal_scale, \"normal_scale\", validate_args)\n\n grid, probs = np.polynomial.hermite.hermgauss(deg=quadrature_size)\n grid = grid.astype(npdt)\n probs = probs.astype(npdt)\n probs /= np.linalg.norm(probs, ord=1, keepdims=True)\n probs = tf.convert_to_tensor(value=probs, name=\"probs\", dtype=npdt)\n\n grid = softmax(\n -distribution_util.pad(\n (normal_loc[..., tf.newaxis] +\n np.sqrt(2.) * normal_scale[..., tf.newaxis] * grid),\n axis=-2,\n front=True),\n axis=-2) # shape: [B, components, deg]\n\n return grid, probs", "response": "A simplex that uses Gauss - Hermite quadrature to form a quadrature on a given set of points."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef quadrature_scheme_softmaxnormal_quantiles(\n normal_loc, normal_scale, quadrature_size,\n validate_args=False, name=None):\n \"\"\"Use SoftmaxNormal quantiles to form quadrature on `K - 1` simplex.\n\n A `SoftmaxNormal` random variable `Y` may be generated via\n\n ```\n Y = SoftmaxCentered(X),\n X = Normal(normal_loc, normal_scale)\n ```\n\n Args:\n normal_loc: `float`-like `Tensor` with shape `[b1, ..., bB, K-1]`, B>=0.\n The location parameter of the Normal used to construct the SoftmaxNormal.\n normal_scale: `float`-like `Tensor`. Broadcastable with `normal_loc`.\n The scale parameter of the Normal used to construct the SoftmaxNormal.\n quadrature_size: Python `int` scalar representing the number of quadrature\n points.\n validate_args: Python `bool`, default `False`. When `True` distribution\n parameters are checked for validity despite possibly degrading runtime\n performance. When `False` invalid inputs may silently render incorrect\n outputs.\n name: Python `str` name prefixed to Ops created by this class.\n\n Returns:\n grid: Shape `[b1, ..., bB, K, quadrature_size]` `Tensor` representing the\n convex combination of affine parameters for `K` components.\n `grid[..., :, n]` is the `n`-th grid point, living in the `K - 1` simplex.\n probs: Shape `[b1, ..., bB, K, quadrature_size]` `Tensor` representing the\n associated with each grid point.\n \"\"\"\n with tf.name_scope(name or \"softmax_normal_grid_and_probs\"):\n normal_loc = tf.convert_to_tensor(value=normal_loc, name=\"normal_loc\")\n dt = dtype_util.base_dtype(normal_loc.dtype)\n normal_scale = tf.convert_to_tensor(\n value=normal_scale, dtype=dt, name=\"normal_scale\")\n\n normal_scale = maybe_check_quadrature_param(\n normal_scale, \"normal_scale\", validate_args)\n\n dist = normal.Normal(loc=normal_loc, scale=normal_scale)\n\n def _get_batch_ndims():\n \"\"\"Helper to get rank(dist.batch_shape), statically if possible.\"\"\"\n ndims = tensorshape_util.rank(dist.batch_shape)\n if ndims is None:\n ndims = tf.shape(input=dist.batch_shape_tensor())[0]\n return ndims\n batch_ndims = _get_batch_ndims()\n\n def _get_final_shape(qs):\n \"\"\"Helper to build `TensorShape`.\"\"\"\n bs = tensorshape_util.with_rank_at_least(dist.batch_shape, 1)\n num_components = tf.compat.dimension_value(bs[-1])\n if num_components is not None:\n num_components += 1\n tail = tf.TensorShape([num_components, qs])\n return bs[:-1].concatenate(tail)\n\n def _compute_quantiles():\n \"\"\"Helper to build quantiles.\"\"\"\n # Omit {0, 1} since they might lead to Inf/NaN.\n zero = tf.zeros([], dtype=dist.dtype)\n edges = tf.linspace(zero, 1., quadrature_size + 3)[1:-1]\n # Expand edges so its broadcast across batch dims.\n edges = tf.reshape(\n edges,\n shape=tf.concat(\n [[-1], tf.ones([batch_ndims], dtype=tf.int32)], axis=0))\n quantiles = dist.quantile(edges)\n quantiles = softmax_centered_bijector.SoftmaxCentered().forward(quantiles)\n # Cyclically permute left by one.\n perm = tf.concat([tf.range(1, 1 + batch_ndims), [0]], axis=0)\n quantiles = tf.transpose(a=quantiles, perm=perm)\n tensorshape_util.set_shape(\n quantiles, _get_final_shape(quadrature_size + 1))\n return quantiles\n quantiles = _compute_quantiles()\n\n # Compute grid as quantile midpoints.\n grid = (quantiles[..., :-1] + quantiles[..., 1:]) / 2.\n # Set shape hints.\n tensorshape_util.set_shape(grid, _get_final_shape(quadrature_size))\n\n # By construction probs is constant, i.e., `1 / quadrature_size`. This is\n # important, because non-constant probs leads to non-reparameterizable\n # samples.\n probs = tf.fill(\n dims=[quadrature_size], value=1. / tf.cast(quadrature_size, dist.dtype))\n\n return grid, probs", "response": "A function that generates a graph of uniform uniform."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef maybe_check_quadrature_param(param, name, validate_args):\n with tf.name_scope(\"check_\" + name):\n assertions = []\n if tensorshape_util.rank(param.shape) is not None:\n if tensorshape_util.rank(param.shape) == 0:\n raise ValueError(\"Mixing params must be a (batch of) vector; \"\n \"{}.rank={} is not at least one.\".format(\n name, tensorshape_util.rank(param.shape)))\n elif validate_args:\n assertions.append(\n assert_util.assert_rank_at_least(\n param,\n 1,\n message=(\"Mixing params must be a (batch of) vector; \"\n \"{}.rank is not at least one.\".format(name))))\n\n # TODO(jvdillon): Remove once we support k-mixtures.\n if tensorshape_util.with_rank_at_least(param.shape, 1)[-1] is not None:\n if tf.compat.dimension_value(param.shape[-1]) != 1:\n raise NotImplementedError(\"Currently only bimixtures are supported; \"\n \"{}.shape[-1]={} is not 1.\".format(\n name,\n tf.compat.dimension_value(\n param.shape[-1])))\n elif validate_args:\n assertions.append(\n assert_util.assert_equal(\n tf.shape(input=param)[-1],\n 1,\n message=(\"Currently only bimixtures are supported; \"\n \"{}.shape[-1] is not 1.\".format(name))))\n\n if assertions:\n return distribution_util.with_dependencies(assertions, param)\n return param", "response": "Helper which checks validity of loc and scale init args."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef determine_batch_event_shapes(grid, endpoint_affine):\n with tf.name_scope(\"determine_batch_event_shapes\"):\n # grid # shape: [B, k, q]\n # endpoint_affine # len=k, shape: [B, d, d]\n batch_shape = grid.shape[:-2]\n batch_shape_tensor = tf.shape(input=grid)[:-2]\n event_shape = None\n event_shape_tensor = None\n\n def _set_event_shape(shape, shape_tensor):\n if event_shape is None:\n return shape, shape_tensor\n return (tf.broadcast_static_shape(event_shape, shape),\n tf.broadcast_dynamic_shape(event_shape_tensor, shape_tensor))\n\n for aff in endpoint_affine:\n if aff.shift is not None:\n batch_shape = tf.broadcast_static_shape(batch_shape,\n aff.shift.shape[:-1])\n batch_shape_tensor = tf.broadcast_dynamic_shape(\n batch_shape_tensor,\n tf.shape(input=aff.shift)[:-1])\n event_shape, event_shape_tensor = _set_event_shape(\n aff.shift.shape[-1:],\n tf.shape(input=aff.shift)[-1:])\n\n if aff.scale is not None:\n batch_shape = tf.broadcast_static_shape(batch_shape,\n aff.scale.batch_shape)\n batch_shape_tensor = tf.broadcast_dynamic_shape(\n batch_shape_tensor, aff.scale.batch_shape_tensor())\n event_shape, event_shape_tensor = _set_event_shape(\n tf.TensorShape([aff.scale.range_dimension]),\n aff.scale.range_dimension_tensor()[tf.newaxis])\n\n return batch_shape, batch_shape_tensor, event_shape, event_shape_tensor", "response": "Helper to infer batch_shape and event_shape."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef interpolate_loc(grid, loc):\n if len(loc) != 2:\n raise NotImplementedError(\"Currently only bimixtures are supported; \"\n \"len(scale)={} is not 2.\".format(len(loc)))\n deg = tf.compat.dimension_value(\n tensorshape_util.with_rank_at_least(grid.shape, 1)[-1])\n if deg is None:\n raise ValueError(\"Num quadrature grid points must be known prior \"\n \"to graph execution.\")\n with tf.name_scope(\"interpolate_loc\"):\n if loc is None or loc[0] is None and loc[1] is None:\n return [None]*deg\n # shape: [B, 1, k, deg]\n w = grid[..., tf.newaxis, :, :]\n loc = [\n x[..., tf.newaxis] # shape: [B, e, 1]\n if x is not None else None for x in loc\n ]\n if loc[0] is None:\n x = w[..., 1, :] * loc[1] # shape: [B, e, deg]\n elif loc[1] is None:\n x = w[..., 0, :] * loc[0] # shape: [B, e, deg]\n else:\n delta = loc[0] - loc[1]\n x = w[..., 0, :] * delta + loc[1] # shape: [B, e, deg]\n return [x[..., k] for k in range(deg)]", "response": "Helper which interpolates between two locs."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\ncreates weighted LinearOp from an existing LinOp.", "response": "def linop_scale(w, op):\n \"\"\"Creates weighted `LinOp` from existing `LinOp`.\"\"\"\n # We assume w > 0. (This assumption only relates to the is_* attributes.)\n with tf.name_scope(\"linop_scale\"):\n # TODO(b/35301104): LinearOperatorComposition doesn't combine operators, so\n # special case combinations here. Once it does, this function can be\n # replaced by:\n # return linop_composition_lib.LinearOperatorComposition([\n # scaled_identity(w), op])\n def scaled_identity(w):\n return tf.linalg.LinearOperatorScaledIdentity(\n num_rows=op.range_dimension_tensor(),\n multiplier=w,\n is_non_singular=op.is_non_singular,\n is_self_adjoint=op.is_self_adjoint,\n is_positive_definite=op.is_positive_definite)\n\n if isinstance(op, tf.linalg.LinearOperatorIdentity):\n return scaled_identity(w)\n if isinstance(op, tf.linalg.LinearOperatorScaledIdentity):\n return scaled_identity(w * op.multiplier)\n if isinstance(op, tf.linalg.LinearOperatorDiag):\n return tf.linalg.LinearOperatorDiag(\n diag=w[..., tf.newaxis] * op.diag_part(),\n is_non_singular=op.is_non_singular,\n is_self_adjoint=op.is_self_adjoint,\n is_positive_definite=op.is_positive_definite)\n if isinstance(op, tf.linalg.LinearOperatorLowerTriangular):\n return tf.linalg.LinearOperatorLowerTriangular(\n tril=w[..., tf.newaxis, tf.newaxis] * op.to_dense(),\n is_non_singular=op.is_non_singular,\n is_self_adjoint=op.is_self_adjoint,\n is_positive_definite=op.is_positive_definite)\n raise NotImplementedError(\n \"Unsupported Linop type ({})\".format(type(op).__name__))"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nconcatenate input vectors statically if possible.", "response": "def concat_vectors(*args):\n \"\"\"Concatenates input vectors, statically if possible.\"\"\"\n args_ = [tf.get_static_value(x) for x in args]\n if any(vec is None for vec in args_):\n return tf.concat(args, axis=0)\n return [val for vec in args_ for val in vec]"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef softmax(x, axis, name=None):\n with tf.name_scope(name or \"softmax\"):\n x = tf.convert_to_tensor(value=x, name=\"x\")\n ndims = (\n tensorshape_util.rank(x.shape)\n if tensorshape_util.rank(x.shape) is not None else tf.rank(\n x, name=\"ndims\"))\n axis = tf.convert_to_tensor(value=axis, dtype=tf.int32, name=\"axis\")\n axis_ = tf.get_static_value(axis)\n if axis_ is not None:\n axis = np.int(ndims + axis_ if axis_ < 0 else axis_)\n else:\n axis = tf.where(axis < 0, ndims + axis, axis)\n return tf.nn.softmax(x, axis=axis)", "response": "Equivalent to tf. nn. softmax but works around b/70297725."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nensures self. distribution. mean has [ batch event shape ].", "response": "def _expand_base_distribution_mean(self):\n \"\"\"Ensures `self.distribution.mean()` has `[batch, event]` shape.\"\"\"\n single_draw_shape = concat_vectors(self.batch_shape_tensor(),\n self.event_shape_tensor())\n m = tf.reshape(\n self.distribution.mean(), # A scalar.\n shape=tf.ones_like(single_draw_shape, dtype=tf.int32))\n m = tf.tile(m, multiples=single_draw_shape)\n tensorshape_util.set_shape(\n m, tensorshape_util.concatenate(self.batch_shape, self.event_shape))\n return m"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nmultiplies tensor of vectors by matrices assuming values stored are logs.", "response": "def _log_vector_matrix(vs, ms):\n \"\"\"Multiply tensor of vectors by matrices assuming values stored are logs.\"\"\"\n\n return tf.reduce_logsumexp(input_tensor=vs[..., tf.newaxis] + ms, axis=-2)"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _log_matrix_vector(ms, vs):\n\n return tf.reduce_logsumexp(input_tensor=ms + vs[..., tf.newaxis, :], axis=-1)", "response": "Multiply tensor of matrices by vectors assuming values stored are logs."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nmultiply tensor of vectors by matrices.", "response": "def _vector_matrix(vs, ms):\n \"\"\"Multiply tensor of vectors by matrices.\"\"\"\n\n return tf.reduce_sum(input_tensor=vs[..., tf.newaxis] * ms, axis=-2)"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef _extract_log_probs(num_states, dist):\n\n states = tf.reshape(tf.range(num_states),\n tf.concat([[num_states],\n tf.ones_like(dist.batch_shape_tensor())],\n axis=0))\n return distribution_util.move_dimension(dist.log_prob(states), 0, -1)", "response": "Tabulate log probabilities from a batch of distributions."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\ncompute marginal pdf for each individual observable.", "response": "def _marginal_hidden_probs(self):\n \"\"\"Compute marginal pdf for each individual observable.\"\"\"\n\n initial_log_probs = tf.broadcast_to(self._log_init,\n tf.concat([self.batch_shape_tensor(),\n [self._num_states]],\n axis=0))\n # initial_log_probs :: batch_shape num_states\n\n if self._num_steps > 1:\n transition_log_probs = self._log_trans\n\n def forward_step(log_probs, _):\n return _log_vector_matrix(log_probs, transition_log_probs)\n\n dummy_index = tf.zeros(self._num_steps - 1, dtype=tf.float32)\n\n forward_log_probs = tf.scan(forward_step, dummy_index,\n initializer=initial_log_probs,\n name=\"forward_log_probs\")\n\n forward_log_probs = tf.concat([[initial_log_probs], forward_log_probs],\n axis=0)\n else:\n forward_log_probs = initial_log_probs[tf.newaxis, ...]\n\n # returns :: num_steps batch_shape num_states\n\n return tf.exp(forward_log_probs)"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef posterior_marginals(self, observations, name=None):\n\n with tf.name_scope(name or \"posterior_marginals\"):\n with tf.control_dependencies(self._runtime_assertions):\n observation_tensor_shape = tf.shape(input=observations)\n\n with self._observation_shape_preconditions(observation_tensor_shape):\n observation_batch_shape = observation_tensor_shape[\n :-1 - self._underlying_event_rank]\n observation_event_shape = observation_tensor_shape[\n -1 - self._underlying_event_rank:]\n\n batch_shape = tf.broadcast_dynamic_shape(observation_batch_shape,\n self.batch_shape_tensor())\n log_init = tf.broadcast_to(self._log_init,\n tf.concat([batch_shape,\n [self._num_states]],\n axis=0))\n log_transition = self._log_trans\n\n observations = tf.broadcast_to(observations,\n tf.concat([batch_shape,\n observation_event_shape],\n axis=0))\n observation_rank = tf.rank(observations)\n underlying_event_rank = self._underlying_event_rank\n observations = distribution_util.move_dimension(\n observations, observation_rank - underlying_event_rank - 1, 0)\n observations = tf.expand_dims(\n observations,\n observation_rank - underlying_event_rank)\n observation_log_probs = self._observation_distribution.log_prob(\n observations)\n\n log_adjoint_prob = tf.zeros_like(log_init)\n\n def forward_step(log_previous_step, log_prob_observation):\n return _log_vector_matrix(log_previous_step,\n log_transition) + log_prob_observation\n\n log_prob = log_init + observation_log_probs[0]\n\n forward_log_probs = tf.scan(forward_step, observation_log_probs[1:],\n initializer=log_prob,\n name=\"forward_log_probs\")\n\n forward_log_probs = tf.concat([[log_prob], forward_log_probs], axis=0)\n\n def backward_step(log_previous_step, log_prob_observation):\n return _log_matrix_vector(log_transition,\n log_prob_observation + log_previous_step)\n\n backward_log_adjoint_probs = tf.scan(\n backward_step,\n observation_log_probs[1:],\n initializer=log_adjoint_prob,\n reverse=True,\n name=\"backward_log_adjoint_probs\")\n\n total_log_prob = tf.reduce_logsumexp(\n input_tensor=forward_log_probs[-1], axis=-1)\n\n backward_log_adjoint_probs = tf.concat([backward_log_adjoint_probs,\n [log_adjoint_prob]], axis=0)\n\n log_likelihoods = forward_log_probs + backward_log_adjoint_probs\n\n marginal_log_probs = distribution_util.move_dimension(\n log_likelihoods - total_log_prob[..., tf.newaxis], 0, -2)\n\n return categorical.Categorical(logits=marginal_log_probs)", "response": "Compute the marginal posterior probability for each state."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef posterior_mode(self, observations, name=None):\n\n with tf.name_scope(name or \"posterior_mode\"):\n with tf.control_dependencies(self._runtime_assertions):\n observation_tensor_shape = tf.shape(input=observations)\n\n with self._observation_shape_preconditions(observation_tensor_shape):\n observation_batch_shape = observation_tensor_shape[\n :-1 - self._underlying_event_rank]\n observation_event_shape = observation_tensor_shape[\n -1 - self._underlying_event_rank:]\n\n batch_shape = tf.broadcast_dynamic_shape(observation_batch_shape,\n self.batch_shape_tensor())\n log_init = tf.broadcast_to(self._log_init,\n tf.concat([batch_shape,\n [self._num_states]],\n axis=0))\n\n observations = tf.broadcast_to(observations,\n tf.concat([batch_shape,\n observation_event_shape],\n axis=0))\n observation_rank = tf.rank(observations)\n underlying_event_rank = self._underlying_event_rank\n observations = distribution_util.move_dimension(\n observations, observation_rank - underlying_event_rank - 1, 0)\n\n # We need to compute the probability of each observation for\n # each possible state.\n # This requires inserting an extra index just before the\n # observation event indices that will be broadcast with the\n # last batch index in `observation_distribution`.\n observations = tf.expand_dims(\n observations,\n observation_rank - underlying_event_rank)\n observation_log_probs = self._observation_distribution.log_prob(\n observations)\n\n log_prob = log_init + observation_log_probs[0]\n\n if self._num_steps == 1:\n most_likely_end = tf.argmax(input=log_prob, axis=-1)\n return most_likely_end[..., tf.newaxis]\n\n def forward_step(previous_step_pair, log_prob_observation):\n log_prob_previous = previous_step_pair[0]\n log_prob = (log_prob_previous[..., tf.newaxis] +\n self._log_trans +\n log_prob_observation[..., tf.newaxis, :])\n most_likely_given_successor = tf.argmax(input=log_prob, axis=-2)\n max_log_p_given_successor = tf.reduce_max(input_tensor=log_prob,\n axis=-2)\n return (max_log_p_given_successor, most_likely_given_successor)\n\n forward_log_probs, all_most_likely_given_successor = tf.scan(\n forward_step,\n observation_log_probs[1:],\n initializer=(log_prob,\n tf.zeros(tf.shape(input=log_init), dtype=tf.int64)),\n name=\"forward_log_probs\")\n\n most_likely_end = tf.argmax(input=forward_log_probs[-1], axis=-1)\n\n # We require the operation that gives C from A and B where\n # C[i...j] = A[i...j, B[i...j]]\n # and A = most_likely_given_successor\n # B = most_likely_successor.\n # tf.gather requires indices of known shape so instead we use\n # reduction with tf.one_hot(B) to pick out elements from B\n def backward_step(most_likely_successor, most_likely_given_successor):\n return tf.reduce_sum(\n input_tensor=(most_likely_given_successor *\n tf.one_hot(most_likely_successor,\n self._num_states,\n dtype=tf.int64)),\n axis=-1)\n\n backward_scan = tf.scan(\n backward_step,\n all_most_likely_given_successor,\n most_likely_end,\n reverse=True)\n most_likely_sequences = tf.concat([backward_scan, [most_likely_end]],\n axis=0)\n return distribution_util.move_dimension(most_likely_sequences, 0, -1)", "response": "This function computes the maximum likelihood sequence of hidden states for a given set of observations."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef _choose_random_direction(current_state_parts, batch_rank, seed=None):\n seed_gen = distributions.SeedStream(seed, salt='_choose_random_direction')\n # Chooses the random directions across each of the input components.\n rnd_direction_parts = [\n tf.random.normal(\n tf.shape(input=current_state_part), dtype=tf.float32, seed=seed_gen())\n for current_state_part in current_state_parts\n ]\n\n # Sum squares over all of the input components. Note this takes all\n # components into account.\n sum_squares = sum(\n tf.reduce_sum(\n input_tensor=rnd_direction**2.,\n axis=tf.range(batch_rank, tf.rank(rnd_direction)),\n keepdims=True) for rnd_direction in rnd_direction_parts)\n\n # Normalizes the random direction fragments.\n rnd_direction_parts = [rnd_direction / tf.sqrt(sum_squares)\n for rnd_direction in rnd_direction_parts]\n\n return rnd_direction_parts", "response": "Chooses a random direction in the event space."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\napplying a single iteration of slice sampling update.", "response": "def _sample_next(target_log_prob_fn,\n current_state_parts,\n step_sizes,\n max_doublings,\n current_target_log_prob,\n batch_rank,\n seed=None,\n name=None):\n \"\"\"Applies a single iteration of slice sampling update.\n\n Applies hit and run style slice sampling. Chooses a uniform random direction\n on the unit sphere in the event space. Applies the one dimensional slice\n sampling update along that direction.\n\n Args:\n target_log_prob_fn: Python callable which takes an argument like\n `*current_state_parts` and returns its (possibly unnormalized) log-density\n under the target distribution.\n current_state_parts: Python `list` of `Tensor`s representing the current\n state(s) of the Markov chain(s). The first `independent_chain_ndims` of\n the `Tensor`(s) index different chains.\n step_sizes: Python `list` of `Tensor`s. Provides a measure of the width\n of the density. Used to find the slice bounds. Must broadcast with the\n shape of `current_state_parts`.\n max_doublings: Integer number of doublings to allow while locating the slice\n boundaries.\n current_target_log_prob: `Tensor` representing the value of\n `target_log_prob_fn(*current_state_parts)`. The only reason to specify\n this argument is to reduce TF graph size.\n batch_rank: Integer. The number of axes in the state that correspond to\n independent batches.\n seed: Python integer to seed random number generators.\n name: Python `str` name prefixed to Ops created by this function.\n Default value: `None` (i.e., 'find_slice_bounds').\n\n Returns:\n proposed_state_parts: Tensor or Python list of `Tensor`s representing the\n state(s) of the Markov chain(s) at each result step. Has same shape as\n input `current_state_parts`.\n proposed_target_log_prob: `Tensor` representing the value of\n `target_log_prob_fn` at `next_state`.\n bounds_satisfied: Boolean `Tensor` of the same shape as the log density.\n True indicates whether the an interval containing the slice for that\n batch was found successfully.\n direction: `Tensor` or Python list of `Tensors`s representing the direction\n along which the slice was sampled. Has the same shape and dtype(s) as\n `current_state_parts`.\n upper_bounds: `Tensor` of batch shape and the dtype of the input state. The\n upper bounds of the slices along the sampling direction.\n lower_bounds: `Tensor` of batch shape and the dtype of the input state. The\n lower bounds of the slices along the sampling direction.\n \"\"\"\n with tf.compat.v1.name_scope(name, 'sample_next', [\n current_state_parts, step_sizes, max_doublings, current_target_log_prob,\n batch_rank\n ]):\n # First step: Choose a random direction.\n # Direction is a list of tensors. The i'th tensor should have the same shape\n # as the i'th state part.\n direction = _choose_random_direction(current_state_parts,\n batch_rank=batch_rank,\n seed=seed)\n\n # Interpolates the step sizes for the chosen direction.\n # Applies an ellipsoidal interpolation to compute the step direction for\n # the chosen direction. Suppose we are given step sizes for each direction.\n # Label these s_1, s_2, ... s_k. These are the step sizes to use if moving\n # in a direction parallel to one of the axes. Consider an ellipsoid which\n # intercepts the i'th axis at s_i. The step size for a direction specified\n # by the unit vector (n_1, n_2 ...n_k) is then defined as the intersection\n # of the line through this vector with this ellipsoid.\n #\n # One can show that the length of the vector from the origin to the\n # intersection point is given by:\n # 1 / sqrt(n_1^2 / s_1^2 + n_2^2 / s_2^2 + ...).\n #\n # Proof:\n # The equation of the ellipsoid is:\n # Sum_i [x_i^2 / s_i^2 ] = 1. Let n be a unit direction vector. Points\n # along the line given by n may be parameterized as alpha*n where alpha is\n # the distance along the vector. Plugging this into the equation for the\n # ellipsoid, we get:\n # alpha^2 ( n_1^2 / s_1^2 + n_2^2 / s_2^2 + ...) = 1\n # so alpha = \\sqrt { \\frac{1} { ( n_1^2 / s_1^2 + n_2^2 / s_2^2 + ...) } }\n reduce_axes = [tf.range(batch_rank, tf.rank(dirn_part))\n for dirn_part in direction]\n\n components = [\n tf.reduce_sum(\n input_tensor=(dirn_part / step_size)**2, axis=reduce_axes[i])\n for i, (step_size, dirn_part) in enumerate(zip(step_sizes, direction))\n ]\n step_size = tf.math.rsqrt(tf.add_n(components))\n # Computes the rank of a tensor. Uses the static rank if possible.\n def _get_rank(x):\n return (len(x.shape.as_list()) if x.shape.dims is not None\n else tf.rank(x))\n state_part_ranks = [_get_rank(part) for part in current_state_parts]\n\n def _step_along_direction(alpha):\n \"\"\"Converts the scalar alpha into an n-dim vector with full state info.\n\n Computes x_0 + alpha * direction where x_0 is the current state and\n direction is the direction chosen above.\n\n Args:\n alpha: A tensor of shape equal to the batch dimensions of\n `current_state_parts`.\n\n Returns:\n state_parts: Tensor or Python list of `Tensor`s representing the\n state(s) of the Markov chain(s) for a given alpha and a given chosen\n direction. Has the same shape as `current_state_parts`.\n \"\"\"\n padded_alphas = [_right_pad(alpha, final_rank=part_rank)\n for part_rank in state_part_ranks]\n\n state_parts = [state_part + padded_alpha\n * direction_part for state_part, direction_part,\n padded_alpha in\n zip(current_state_parts, direction, padded_alphas)]\n return state_parts\n\n def projected_target_log_prob_fn(alpha):\n \"\"\"The target log density projected along the chosen direction.\n\n Args:\n alpha: A tensor of shape equal to the batch dimensions of\n `current_state_parts`.\n\n Returns:\n Target log density evaluated at x_0 + alpha * direction where x_0 is the\n current state and direction is the direction chosen above. Has the same\n shape as `alpha`.\n \"\"\"\n return target_log_prob_fn(*_step_along_direction(alpha))\n\n alpha_init = tf.zeros_like(current_target_log_prob,\n dtype=current_state_parts[0].dtype.base_dtype)\n [\n next_alpha,\n next_target_log_prob,\n bounds_satisfied,\n upper_bounds,\n lower_bounds\n ] = ssu.slice_sampler_one_dim(projected_target_log_prob_fn,\n x_initial=alpha_init,\n max_doublings=max_doublings,\n step_size=step_size, seed=seed)\n return [\n _step_along_direction(next_alpha),\n next_target_log_prob,\n bounds_satisfied,\n direction,\n upper_bounds,\n lower_bounds\n ]"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _right_pad(x, final_rank):\n padded_shape = tf.concat(\n [tf.shape(input=x),\n tf.ones(final_rank - tf.rank(x), dtype=tf.int32)],\n axis=0)\n static_padded_shape = None\n if x.shape.is_fully_defined() and isinstance(final_rank, int):\n static_padded_shape = x.shape.as_list()\n extra_dims = final_rank - len(static_padded_shape)\n static_padded_shape.extend([1] * extra_dims)\n\n padded_x = tf.reshape(x, static_padded_shape or padded_shape)\n return padded_x", "response": "Pads the shape of x to the right to be of rank final_rank."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef _prepare_args(target_log_prob_fn, state, step_size,\n target_log_prob=None, maybe_expand=False,\n description='target_log_prob'):\n \"\"\"Processes input args to meet list-like assumptions.\"\"\"\n state_parts = list(state) if mcmc_util.is_list_like(state) else [state]\n state_parts = [\n tf.convert_to_tensor(value=s, name='current_state') for s in state_parts\n ]\n\n target_log_prob = _maybe_call_fn(\n target_log_prob_fn,\n state_parts,\n target_log_prob,\n description)\n step_sizes = (list(step_size) if mcmc_util.is_list_like(step_size)\n else [step_size])\n step_sizes = [\n tf.convert_to_tensor(\n value=s, name='step_size', dtype=target_log_prob.dtype)\n for s in step_sizes\n ]\n if len(step_sizes) == 1:\n step_sizes *= len(state_parts)\n if len(state_parts) != len(step_sizes):\n raise ValueError('There should be exactly one `step_size` or it should '\n 'have same length as `current_state`.')\n def maybe_flatten(x):\n return x if maybe_expand or mcmc_util.is_list_like(state) else x[0]\n return [\n maybe_flatten(state_parts),\n maybe_flatten(step_sizes),\n target_log_prob\n ]", "response": "Processes input args to meet list - like assumptions."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nrun one iteration of Slice Sampler.", "response": "def one_step(self, current_state, previous_kernel_results):\n \"\"\"Runs one iteration of Slice Sampler.\n\n Args:\n current_state: `Tensor` or Python `list` of `Tensor`s representing the\n current state(s) of the Markov chain(s). The first `r` dimensions\n index independent chains,\n `r = tf.rank(target_log_prob_fn(*current_state))`.\n previous_kernel_results: `collections.namedtuple` containing `Tensor`s\n representing values from previous calls to this function (or from the\n `bootstrap_results` function.)\n\n Returns:\n next_state: Tensor or Python list of `Tensor`s representing the state(s)\n of the Markov chain(s) after taking exactly one step. Has same type and\n shape as `current_state`.\n kernel_results: `collections.namedtuple` of internal calculations used to\n advance the chain.\n\n Raises:\n ValueError: if there isn't one `step_size` or a list with same length as\n `current_state`.\n TypeError: if `not target_log_prob.dtype.is_floating`.\n \"\"\"\n with tf.compat.v1.name_scope(\n name=mcmc_util.make_name(self.name, 'slice', 'one_step'),\n values=[\n self.step_size, self.max_doublings, self._seed_stream,\n current_state, previous_kernel_results.target_log_prob\n ]):\n with tf.compat.v1.name_scope('initialize'):\n [\n current_state_parts,\n step_sizes,\n current_target_log_prob\n ] = _prepare_args(\n self.target_log_prob_fn,\n current_state,\n self.step_size,\n previous_kernel_results.target_log_prob,\n maybe_expand=True)\n\n max_doublings = tf.convert_to_tensor(\n value=self.max_doublings, dtype=tf.int32, name='max_doublings')\n\n independent_chain_ndims = distribution_util.prefer_static_rank(\n current_target_log_prob)\n\n [\n next_state_parts,\n next_target_log_prob,\n bounds_satisfied,\n direction,\n upper_bounds,\n lower_bounds\n ] = _sample_next(\n self.target_log_prob_fn,\n current_state_parts,\n step_sizes,\n max_doublings,\n current_target_log_prob,\n independent_chain_ndims,\n seed=self._seed_stream()\n )\n\n def maybe_flatten(x):\n return x if mcmc_util.is_list_like(current_state) else x[0]\n\n return [\n maybe_flatten(next_state_parts),\n SliceSamplerKernelResults(\n target_log_prob=next_target_log_prob,\n bounds_satisfied=bounds_satisfied,\n direction=direction,\n upper_bounds=upper_bounds,\n lower_bounds=lower_bounds\n ),\n ]"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\ninitialize from a uniform - 2 distribution in unconstrained space.", "response": "def sample_uniform_initial_state(parameter,\n return_constrained=True,\n init_sample_shape=(),\n seed=None):\n \"\"\"Initialize from a uniform [-2, 2] distribution in unconstrained space.\n\n Args:\n parameter: `sts.Parameter` named tuple instance.\n return_constrained: if `True`, re-applies the constraining bijector\n to return initializations in the original domain. Otherwise, returns\n initializations in the unconstrained space.\n Default value: `True`.\n init_sample_shape: `sample_shape` of the sampled initializations.\n Default value: `[]`.\n seed: Python integer to seed the random number generator.\n\n Returns:\n uniform_initializer: `Tensor` of shape `concat([init_sample_shape,\n parameter.prior.batch_shape, transformed_event_shape])`, where\n `transformed_event_shape` is `parameter.prior.event_shape`, if\n `return_constrained=True`, and otherwise it is\n `parameter.bijector.inverse_event_shape(parameteter.prior.event_shape)`.\n \"\"\"\n unconstrained_prior_sample = parameter.bijector.inverse(\n parameter.prior.sample(init_sample_shape, seed=seed))\n uniform_initializer = 4 * tf.random.uniform(\n tf.shape(input=unconstrained_prior_sample),\n dtype=unconstrained_prior_sample.dtype,\n seed=seed) - 2\n if return_constrained:\n return parameter.bijector.forward(uniform_initializer)\n else:\n return uniform_initializer"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nbuilds a transformed - normal variational distribution over a parameter s support.", "response": "def _build_trainable_posterior(param, initial_loc_fn):\n \"\"\"Built a transformed-normal variational dist over a parameter's support.\"\"\"\n loc = tf.compat.v1.get_variable(\n param.name + '_loc',\n initializer=lambda: initial_loc_fn(param),\n dtype=param.prior.dtype,\n use_resource=True)\n scale = tf.nn.softplus(\n tf.compat.v1.get_variable(\n param.name + '_scale',\n initializer=lambda: -4 * tf.ones_like(initial_loc_fn(param)),\n dtype=param.prior.dtype,\n use_resource=True))\n\n q = tfd.Normal(loc=loc, scale=scale)\n\n # Ensure the `event_shape` of the variational distribution matches the\n # parameter.\n if (param.prior.event_shape.ndims is None\n or param.prior.event_shape.ndims > 0):\n q = tfd.Independent(\n q, reinterpreted_batch_ndims=param.prior.event_shape.ndims)\n\n # Transform to constrained parameter space.\n return tfd.TransformedDistribution(q, param.bijector)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nbuild a loss function for a variational inference in STS models.", "response": "def build_factored_variational_loss(model,\n observed_time_series,\n init_batch_shape=(),\n seed=None,\n name=None):\n \"\"\"Build a loss function for variational inference in STS models.\n\n Variational inference searches for the distribution within some family of\n approximate posteriors that minimizes a divergence between the approximate\n posterior `q(z)` and true posterior `p(z|observed_time_series)`. By converting\n inference to optimization, it's generally much faster than sampling-based\n inference algorithms such as HMC. The tradeoff is that the approximating\n family rarely contains the true posterior, so it may miss important aspects of\n posterior structure (in particular, dependence between variables) and should\n not be blindly trusted. Results may vary; it's generally wise to compare to\n HMC to evaluate whether inference quality is sufficient for your task at hand.\n\n This method constructs a loss function for variational inference using the\n Kullback-Liebler divergence `KL[q(z) || p(z|observed_time_series)]`, with an\n approximating family given by independent Normal distributions transformed to\n the appropriate parameter space for each parameter. Minimizing this loss (the\n negative ELBO) maximizes a lower bound on the log model evidence `-log\n p(observed_time_series)`. This is equivalent to the 'mean-field' method\n implemented in [1]. and is a standard approach. The resulting posterior\n approximations are unimodal; they will tend to underestimate posterior\n uncertainty when the true posterior contains multiple modes (the `KL[q||p]`\n divergence encourages choosing a single mode) or dependence between variables.\n\n Args:\n model: An instance of `StructuralTimeSeries` representing a\n time-series model. This represents a joint distribution over\n time-series and their parameters with batch shape `[b1, ..., bN]`.\n observed_time_series: `float` `Tensor` of shape\n `concat([sample_shape, model.batch_shape, [num_timesteps, 1]]) where\n `sample_shape` corresponds to i.i.d. observations, and the trailing `[1]`\n dimension may (optionally) be omitted if `num_timesteps > 1`. May\n optionally be an instance of `tfp.sts.MaskedTimeSeries`, which includes\n a mask `Tensor` to specify timesteps with missing observations.\n init_batch_shape: Batch shape (Python `tuple`, `list`, or `int`) of initial\n states to optimize in parallel.\n Default value: `()`. (i.e., just run a single optimization).\n seed: Python integer to seed the random number generator.\n name: Python `str` name prefixed to ops created by this function.\n Default value: `None` (i.e., 'build_factored_variational_loss').\n\n Returns:\n variational_loss: `float` `Tensor` of shape\n `concat([init_batch_shape, model.batch_shape])`, encoding a stochastic\n estimate of an upper bound on the negative model evidence `-log p(y)`.\n Minimizing this loss performs variational inference; the gap between the\n variational bound and the true (generally unknown) model evidence\n corresponds to the divergence `KL[q||p]` between the approximate and true\n posterior.\n variational_distributions: `collections.OrderedDict` giving\n the approximate posterior for each model parameter. The keys are\n Python `str` parameter names in order, corresponding to\n `[param.name for param in model.parameters]`. The values are\n `tfd.Distribution` instances with batch shape\n `concat([init_batch_shape, model.batch_shape])`; these will typically be\n of the form `tfd.TransformedDistribution(tfd.Normal(...),\n bijector=param.bijector)`.\n\n #### Examples\n\n Assume we've built a structural time-series model:\n\n ```python\n day_of_week = tfp.sts.Seasonal(\n num_seasons=7,\n observed_time_series=observed_time_series,\n name='day_of_week')\n local_linear_trend = tfp.sts.LocalLinearTrend(\n observed_time_series=observed_time_series,\n name='local_linear_trend')\n model = tfp.sts.Sum(components=[day_of_week, local_linear_trend],\n observed_time_series=observed_time_series)\n ```\n\n To run variational inference, we simply construct the loss and optimize\n it:\n\n ```python\n (variational_loss,\n variational_distributions) = tfp.sts.build_factored_variational_loss(\n model=model, observed_time_series=observed_time_series)\n\n train_op = tf.train.AdamOptimizer(0.1).minimize(variational_loss)\n with tf.Session() as sess:\n sess.run(tf.global_variables_initializer())\n\n for step in range(200):\n _, loss_ = sess.run((train_op, variational_loss))\n\n if step % 20 == 0:\n print(\"step {} loss {}\".format(step, loss_))\n\n posterior_samples_ = sess.run({\n param_name: q.sample(50)\n for param_name, q in variational_distributions.items()})\n ```\n\n As a more complex example, we might try to avoid local optima by optimizing\n from multiple initializations in parallel, and selecting the result with the\n lowest loss:\n\n ```python\n (variational_loss,\n variational_distributions) = tfp.sts.build_factored_variational_loss(\n model=model, observed_time_series=observed_time_series,\n init_batch_shape=[10])\n\n train_op = tf.train.AdamOptimizer(0.1).minimize(variational_loss)\n with tf.Session() as sess:\n sess.run(tf.global_variables_initializer())\n\n for step in range(200):\n _, loss_ = sess.run((train_op, variational_loss))\n\n if step % 20 == 0:\n print(\"step {} losses {}\".format(step, loss_))\n\n # Draw multiple samples to reduce Monte Carlo error in the optimized\n # variational bounds.\n avg_loss = np.mean(\n [sess.run(variational_loss) for _ in range(25)], axis=0)\n best_posterior_idx = np.argmin(avg_loss, axis=0).astype(np.int32)\n ```\n\n #### References\n\n [1]: Alp Kucukelbir, Dustin Tran, Rajesh Ranganath, Andrew Gelman, and\n David M. Blei. Automatic Differentiation Variational Inference. In\n _Journal of Machine Learning Research_, 2017.\n https://arxiv.org/abs/1603.00788\n\n \"\"\"\n\n with tf.compat.v1.name_scope(\n name, 'build_factored_variational_loss',\n values=[observed_time_series]) as name:\n seed = tfd.SeedStream(\n seed, salt='StructuralTimeSeries_build_factored_variational_loss')\n\n variational_distributions = collections.OrderedDict()\n variational_samples = []\n for param in model.parameters:\n def initial_loc_fn(param):\n return sample_uniform_initial_state(\n param, return_constrained=True,\n init_sample_shape=init_batch_shape,\n seed=seed())\n q = _build_trainable_posterior(param, initial_loc_fn=initial_loc_fn)\n variational_distributions[param.name] = q\n variational_samples.append(q.sample(seed=seed()))\n\n # Multiple initializations (similar to HMC chains) manifest as an extra\n # param batch dimension, so we need to add corresponding batch dimension(s)\n # to `observed_time_series`.\n observed_time_series = sts_util.pad_batch_dimension_for_multiple_chains(\n observed_time_series, model, chain_batch_shape=init_batch_shape)\n\n # Construct the variational bound.\n log_prob_fn = model.joint_log_prob(observed_time_series)\n expected_log_joint = log_prob_fn(*variational_samples)\n entropy = tf.reduce_sum(\n input_tensor=[\n -q.log_prob(sample) for (q, sample) in zip(\n variational_distributions.values(), variational_samples)\n ],\n axis=0)\n variational_loss = -(expected_log_joint + entropy) # -ELBO\n\n return variational_loss, variational_distributions"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _minimize_in_graph(build_loss_fn, num_steps=200, optimizer=None):\n optimizer = tf.compat.v1.train.AdamOptimizer(\n 0.1) if optimizer is None else optimizer\n\n def train_loop_body(step):\n train_op = optimizer.minimize(\n build_loss_fn if tf.executing_eagerly() else build_loss_fn())\n return tf.tuple(tensors=[tf.add(step, 1)], control_inputs=[train_op])\n\n minimize_op = tf.compat.v1.while_loop(\n cond=lambda step: step < num_steps,\n body=train_loop_body,\n loop_vars=[tf.constant(0)],\n return_same_structure=True)[0] # Always return a single op.\n return minimize_op", "response": "Run an optimizer within the graph to minimize a loss function."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nfits a model to the posterior distribution using the Markov chain Monte Carlo method.", "response": "def fit_with_hmc(model,\n observed_time_series,\n num_results=100,\n num_warmup_steps=50,\n num_leapfrog_steps=15,\n initial_state=None,\n initial_step_size=None,\n chain_batch_shape=(),\n num_variational_steps=150,\n variational_optimizer=None,\n seed=None,\n name=None):\n \"\"\"Draw posterior samples using Hamiltonian Monte Carlo (HMC).\n\n Markov chain Monte Carlo (MCMC) methods are considered the gold standard of\n Bayesian inference; under suitable conditions and in the limit of infinitely\n many draws they generate samples from the true posterior distribution. HMC [1]\n uses gradients of the model's log-density function to propose samples,\n allowing it to exploit posterior geometry. However, it is computationally more\n expensive than variational inference and relatively sensitive to tuning.\n\n This method attempts to provide a sensible default approach for fitting\n StructuralTimeSeries models using HMC. It first runs variational inference as\n a fast posterior approximation, and initializes the HMC sampler from the\n variational posterior, using the posterior standard deviations to set\n per-variable step sizes (equivalently, a diagonal mass matrix). During the\n warmup phase, it adapts the step size to target an acceptance rate of 0.75,\n which is thought to be in the desirable range for optimal mixing [2].\n\n\n Args:\n model: An instance of `StructuralTimeSeries` representing a\n time-series model. This represents a joint distribution over\n time-series and their parameters with batch shape `[b1, ..., bN]`.\n observed_time_series: `float` `Tensor` of shape\n `concat([sample_shape, model.batch_shape, [num_timesteps, 1]]) where\n `sample_shape` corresponds to i.i.d. observations, and the trailing `[1]`\n dimension may (optionally) be omitted if `num_timesteps > 1`. May\n optionally be an instance of `tfp.sts.MaskedTimeSeries`, which includes\n a mask `Tensor` to specify timesteps with missing observations.\n num_results: Integer number of Markov chain draws.\n Default value: `100`.\n num_warmup_steps: Integer number of steps to take before starting to\n collect results. The warmup steps are also used to adapt the step size\n towards a target acceptance rate of 0.75.\n Default value: `50`.\n num_leapfrog_steps: Integer number of steps to run the leapfrog integrator\n for. Total progress per HMC step is roughly proportional to\n `step_size * num_leapfrog_steps`.\n Default value: `15`.\n initial_state: Optional Python `list` of `Tensor`s, one for each model\n parameter, representing the initial state(s) of the Markov chain(s). These\n should have shape `concat([chain_batch_shape, param.prior.batch_shape,\n param.prior.event_shape])`. If `None`, the initial state is set\n automatically using a sample from a variational posterior.\n Default value: `None`.\n initial_step_size: Python `list` of `Tensor`s, one for each model parameter,\n representing the step size for the leapfrog integrator. Must\n broadcast with the shape of `initial_state`. Larger step sizes lead to\n faster progress, but too-large step sizes make rejection exponentially\n more likely. If `None`, the step size is set automatically using the\n standard deviation of a variational posterior.\n Default value: `None`.\n chain_batch_shape: Batch shape (Python `tuple`, `list`, or `int`) of chains\n to run in parallel.\n Default value: `[]` (i.e., a single chain).\n num_variational_steps: Python `int` number of steps to run the variational\n optimization to determine the initial state and step sizes.\n Default value: `150`.\n variational_optimizer: Optional `tf.train.Optimizer` instance to use in\n the variational optimization. If `None`, defaults to\n `tf.train.AdamOptimizer(0.1)`.\n Default value: `None`.\n seed: Python integer to seed the random number generator.\n name: Python `str` name prefixed to ops created by this function.\n Default value: `None` (i.e., 'fit_with_hmc').\n\n Returns:\n samples: Python `list` of `Tensors` representing posterior samples of model\n parameters, with shapes `[concat([[num_results], chain_batch_shape,\n param.prior.batch_shape, param.prior.event_shape]) for param in\n model.parameters]`.\n kernel_results: A (possibly nested) `tuple`, `namedtuple` or `list` of\n `Tensor`s representing internal calculations made within the HMC sampler.\n\n #### Examples\n\n Assume we've built a structural time-series model:\n\n ```python\n day_of_week = tfp.sts.Seasonal(\n num_seasons=7,\n observed_time_series=observed_time_series,\n name='day_of_week')\n local_linear_trend = tfp.sts.LocalLinearTrend(\n observed_time_series=observed_time_series,\n name='local_linear_trend')\n model = tfp.sts.Sum(components=[day_of_week, local_linear_trend],\n observed_time_series=observed_time_series)\n ```\n\n To draw posterior samples using HMC under default settings:\n\n ```python\n samples, kernel_results = tfp.sts.fit_with_hmc(model, observed_time_series)\n\n with tf.Session() as sess:\n sess.run(tf.global_variables_initializer())\n samples_, kernel_results_ = sess.run((samples, kernel_results))\n\n print(\"acceptance rate: {}\".format(\n np.mean(kernel_results_.inner_results.is_accepted, axis=0)))\n print(\"posterior means: {}\".format(\n {param.name: np.mean(param_draws, axis=0)\n for (param, param_draws) in zip(model.parameters, samples_)}))\n ```\n\n We can also run multiple chains. This may help diagnose convergence issues\n and allows us to exploit vectorization to draw samples more quickly, although\n warmup still requires the same number of sequential steps.\n\n ```python\n from matplotlib import pylab as plt\n\n samples, kernel_results = tfp.sts.fit_with_hmc(\n model, observed_time_series, chain_batch_shape=[10])\n\n with tf.Session() as sess:\n sess.run(tf.global_variables_initializer())\n samples_, kernel_results_ = sess.run((samples, kernel_results))\n\n print(\"acceptance rate: {}\".format(\n np.mean(kernel_results_.inner_results.inner_results.is_accepted, axis=0)))\n\n # Plot the sampled traces for each parameter. If the chains have mixed, their\n # traces should all cover the same region of state space, frequently crossing\n # over each other.\n for (param, param_draws) in zip(model.parameters, samples_):\n if param.prior.event_shape.ndims > 0:\n print(\"Only plotting traces for scalar parameters, skipping {}\".format(\n param.name))\n continue\n plt.figure(figsize=[10, 4])\n plt.title(param.name)\n plt.plot(param_draws)\n plt.ylabel(param.name)\n plt.xlabel(\"HMC step\")\n\n # Combining the samples from multiple chains into a single dimension allows\n # us to easily pass sampled parameters to downstream forecasting methods.\n combined_samples_ = [np.reshape(param_draws,\n [-1] + list(param_draws.shape[2:]))\n for param_draws in samples_]\n ```\n\n For greater flexibility, you may prefer to implement your own sampler using\n the TensorFlow Probability primitives in `tfp.mcmc`. The following recipe\n constructs a basic HMC sampler, using a `TransformedTransitionKernel` to\n incorporate constraints on the parameter space.\n\n ```python\n transformed_hmc_kernel = mcmc.TransformedTransitionKernel(\n inner_kernel=mcmc.SimpleStepSizeAdaptation(\n inner_kernel=mcmc.HamiltonianMonteCarlo(\n target_log_prob_fn=model.joint_log_prob(observed_time_series),\n step_size=step_size,\n num_leapfrog_steps=num_leapfrog_steps,\n state_gradients_are_stopped=True,\n seed=seed),\n num_adaptation_steps = int(0.8 * num_warmup_steps)),\n bijector=[param.bijector for param in model.parameters])\n\n # Initialize from a Uniform[-2, 2] distribution in unconstrained space.\n initial_state = [tfp.sts.sample_uniform_initial_state(\n param, return_constrained=True) for param in model.parameters]\n\n samples, kernel_results = tfp.mcmc.sample_chain(\n kernel=transformed_hmc_kernel,\n num_results=num_results,\n current_state=initial_state,\n num_burnin_steps=num_warmup_steps)\n ```\n\n #### References\n\n [1]: Radford Neal. MCMC Using Hamiltonian Dynamics. _Handbook of Markov Chain\n Monte Carlo_, 2011. https://arxiv.org/abs/1206.1901\n [2] M.J. Betancourt, Simon Byrne, and Mark Girolami. Optimizing The\n Integrator Step Size for Hamiltonian Monte Carlo.\n https://arxiv.org/abs/1411.6669\n\n \"\"\"\n with tf.compat.v1.name_scope(\n name, 'fit_with_hmc', values=[observed_time_series]) as name:\n seed = tfd.SeedStream(seed, salt='StructuralTimeSeries_fit_with_hmc')\n\n # Initialize state and step sizes from a variational posterior if not\n # specified.\n if initial_step_size is None or initial_state is None:\n\n # To avoid threading variational distributions through the training\n # while loop, we build our own copy here. `make_template` ensures\n # that our variational distributions share the optimized parameters.\n def make_variational():\n return build_factored_variational_loss(\n model, observed_time_series,\n init_batch_shape=chain_batch_shape, seed=seed())\n\n make_variational = tf.compat.v1.make_template('make_variational',\n make_variational)\n _, variational_distributions = make_variational()\n minimize_op = _minimize_in_graph(\n build_loss_fn=lambda: make_variational()[0], # return just the loss.\n num_steps=num_variational_steps,\n optimizer=variational_optimizer)\n\n with tf.control_dependencies([minimize_op]):\n if initial_state is None:\n initial_state = [tf.stop_gradient(d.sample())\n for d in variational_distributions.values()]\n\n # Set step sizes using the unconstrained variational distribution.\n if initial_step_size is None:\n initial_step_size = [\n transformed_q.distribution.stddev()\n for transformed_q in variational_distributions.values()]\n\n # Multiple chains manifest as an extra param batch dimension, so we need to\n # add a corresponding batch dimension to `observed_time_series`.\n observed_time_series = sts_util.pad_batch_dimension_for_multiple_chains(\n observed_time_series, model, chain_batch_shape=chain_batch_shape)\n\n # Run HMC to sample from the posterior on parameters.\n samples, kernel_results = mcmc.sample_chain(\n num_results=num_results,\n current_state=initial_state,\n num_burnin_steps=num_warmup_steps,\n kernel=mcmc.SimpleStepSizeAdaptation(\n inner_kernel=mcmc.TransformedTransitionKernel(\n inner_kernel=mcmc.HamiltonianMonteCarlo(\n target_log_prob_fn=model.joint_log_prob(\n observed_time_series),\n step_size=initial_step_size,\n num_leapfrog_steps=num_leapfrog_steps,\n state_gradients_are_stopped=True,\n seed=seed()),\n bijector=[param.bijector for param in model.parameters]),\n num_adaptation_steps=int(num_warmup_steps * 0.8),\n adaptation_rate=tf.convert_to_tensor(\n value=0.1, dtype=initial_state[0].dtype)),\n parallel_iterations=1 if seed is not None else 10)\n\n return samples, kernel_results"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef moments_of_masked_time_series(time_series_tensor, broadcast_mask):\n num_unmasked_entries = tf.cast(\n tf.reduce_sum(input_tensor=tf.cast(~broadcast_mask, tf.int32), axis=-1),\n time_series_tensor.dtype)\n\n # Manually compute mean and variance, excluding masked entries.\n mean = (tf.reduce_sum(input_tensor=tf.where(\n broadcast_mask,\n tf.zeros_like(time_series_tensor),\n time_series_tensor), axis=-1) / num_unmasked_entries)\n variance = (tf.reduce_sum(input_tensor=tf.where(\n broadcast_mask,\n tf.zeros_like(time_series_tensor),\n (time_series_tensor - mean[..., tf.newaxis]) ** 2), axis=-1)\n / num_unmasked_entries)\n return mean, variance", "response": "Compute mean and variance accounting for a mask."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nreturns the initial value of the first unmasked entry of each time series.", "response": "def initial_value_of_masked_time_series(time_series_tensor, broadcast_mask):\n \"\"\"Get the first unmasked entry of each time series in the batch.\n\n Args:\n time_series_tensor: float `Tensor` of shape [..., num_timesteps].\n broadcast_mask: bool `Tensor` of same shape as `time_series`.\n \"\"\"\n\n num_timesteps = tf.shape(input=time_series_tensor)[-1]\n\n # Compute the index of the first unmasked entry for each series in the batch.\n unmasked_negindices = (\n tf.cast(~broadcast_mask, tf.int32) *\n tf.range(num_timesteps, 0, -1))\n first_unmasked_indices = num_timesteps - tf.reduce_max(\n input_tensor=unmasked_negindices, axis=-1)\n\n if first_unmasked_indices.shape.ndims is None:\n raise NotImplementedError(\n 'Cannot compute initial values of a masked time series with'\n 'dynamic rank.') # `batch_gather` requires static rank\n\n # Extract the initial value for each series in the batch.\n return tf.squeeze(tf.compat.v1.batch_gather(\n params=time_series_tensor,\n indices=first_unmasked_indices[..., tf.newaxis]), axis=-1)"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nget broadcast batch shape from distributions.", "response": "def broadcast_batch_shape(distributions):\n \"\"\"Get broadcast batch shape from distributions, statically if possible.\"\"\"\n\n # Static case\n batch_shape = distributions[0].batch_shape\n for distribution in distributions:\n batch_shape = tf.broadcast_static_shape(batch_shape,\n distribution.batch_shape)\n if batch_shape.is_fully_defined():\n return batch_shape.as_list()\n\n # Fallback on dynamic.\n batch_shape = distributions[0].batch_shape_tensor()\n for distribution in distributions:\n batch_shape = tf.broadcast_dynamic_shape(batch_shape,\n distribution.batch_shape_tensor())\n\n return tf.convert_to_tensor(value=batch_shape)"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef pad_batch_dimension_for_multiple_chains(\n observed_time_series, model, chain_batch_shape):\n \"\"\"\"Expand the observed time series with extra batch dimension(s).\"\"\"\n # Running with multiple chains introduces an extra batch dimension. In\n # general we also need to pad the observed time series with a matching batch\n # dimension.\n #\n # For example, suppose our model has batch shape [3, 4] and\n # the observed time series has shape `concat([[5], [3, 4], [100])`,\n # corresponding to `sample_shape`, `batch_shape`, and `num_timesteps`\n # respectively. The model will produce distributions with batch shape\n # `concat([chain_batch_shape, [3, 4]])`, so we pad `observed_time_series` to\n # have matching shape `[5, 1, 3, 4, 100]`, where the added `1` dimension\n # between the sample and batch shapes will broadcast to `chain_batch_shape`.\n\n [ # Extract mask and guarantee `event_ndims=2`.\n observed_time_series,\n is_missing\n ] = canonicalize_observed_time_series_with_mask(observed_time_series)\n\n event_ndims = 2 # event_shape = [num_timesteps, observation_size=1]\n\n model_batch_ndims = (\n model.batch_shape.ndims if model.batch_shape.ndims is not None else\n tf.shape(input=model.batch_shape_tensor())[0])\n\n # Compute ndims from chain_batch_shape.\n chain_batch_shape = tf.convert_to_tensor(\n value=chain_batch_shape, name='chain_batch_shape', dtype=tf.int32)\n if not chain_batch_shape.shape.is_fully_defined():\n raise ValueError('Batch shape must have static rank. (given: {})'.format(\n chain_batch_shape))\n if chain_batch_shape.shape.ndims == 0: # expand int `k` to `[k]`.\n chain_batch_shape = chain_batch_shape[tf.newaxis]\n chain_batch_ndims = tf.compat.dimension_value(chain_batch_shape.shape[0])\n\n def do_padding(observed_time_series_tensor):\n current_sample_shape = tf.shape(\n input=observed_time_series_tensor)[:-(model_batch_ndims + event_ndims)]\n current_batch_and_event_shape = tf.shape(\n input=observed_time_series_tensor)[-(model_batch_ndims + event_ndims):]\n return tf.reshape(\n tensor=observed_time_series_tensor,\n shape=tf.concat([\n current_sample_shape,\n tf.ones([chain_batch_ndims], dtype=tf.int32),\n current_batch_and_event_shape], axis=0))\n\n # Padding is only needed if the observed time series has sample shape.\n observed_time_series = prefer_static.cond(\n (dist_util.prefer_static_rank(observed_time_series) >\n model_batch_ndims + event_ndims),\n lambda: do_padding(observed_time_series),\n lambda: observed_time_series)\n\n if is_missing is not None:\n is_missing = prefer_static.cond(\n (dist_util.prefer_static_rank(is_missing) >\n model_batch_ndims + event_ndims),\n lambda: do_padding(is_missing),\n lambda: is_missing)\n return missing_values_util.MaskedTimeSeries(observed_time_series,\n is_missing=is_missing)\n\n return observed_time_series", "response": "Expand the observed time series with extra batch dimension ( s )."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef factored_joint_mvn(distributions):\n\n graph_parents = [tensor for distribution in distributions\n for tensor in distribution._graph_parents] # pylint: disable=protected-access\n with tf.compat.v1.name_scope('factored_joint_mvn', values=graph_parents):\n\n # We explicitly broadcast the `locs` so that we can concatenate them.\n # We don't have direct numerical access to the `scales`, which are arbitrary\n # linear operators, but `LinearOperatorBlockDiag` appears to do the right\n # thing without further intervention.\n dtype = tf.debugging.assert_same_float_dtype(distributions)\n broadcast_ones = tf.ones(broadcast_batch_shape(distributions),\n dtype=dtype)[..., tf.newaxis]\n return MultivariateNormalLinearOperator(\n loc=tf.concat([mvn.mean() * broadcast_ones for mvn in distributions],\n axis=-1),\n scale=tfl.LinearOperatorBlockDiag([mvn.scale for mvn in distributions],\n is_square=True))", "response": "Combine a list of multivariate normal distributions into a factored multivariate normal linear operator."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nattempts to sum MultivariateNormal distributions. The sum of (multivariate) normal random variables is itself (multivariate) normal, with mean given by the sum of means and (co)variance given by the sum of (co)variances. This method exploits this fact to compute the sum of a list of `tfd.MultivariateNormalDiag` objects. It may in the future be extended to support summation of other forms of (Multivariate)Normal distributions. Args: distributions: Python `iterable` of `tfd.MultivariateNormalDiag` distribution instances. These must all have the same event shape, and broadcast to a consistent batch shape. Returns: sum_distribution: A `tfd.MultivariateNormalDiag` instance with mean equal to the sum of input means and covariance equal to the sum of input covariances.", "response": "def sum_mvns(distributions):\n \"\"\"Attempt to sum MultivariateNormal distributions.\n\n The sum of (multivariate) normal random variables is itself (multivariate)\n normal, with mean given by the sum of means and (co)variance given by the\n sum of (co)variances. This method exploits this fact to compute the\n sum of a list of `tfd.MultivariateNormalDiag` objects.\n\n It may in the future be extended to support summation of other forms of\n (Multivariate)Normal distributions.\n\n Args:\n distributions: Python `iterable` of `tfd.MultivariateNormalDiag`\n distribution instances. These must all have the same event\n shape, and broadcast to a consistent batch shape.\n\n Returns:\n sum_distribution: A `tfd.MultivariateNormalDiag` instance with mean\n equal to the sum of input means and covariance equal to the sum of\n input covariances.\n \"\"\"\n\n graph_parents = [tensor for distribution in distributions\n for tensor in distribution._graph_parents] # pylint: disable=protected-access\n with tf.compat.v1.name_scope('sum_mvns', values=graph_parents):\n if all([isinstance(mvn, tfd.MultivariateNormalDiag)\n for mvn in distributions]):\n return tfd.MultivariateNormalDiag(\n loc=sum([mvn.mean() for mvn in distributions]),\n scale_diag=tf.sqrt(sum([\n mvn.scale.diag**2 for mvn in distributions])))\n else:\n raise NotImplementedError(\n 'Sums of distributions other than MultivariateNormalDiag are not '\n 'currently implemented. (given: {})'.format(distributions))"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef empirical_statistics(observed_time_series):\n\n with tf.compat.v1.name_scope(\n 'empirical_statistics', values=[observed_time_series]):\n\n [\n observed_time_series,\n mask\n ] = canonicalize_observed_time_series_with_mask(observed_time_series)\n\n squeezed_series = observed_time_series[..., 0]\n if mask is None:\n observed_mean, observed_variance = tf.nn.moments(\n x=squeezed_series, axes=-1)\n observed_initial = squeezed_series[..., 0]\n else:\n broadcast_mask = tf.broadcast_to(tf.cast(mask, tf.bool),\n tf.shape(input=squeezed_series))\n observed_mean, observed_variance = (\n missing_values_util.moments_of_masked_time_series(\n squeezed_series, broadcast_mask=broadcast_mask))\n try:\n observed_initial = (\n missing_values_util.initial_value_of_masked_time_series(\n squeezed_series, broadcast_mask=broadcast_mask))\n except NotImplementedError:\n tf.compat.v1.logging.warn(\n 'Cannot compute initial values for a masked time series'\n 'with dynamic shape; using the mean instead. This will'\n 'affect heuristic priors and may change the results of'\n 'inference.')\n observed_initial = observed_mean\n\n observed_stddev = tf.sqrt(observed_variance)\n observed_initial_centered = observed_initial - observed_mean\n return observed_mean, observed_stddev, observed_initial_centered", "response": "Compute statistics of a provided time series as heuristic initialization."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef _maybe_expand_trailing_dim(observed_time_series_tensor):\n\n with tf.compat.v1.name_scope(\n 'maybe_expand_trailing_dim', values=[observed_time_series_tensor]):\n if (observed_time_series_tensor.shape.ndims is not None and\n tf.compat.dimension_value(\n observed_time_series_tensor.shape[-1]) is not None):\n expanded_time_series = (\n observed_time_series_tensor\n if observed_time_series_tensor.shape[-1] == 1\n else observed_time_series_tensor[..., tf.newaxis])\n else:\n expanded_time_series = tf.cond(\n pred=tf.equal(tf.shape(input=observed_time_series_tensor)[-1], 1),\n true_fn=lambda: observed_time_series_tensor,\n false_fn=lambda: observed_time_series_tensor[..., tf.newaxis])\n return expanded_time_series", "response": "Ensures that the observed_time_series_tensor has a trailing dimension of size 1."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef canonicalize_observed_time_series_with_mask(\n maybe_masked_observed_time_series):\n \"\"\"Extract a Tensor with canonical shape and optional mask.\n\n Args:\n maybe_masked_observed_time_series: a `Tensor`-like object with shape\n `[..., num_timesteps]` or `[..., num_timesteps, 1]`, or a\n `tfp.sts.MaskedTimeSeries` containing such an object.\n Returns:\n masked_time_series: a `tfp.sts.MaskedTimeSeries` namedtuple, in which\n the `observed_time_series` is converted to `Tensor` with canonical shape\n `[..., num_timesteps, 1]`, and `is_missing` is either `None` or a boolean\n `Tensor`.\n \"\"\"\n\n with tf.compat.v1.name_scope('canonicalize_observed_time_series_with_mask'):\n if hasattr(maybe_masked_observed_time_series, 'is_missing'):\n observed_time_series = (\n maybe_masked_observed_time_series.time_series)\n is_missing = maybe_masked_observed_time_series.is_missing\n else:\n observed_time_series = maybe_masked_observed_time_series\n is_missing = None\n\n observed_time_series = tf.convert_to_tensor(value=observed_time_series,\n name='observed_time_series')\n observed_time_series = _maybe_expand_trailing_dim(observed_time_series)\n\n if is_missing is not None:\n is_missing = tf.convert_to_tensor(\n value=is_missing, name='is_missing', dtype_hint=tf.bool)\n\n return missing_values_util.MaskedTimeSeries(observed_time_series,\n is_missing=is_missing)", "response": "Returns a Tensor with canonical shape and optional mask."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef mix_over_posterior_draws(means, variances):\n # The inputs `means`, `variances` have shape\n # `concat([\n # [num_posterior_draws],\n # sample_shape,\n # batch_shape,\n # [num_timesteps]])`\n # Because MixtureSameFamily mixes over the rightmost batch dimension,\n # we need to move the `num_posterior_draws` dimension to be rightmost\n # in the batch shape. This requires use of `Independent` (to preserve\n # `num_timesteps` as part of the event shape) and `move_dimension`.\n # TODO(b/120245392): enhance `MixtureSameFamily` to reduce along an\n # arbitrary axis, and eliminate `move_dimension` calls here.\n\n with tf.compat.v1.name_scope(\n 'mix_over_posterior_draws', values=[means, variances]):\n num_posterior_draws = dist_util.prefer_static_value(\n tf.shape(input=means))[0]\n\n component_observations = tfd.Independent(\n distribution=tfd.Normal(\n loc=dist_util.move_dimension(means, 0, -2),\n scale=tf.sqrt(dist_util.move_dimension(variances, 0, -2))),\n reinterpreted_batch_ndims=1)\n\n return tfd.MixtureSameFamily(\n mixture_distribution=tfd.Categorical(\n logits=tf.zeros([num_posterior_draws],\n dtype=component_observations.dtype)),\n components_distribution=component_observations)", "response": "Construct a predictive normal distribution that mixes over posterior draws."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef _kl_uniform_uniform(a, b, name=None):\n with tf.name_scope(name or \"kl_uniform_uniform\"):\n # Consistent with\n # http://www.mast.queensu.ca/~communications/Papers/gil-msc11.pdf, page 60\n # Watch out for the change in conventions--they use 'a' and 'b' to refer to\n # lower and upper bounds respectively there.\n final_batch_shape = distribution_util.get_broadcast_shape(\n a.low, b.low, a.high, b.high)\n dtype = dtype_util.common_dtype(\n [a.low, a.high, b.low, b.high], tf.float32)\n return tf.where((b.low <= a.low) & (a.high <= b.high),\n tf.math.log(b.high - b.low) - tf.math.log(a.high - a.low),\n tf.broadcast_to(\n dtype_util.as_numpy_dtype(dtype)(np.inf),\n final_batch_shape))", "response": "Calculate the batched KL divergence of a and b uniform distribution."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef _make_summary_statistic(attr):\n def _fn(self):\n if any(self._dist_fn_args): # pylint: disable=protected-access\n raise ValueError(\n 'Can only compute ' + attr + ' when all distributions are '\n 'independent; {}'.format(self.model))\n return self._unflatten(getattr(d(), attr)() for d in self._dist_fn_wrapped) # pylint: disable=protected-access\n return _fn", "response": "Factory for making summary statistics eg mean mode stddev."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nunifying the call signature of dist_fn with all previous nodes.", "response": "def _unify_call_signature(i, dist_fn):\n \"\"\"Creates `dist_fn_wrapped` which calls `dist_fn` with all prev nodes.\n\n Args:\n i: Python `int` corresponding to position in topologically sorted DAG.\n dist_fn: Python `callable` which takes a subset of previously constructed\n distributions (in reverse order) and produces a new distribution instance.\n\n Returns:\n dist_fn_wrapped: Python `callable` which takes all previous distributions\n (in non reverse order) and produces a new distribution instance.\n args: `tuple` of `str` representing the arg names of `dist_fn` (and in non\n wrapped, \"natural\" order). `None` is returned only if the input is not a\n `callable`.\n \"\"\"\n if distribution_util.is_distribution_instance(dist_fn):\n return (lambda *_: dist_fn), None\n\n if not callable(dist_fn):\n raise TypeError('{} must be either `tfd.Distribution`-like or '\n '`callable`.'.format(dist_fn))\n\n args = _get_required_args(dist_fn)\n if not args:\n return (lambda *_: dist_fn()), ()\n\n @functools.wraps(dist_fn)\n def dist_fn_wrapped(*xs):\n \"\"\"Calls `dist_fn` with reversed and truncated args.\"\"\"\n if i != len(xs):\n raise ValueError(\n 'Internal Error: Unexpected number of inputs provided to {}-th '\n 'distribution maker (dist_fn: {}, expected: {}, saw: {}).'.format(\n i, dist_fn, i, len(xs)))\n if len(xs) < len(args):\n raise ValueError(\n 'Internal Error: Too few inputs provided to {}-th distribution maker '\n '(dist_fn: {}, expected: {}, saw: {}).'.format(\n i, dist_fn, len(args), len(xs)))\n return dist_fn(*reversed(xs[-len(args):]))\n return dist_fn_wrapped, args"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nuse arg names to resolve distribution names.", "response": "def _resolve_distribution_names(dist_fn_args, dist_names, leaf_name):\n \"\"\"Uses arg names to resolve distribution names.\"\"\"\n if dist_names is None:\n dist_names = []\n else:\n dist_names = dist_names.copy()\n n = len(dist_fn_args)\n dist_names.extend([None]*(n - len(dist_names)))\n for i_, args in enumerate(reversed(dist_fn_args)):\n if not args:\n continue # There's no args to analyze.\n i = n - i_ - 1\n for j, arg_name in enumerate(args):\n dist_names[i - j - 1] = arg_name\n j = 0\n for i_ in range(len(dist_names)):\n i = n - i_ - 1\n if dist_names[i] is None:\n dist_names[i] = leaf_name if j == 0 else leaf_name + str(j)\n j += 1\n return tuple(dist_names)"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nreturns the distribution s required args.", "response": "def _get_required_args(fn):\n \"\"\"Returns the distribution's required args.\"\"\"\n argspec = tf_inspect.getfullargspec(fn)\n args = argspec.args\n if tf_inspect.isclass(fn):\n args = args[1:] # Remove the `self` arg.\n if argspec.defaults:\n # Remove the args which have defaults. By convention we only feed\n # *required args*. This means some distributions must always be wrapped\n # with a `lambda`, e.g., `lambda logits: tfd.Bernoulli(logits=logits)`\n # or `lambda probs: tfd.Bernoulli(probs=probs)`.\n args = args[:-len(argspec.defaults)]\n return tuple(args)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\ncomputing the KL divergence between two joint distributions.", "response": "def _kl_joint_joint(d0, d1, name=None):\n \"\"\"Calculate the KL divergence between two `JointDistributionSequential`s.\n\n Args:\n d0: instance of a `JointDistributionSequential` object.\n d1: instance of a `JointDistributionSequential` object.\n name: (optional) Name to use for created operations.\n Default value: `\"kl_joint_joint\"`.\n\n Returns:\n kl_joint_joint: `Tensor` The sum of KL divergences between elemental\n distributions of two joint distributions.\n\n Raises:\n ValueError: when joint distributions have a different number of elemental\n distributions.\n ValueError: when either joint distribution has a distribution with dynamic\n dependency, i.e., when either joint distribution is not a collection of\n independent distributions.\n \"\"\"\n if len(d0._dist_fn_wrapped) != len(d1._dist_fn_wrapped): # pylint: disable=protected-access\n raise ValueError(\n 'Can only compute KL divergence between when each has the'\n 'same number of component distributions.')\n if (not all(a is None for a in d0._dist_fn_args) or # pylint: disable=protected-access\n not all(a is None for a in d1._dist_fn_args)): # pylint: disable=protected-access\n raise ValueError(\n 'Can only compute KL divergence when all distributions are '\n 'independent.')\n with tf.name_scope(name or 'kl_jointseq_jointseq'):\n return sum(kullback_leibler.kl_divergence(d0_(), d1_())\n for d0_, d1_ in zip(d0._dist_fn_wrapped, d1._dist_fn_wrapped))"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\ncreating dist_fn dist_fn_wrapped dist_fn_args.", "response": "def _build(self, model):\n \"\"\"Creates `dist_fn`, `dist_fn_wrapped`, `dist_fn_args`.\"\"\"\n if not isinstance(model, collections.Sequence):\n raise TypeError('`model` must be `list`-like (saw: {}).'.format(\n type(model).__name__))\n self._dist_fn = model\n self._dist_fn_wrapped, self._dist_fn_args = zip(*[\n _unify_call_signature(i, dist_fn)\n for i, dist_fn in enumerate(model)])"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nresolves the graph of the given distribution_names and leaf_name.", "response": "def _resolve_graph(self, distribution_names=None, leaf_name='x'):\n \"\"\"Creates a `tuple` of `tuple`s of dependencies.\n\n This function is **experimental**. That said, we encourage its use\n and ask that you report problems to `tfprobability@tensorflow.org`.\n\n Args:\n distribution_names: `list` of `str` or `None` names corresponding to each\n of `model` elements. (`None`s are expanding into the\n appropriate `str`.)\n leaf_name: `str` used when no maker depends on a particular\n `model` element.\n\n Returns:\n graph: `tuple` of `(str tuple)` pairs representing the name of each\n distribution (maker) and the names of its dependencies.\n\n #### Example\n\n ```python\n d = tfd.JointDistributionSequential([\n tfd.Independent(tfd.Exponential(rate=[100, 120]), 1),\n lambda e: tfd.Gamma(concentration=e[..., 0], rate=e[..., 1]),\n tfd.Normal(loc=0, scale=2.),\n lambda n, g: tfd.Normal(loc=n, scale=g),\n ])\n d._resolve_graph()\n # ==> (\n # ('e', ()),\n # ('g', ('e',)),\n # ('n', ()),\n # ('x', ('n', 'g')),\n # )\n ```\n\n \"\"\"\n # This function additionally depends on:\n # self._dist_fn_args\n # self._dist_fn_wrapped\n # TODO(b/129008220): Robustify this procedure. Eg, handle collisions better,\n # ignore args prefixed with `_`.\n if distribution_names is None or any(self._dist_fn_args):\n distribution_names = _resolve_distribution_names(\n self._dist_fn_args, distribution_names, leaf_name)\n if len(set(distribution_names)) != len(distribution_names):\n raise ValueError('Distribution names must be unique: {}'.format(\n distribution_names))\n if len(distribution_names) != len(self._dist_fn_wrapped):\n raise ValueError('Distribution names must be 1:1 with `rvs`.')\n return tuple(zip(distribution_names,\n tuple(() if a is None else a for a in self._dist_fn_args)))"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\ncompute the Shannon entropy in nats.", "response": "def _entropy(self):\n \"\"\"Shannon entropy in nats.\"\"\"\n if any(self._dist_fn_args):\n raise ValueError(\n 'Can only compute entropy when all distributions are independent.')\n return sum(joint_distribution_lib.maybe_check_wont_broadcast(\n (d().entropy() for d in self._dist_fn_wrapped),\n self.validate_args))"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _extend_support_with_default_value(self, x, f, default_value):\n with tf.name_scope(\"extend_support_with_default_value\"):\n x = tf.convert_to_tensor(value=x, dtype=self.dtype, name=\"x\")\n loc = self.loc + tf.zeros_like(self.scale) + tf.zeros_like(x)\n x = x + tf.zeros_like(loc)\n # Substitute out-of-support values in x with values that are in the\n # support of the distribution before applying f.\n y = f(tf.where(x < loc, self._inv_z(0.5) + tf.zeros_like(x), x))\n if default_value == 0.:\n default_value = tf.zeros_like(y)\n elif default_value == 1.:\n default_value = tf.ones_like(y)\n else:\n default_value = tf.fill(\n dims=tf.shape(input=y),\n value=dtype_util.as_numpy_dtype(self.dtype)(default_value))\n return tf.where(x < loc, default_value, y)", "response": "Extends the support of this distribution with a default value."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nprocess input args to meet list - like assumptions.", "response": "def _prepare_args(log_likelihood_fn, state,\n log_likelihood=None, description='log_likelihood'):\n \"\"\"Processes input args to meet list-like assumptions.\"\"\"\n state_parts = list(state) if mcmc_util.is_list_like(state) else [state]\n state_parts = [tf.convert_to_tensor(s, name='current_state')\n for s in state_parts]\n\n log_likelihood = _maybe_call_fn(\n log_likelihood_fn,\n state_parts,\n log_likelihood,\n description)\n return [state_parts, log_likelihood]"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef one_step(self, current_state, previous_kernel_results):\n with tf.compat.v1.name_scope(\n name=mcmc_util.make_name(self.name, 'elliptical_slice', 'one_step'),\n values=[self._seed_stream,\n current_state,\n previous_kernel_results.log_likelihood]):\n with tf.compat.v1.name_scope('initialize'):\n [\n init_state_parts,\n init_log_likelihood\n ] = _prepare_args(\n self.log_likelihood_fn,\n current_state,\n previous_kernel_results.log_likelihood)\n\n normal_samples = self.normal_sampler_fn(self._seed_stream()) # pylint: disable=not-callable\n normal_samples = list(normal_samples) if mcmc_util.is_list_like(\n normal_samples) else [normal_samples]\n u = tf.random.uniform(\n shape=tf.shape(init_log_likelihood),\n seed=self._seed_stream(),\n dtype=init_log_likelihood.dtype.base_dtype,\n )\n threshold = init_log_likelihood + tf.math.log(u)\n\n starting_angle = tf.random.uniform(\n shape=tf.shape(init_log_likelihood),\n minval=0.,\n maxval=2 * np.pi,\n name='angle',\n seed=self._seed_stream(),\n dtype=init_log_likelihood.dtype.base_dtype,\n )\n starting_angle_min = starting_angle - 2 * np.pi\n starting_angle_max = starting_angle\n\n starting_state_parts = _rotate_on_ellipse(\n init_state_parts, normal_samples, starting_angle)\n starting_log_likelihood = self.log_likelihood_fn(*starting_state_parts) # pylint: disable=not-callable\n\n def chain_not_done(\n angle,\n angle_min,\n angle_max,\n current_state_parts,\n current_log_likelihood):\n del angle, angle_min, angle_max, current_state_parts\n return tf.reduce_any(current_log_likelihood < threshold)\n\n def sample_next_angle(\n angle,\n angle_min,\n angle_max,\n current_state_parts,\n current_log_likelihood):\n \"\"\"Slice sample a new angle, and rotate init_state by that amount.\"\"\"\n chain_not_done = current_log_likelihood < threshold\n # Box in on angle. Only update angles for which we haven't generated a\n # point that beats the threshold.\n angle_min = tf.where(\n tf.math.logical_and(angle < 0, chain_not_done),\n angle,\n angle_min)\n angle_max = tf.where(\n tf.math.logical_and(angle >= 0, chain_not_done),\n angle,\n angle_max)\n new_angle = tf.random.uniform(\n shape=tf.shape(current_log_likelihood),\n minval=angle_min,\n maxval=angle_max,\n seed=self._seed_stream(),\n dtype=angle.dtype.base_dtype\n )\n angle = tf.where(chain_not_done, new_angle, angle)\n next_state_parts = _rotate_on_ellipse(\n init_state_parts, normal_samples, angle)\n\n new_state_parts = []\n broadcasted_chain_not_done = _right_pad_with_ones(\n chain_not_done, tf.rank(next_state_parts[0]))\n for n_state, c_state in zip(next_state_parts, current_state_parts):\n new_state_part = tf.where(\n tf.broadcast_to(\n broadcasted_chain_not_done,\n tf.shape(n_state)),\n n_state,\n c_state)\n new_state_parts.append(new_state_part)\n\n return (\n angle,\n angle_min,\n angle_max,\n new_state_parts,\n self.log_likelihood_fn(*new_state_parts) # pylint: disable=not-callable\n )\n\n [\n next_angle,\n _,\n _,\n next_state_parts,\n next_log_likelihood,\n ] = tf.while_loop(\n cond=chain_not_done,\n body=sample_next_angle,\n loop_vars=[\n starting_angle,\n starting_angle_min,\n starting_angle_max,\n starting_state_parts,\n starting_log_likelihood\n ])\n\n return [\n next_state_parts if mcmc_util.is_list_like(\n current_state) else next_state_parts[0],\n EllipticalSliceSamplerKernelResults(\n log_likelihood=next_log_likelihood,\n angle=next_angle,\n normal_samples=normal_samples,\n ),\n ]", "response": "Runs one iteration of the Elliptical Slice Sampler."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef image_summary(seqs, name, num=None):\n seqs = tf.clip_by_value(seqs, 0., 1.)\n seqs = tf.unstack(seqs[:num])\n joined_seqs = [tf.concat(tf.unstack(seq), 1) for seq in seqs]\n joined_seqs = tf.expand_dims(tf.concat(joined_seqs, 0), 0)\n tf.compat.v2.summary.image(\n name,\n joined_seqs,\n max_outputs=1,\n step=tf.compat.v1.train.get_or_create_global_step())", "response": "Visualizes sequences as TensorBoard summaries."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nvisualizes the reconstruction of inputs in TensorBoard.", "response": "def visualize_reconstruction(inputs, reconstruct, num=3, name=\"reconstruction\"):\n \"\"\"Visualizes the reconstruction of inputs in TensorBoard.\n\n Args:\n inputs: A tensor of the original inputs, of shape [batch, timesteps,\n h, w, c].\n reconstruct: A tensor of a reconstruction of inputs, of shape\n [batch, timesteps, h, w, c].\n num: Integer for the number of examples to visualize.\n name: String name of this summary.\n \"\"\"\n reconstruct = tf.clip_by_value(reconstruct, 0., 1.)\n inputs_and_reconstruct = tf.concat((inputs[:num], reconstruct[:num]), axis=0)\n image_summary(inputs_and_reconstruct, name)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nvisualizing a qualitative analysis of a given model.", "response": "def visualize_qualitative_analysis(inputs, model, samples=1, batch_size=3,\n length=8):\n \"\"\"Visualizes a qualitative analysis of a given model.\n\n Args:\n inputs: A tensor of the original inputs, of shape [batch, timesteps,\n h, w, c].\n model: A DisentangledSequentialVAE model.\n samples: Number of samples to draw from the latent distributions.\n batch_size: Number of sequences to generate.\n length: Number of timesteps to generate for each sequence.\n \"\"\"\n average = lambda dist: tf.reduce_mean(\n input_tensor=dist.mean(), axis=0) # avg over samples\n with tf.compat.v1.name_scope(\"val_reconstruction\"):\n reconstruct = functools.partial(model.reconstruct, inputs=inputs,\n samples=samples)\n visualize_reconstruction(inputs, average(reconstruct()))\n visualize_reconstruction(inputs, average(reconstruct(sample_static=True)),\n name=\"static_prior\")\n visualize_reconstruction(inputs, average(reconstruct(sample_dynamic=True)),\n name=\"dynamic_prior\")\n visualize_reconstruction(inputs, average(reconstruct(swap_static=True)),\n name=\"swap_static\")\n visualize_reconstruction(inputs, average(reconstruct(swap_dynamic=True)),\n name=\"swap_dynamic\")\n\n with tf.compat.v1.name_scope(\"generation\"):\n generate = functools.partial(model.generate, batch_size=batch_size,\n length=length, samples=samples)\n image_summary(average(generate(fix_static=True)), \"fix_static\")\n image_summary(average(generate(fix_dynamic=True)), \"fix_dynamic\")"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef summarize_dist_params(dist, name, name_scope=\"dist_params\"):\n with tf.compat.v1.name_scope(name_scope):\n tf.compat.v2.summary.histogram(\n name=\"{}/{}\".format(name, \"mean\"),\n data=dist.mean(),\n step=tf.compat.v1.train.get_or_create_global_step())\n tf.compat.v2.summary.histogram(\n name=\"{}/{}\".format(name, \"stddev\"),\n data=dist.stddev(),\n step=tf.compat.v1.train.get_or_create_global_step())", "response": "Summarize the parameters of a distribution."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef summarize_mean_in_nats_and_bits(inputs, units, name,\n nats_name_scope=\"nats\",\n bits_name_scope=\"bits_per_dim\"):\n \"\"\"Summarize the mean of a tensor in nats and bits per unit.\n\n Args:\n inputs: A tensor of values measured in nats.\n units: The units of the tensor with which to compute the mean bits\n per unit.\n name: The name of the tensor.\n nats_name_scope: The name scope of the nats summary.\n bits_name_scope: The name scope of the bits summary.\n \"\"\"\n mean = tf.reduce_mean(input_tensor=inputs)\n with tf.compat.v1.name_scope(nats_name_scope):\n tf.compat.v2.summary.scalar(\n name,\n mean,\n step=tf.compat.v1.train.get_or_create_global_step())\n with tf.compat.v1.name_scope(bits_name_scope):\n tf.compat.v2.summary.scalar(\n name,\n mean / units / tf.math.log(2.),\n step=tf.compat.v1.train.get_or_create_global_step())", "response": "Summarize the mean of a tensor in nats and bits per unit."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nrun the model to generate multivariate normal distribution.", "response": "def call(self, inputs):\n \"\"\"Runs the model to generate multivariate normal distribution.\n\n Args:\n inputs: Unused.\n\n Returns:\n A MultivariateNormalDiag distribution with event shape\n [dimensions], batch shape [], and sample shape [sample_shape,\n dimensions].\n \"\"\"\n del inputs # unused\n with tf.compat.v1.name_scope(self._name):\n return tfd.MultivariateNormalDiag(self.loc, self.scale_diag)"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef zero_state(self, sample_batch_shape=()):\n h0 = tf.zeros([1, self.hidden_size])\n c0 = tf.zeros([1, self.hidden_size])\n combined_shape = tf.concat((tf.convert_to_tensor(\n value=sample_batch_shape, dtype=tf.int32), [self.dimensions]),\n axis=-1)\n previous_output = tf.zeros(combined_shape)\n return previous_output, (h0, c0)", "response": "Returns an initial state for the LSTM cell."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef call(self, inputs, state):\n # In order to allow the user to pass in a single example without a batch\n # dimension, we always expand the input to at least two dimensions, then\n # fix the output shape to remove the batch dimension if necessary.\n original_shape = inputs.shape\n if len(original_shape) < 2:\n inputs = tf.reshape(inputs, [1, -1])\n out, state = self.lstm_cell(inputs, state)\n out = self.output_layer(out)\n correct_shape = tf.concat((original_shape[:-1], tf.shape(input=out)[-1:]),\n 0)\n out = tf.reshape(out, correct_shape)\n loc = out[..., :self.dimensions]\n scale_diag = tf.nn.softplus(out[..., self.dimensions:]) + 1e-5 # keep > 0\n return tfd.MultivariateNormalDiag(loc=loc, scale_diag=scale_diag), state", "response": "Runs the model to generate a batched MultivariateNormalDiag distribution for a single timestep."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nrunning the model to generate a set of IndependentDistributions over the pixels of x_t.", "response": "def call(self, inputs):\n \"\"\"Runs the model to generate a distribution p(x_t | z_t, f).\n\n Args:\n inputs: A tuple of (z_{1:T}, f), where `z_{1:T}` is a tensor of\n shape [..., batch_size, timesteps, latent_size_dynamic], and `f`\n is of shape [..., batch_size, latent_size_static].\n\n Returns:\n A batched Independent distribution wrapping a set of Normal\n distributions over the pixels of x_t, where the Independent\n distribution has event shape [height, width, channels], batch\n shape [batch_size, timesteps], and sample shape [sample_shape,\n batch_size, timesteps, height, width, channels].\n \"\"\"\n # We explicitly broadcast f to the same shape as z other than the final\n # dimension, because `tf.concat` can't automatically do this.\n dynamic, static = inputs\n timesteps = tf.shape(input=dynamic)[-2]\n static = static[..., tf.newaxis, :] + tf.zeros([timesteps, 1])\n latents = tf.concat([dynamic, static], axis=-1) # (sample, N, T, latents)\n out = self.dense(latents)\n out = tf.reshape(out, (-1, 1, 1, self.hidden_size))\n out = self.conv_transpose1(out)\n out = self.conv_transpose2(out)\n out = self.conv_transpose3(out)\n out = self.conv_transpose4(out) # (sample*N*T, h, w, c)\n expanded_shape = tf.concat(\n (tf.shape(input=latents)[:-1], tf.shape(input=out)[1:]), axis=0)\n out = tf.reshape(out, expanded_shape) # (sample, N, T, h, w, c)\n return tfd.Independent(\n distribution=tfd.Normal(loc=out, scale=1.),\n reinterpreted_batch_ndims=3, # wrap (h, w, c)\n name=\"decoded_image\")"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nrunning the model to generate an intermediate representation of x_t.", "response": "def call(self, inputs):\n \"\"\"Runs the model to generate an intermediate representation of x_t.\n\n Args:\n inputs: A batch of image sequences `x_{1:T}` of shape\n `[sample_shape, batch_size, timesteps, height, width,\n channels]`.\n\n Returns:\n A batch of intermediate representations of shape [sample_shape,\n batch_size, timesteps, hidden_size].\n \"\"\"\n image_shape = tf.shape(input=inputs)[-3:]\n collapsed_shape = tf.concat(([-1], image_shape), axis=0)\n out = tf.reshape(inputs, collapsed_shape) # (sample*batch*T, h, w, c)\n out = self.conv1(out)\n out = self.conv2(out)\n out = self.conv3(out)\n out = self.conv4(out)\n expanded_shape = tf.concat((tf.shape(input=inputs)[:-3], [-1]), axis=0)\n return tf.reshape(out, expanded_shape)"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nruns the model to generate a list of MultivariateNormalDiag distributions using the output of the recurrent model at each timestep.", "response": "def call(self, inputs):\n \"\"\"Runs the model to generate a distribution `q(f | x_{1:T})`.\n\n This generates a list of batched MultivariateNormalDiag\n distributions using the output of the recurrent model at each\n timestep to parameterize each distribution.\n\n Args:\n inputs: A batch of intermediate representations of image frames\n across all timesteps, of shape [..., batch_size, timesteps,\n hidden_size].\n\n Returns:\n A batched MultivariateNormalDiag distribution with event shape\n [latent_size], batch shape [..., batch_size], and sample shape\n [sample_shape, ..., batch_size, latent_size].\n \"\"\"\n # TODO(dusenberrymw): Remove these reshaping commands after b/113126249 is\n # fixed.\n collapsed_shape = tf.concat(([-1], tf.shape(input=inputs)[-2:]), axis=0)\n out = tf.reshape(inputs, collapsed_shape) # (sample*batch_size, T, hidden)\n out = self.bilstm(out) # (sample*batch_size, hidden)\n expanded_shape = tf.concat((tf.shape(input=inputs)[:-2], [-1]), axis=0)\n out = tf.reshape(out, expanded_shape) # (sample, batch_size, hidden)\n out = self.output_layer(out) # (sample, batch_size, 2*latent_size)\n loc = out[..., :self.latent_size]\n scale_diag = tf.nn.softplus(out[..., self.latent_size:]) + 1e-5 # keep > 0\n return tfd.MultivariateNormalDiag(loc=loc, scale_diag=scale_diag)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef call(self, inputs):\n out = self.dense(inputs) # (..., batch, time, hidden)\n out = self.output_layer(out) # (..., batch, time, 2*latent)\n loc = out[..., :self.latent_size]\n scale_diag = tf.nn.softplus(out[..., self.latent_size:]) + 1e-5 # keep > 0\n return tfd.MultivariateNormalDiag(loc=loc, scale_diag=scale_diag)", "response": "Runs the model to generate a MultivariateNormalDiag distribution."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nrun the model to generate a MultivariateNormalDiag .", "response": "def call(self, inputs):\n \"\"\"Runs the model to generate a distribution `q(z_{1:T} | x_{1:T}, f)`.\n\n This generates a list of batched MultivariateNormalDiag\n distributions using the output of the recurrent model at each\n timestep to parameterize each distribution.\n\n Args:\n inputs: A tuple of a batch of intermediate representations of\n image frames across all timesteps of shape [..., batch_size,\n timesteps, dimensions], and a sample of the static latent\n variable `f` of shape [..., batch_size, latent_size].\n\n Returns:\n A batch of MultivariateNormalDiag distributions with event shape\n [latent_size], batch shape [broadcasted_shape, batch_size,\n timesteps], and sample shape [sample_shape, broadcasted_shape,\n batch_size, timesteps, latent_size], where `broadcasted_shape` is\n the broadcasted sampled shape between the inputs and static\n sample.\n \"\"\"\n # We explicitly broadcast `x` and `f` to the same shape other than the final\n # dimension, because `tf.concat` can't automatically do this. This will\n # entail adding a `timesteps` dimension to `f` to give the shape `(...,\n # batch, timesteps, latent)`, and then broadcasting the sample shapes of\n # both tensors to the same shape.\n features, static_sample = inputs\n length = tf.shape(input=features)[-2]\n static_sample = static_sample[..., tf.newaxis, :] + tf.zeros([length, 1])\n sample_shape_static = tf.shape(input=static_sample)[:-3]\n sample_shape_inputs = tf.shape(input=features)[:-3]\n broadcast_shape_inputs = tf.concat((sample_shape_static, [1, 1, 1]), 0)\n broadcast_shape_static = tf.concat((sample_shape_inputs, [1, 1, 1]), 0)\n features = features + tf.zeros(broadcast_shape_inputs)\n static_sample = static_sample + tf.zeros(broadcast_shape_static)\n # `combined` will have shape (..., batch, T, hidden+latent).\n combined = tf.concat((features, static_sample), axis=-1)\n # TODO(dusenberrymw): Remove these reshaping commands after b/113126249 is\n # fixed.\n collapsed_shape = tf.concat(([-1], tf.shape(input=combined)[-2:]), axis=0)\n out = tf.reshape(combined, collapsed_shape)\n out = self.bilstm(out) # (sample*batch, T, hidden_size)\n out = self.rnn(out) # (sample*batch, T, hidden_size)\n expanded_shape = tf.concat(\n (tf.shape(input=combined)[:-2], tf.shape(input=out)[1:]), axis=0)\n out = tf.reshape(out, expanded_shape) # (sample, batch, T, hidden_size)\n out = self.output_layer(out) # (sample, batch, T, 2*latent_size)\n loc = out[..., :self.latent_size]\n scale_diag = tf.nn.softplus(out[..., self.latent_size:]) + 1e-5 # keep > 0\n return tfd.MultivariateNormalDiag(loc=loc, scale_diag=scale_diag)"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\ngenerate a new set of normalised independent distributions.", "response": "def generate(self, batch_size, length, samples=1, fix_static=False,\n fix_dynamic=False):\n \"\"\"Generate new sequences.\n\n Args:\n batch_size: Number of sequences to generate.\n length: Number of timesteps to generate for each sequence.\n samples: Number of samples to draw from the latent distributions.\n fix_static: Boolean for whether or not to share the same random\n sample of the static latent variable `f` from its prior across\n all examples.\n fix_dynamic: Boolean for whether or not to share the same random\n sample of the dynamic latent variable `z_{1:T}` from its prior\n across all examples.\n\n Returns:\n A batched Independent distribution wrapping a set of Normal\n distributions over the pixels of the generated sequences, where\n the Independent distribution has event shape [height, width,\n channels], batch shape [samples, batch_size, timesteps], and\n sample shape [sample_shape, samples, batch_size, timesteps,\n height, width, channels].\n \"\"\"\n static_sample, _ = self.sample_static_prior(samples, batch_size, fix_static)\n dynamic_sample, _ = self.sample_dynamic_prior(samples, batch_size, length,\n fix_dynamic)\n likelihood = self.decoder((dynamic_sample, static_sample))\n return likelihood"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef reconstruct(self, inputs, samples=1, sample_static=False,\n sample_dynamic=False, swap_static=False, swap_dynamic=False,\n fix_static=False, fix_dynamic=False):\n \"\"\"Reconstruct the given input sequences.\n\n Args:\n inputs: A batch of image sequences `x_{1:T}` of shape\n `[batch_size, timesteps, height, width, channels]`.\n samples: Number of samples to draw from the latent distributions.\n sample_static: Boolean for whether or not to randomly sample the\n static latent variable `f` from its prior distribution.\n sample_dynamic: Boolean for whether or not to randomly sample the\n dynamic latent variable `z_{1:T}` from its prior distribution.\n swap_static: Boolean for whether or not to swap the encodings for\n the static latent variable `f` between the examples.\n swap_dynamic: Boolean for whether or not to swap the encodings for\n the dynamic latent variable `z_{1:T}` between the examples.\n fix_static: Boolean for whether or not to share the same random\n sample of the static latent variable `f` from its prior across\n all examples.\n fix_dynamic: Boolean for whether or not to share the same random\n sample of the dynamic latent variable `z_{1:T}` from its prior\n across all examples.\n\n Returns:\n A batched Independent distribution wrapping a set of Normal\n distributions over the pixels of the reconstruction of the input,\n where the Independent distribution has event shape [height, width,\n channels], batch shape [samples, batch_size, timesteps], and\n sample shape [sample_shape, samples, batch_size, timesteps,\n height, width, channels].\n \"\"\"\n batch_size = tf.shape(input=inputs)[-5]\n length = len(tf.unstack(inputs, axis=-4)) # hack for graph mode\n\n features = self.compressor(inputs) # (..., batch, timesteps, hidden)\n\n if sample_static:\n static_sample, _ = self.sample_static_prior(\n samples, batch_size, fix_static)\n else:\n static_sample, _ = self.sample_static_posterior(features, samples)\n\n if swap_static:\n static_sample = tf.reverse(static_sample, axis=[1])\n\n if sample_dynamic:\n dynamic_sample, _ = self.sample_dynamic_prior(\n samples, batch_size, length, fix_dynamic)\n else:\n dynamic_sample, _ = self.sample_dynamic_posterior(\n features, samples, static_sample)\n\n if swap_dynamic:\n dynamic_sample = tf.reverse(dynamic_sample, axis=[1])\n\n likelihood = self.decoder((dynamic_sample, static_sample))\n return likelihood", "response": "Reconstructs the given input sequences."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nsample the static prior.", "response": "def sample_static_prior(self, samples, batch_size, fixed=False):\n \"\"\"Sample the static latent prior.\n\n Args:\n samples: Number of samples to draw from the latent distribution.\n batch_size: Number of sequences to sample.\n fixed: Boolean for whether or not to share the same random\n sample across all sequences.\n\n Returns:\n A tuple of a sample tensor of shape [samples, batch_size,\n latent_size], and a MultivariateNormalDiag distribution from which\n the tensor was sampled, with event shape [latent_size], and batch\n shape [].\n \"\"\"\n dist = self.static_prior()\n if fixed: # in either case, shape is (samples, batch, latent)\n sample = dist.sample((samples, 1)) + tf.zeros([batch_size, 1])\n else:\n sample = dist.sample((samples, batch_size))\n return sample, dist"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nsamples the static latent posterior.", "response": "def sample_static_posterior(self, inputs, samples):\n \"\"\"Sample the static latent posterior.\n\n Args:\n inputs: A batch of intermediate representations of image frames\n across all timesteps, of shape [..., batch_size, timesteps,\n hidden_size].\n samples: Number of samples to draw from the latent distribution.\n\n Returns:\n A tuple of a sample tensor of shape [samples, batch_size,\n latent_size], and a MultivariateNormalDiag distribution from which\n the tensor was sampled, with event shape [latent_size], and batch\n shape [..., batch_size].\n \"\"\"\n dist = self.static_encoder(inputs)\n sample = dist.sample(samples)\n return sample, dist"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef sample_dynamic_prior(self, samples, batch_size, length, fixed=False):\n if fixed:\n sample_batch_size = 1\n else:\n sample_batch_size = batch_size\n\n sample, state = self.dynamic_prior.zero_state([samples, sample_batch_size])\n locs = []\n scale_diags = []\n sample_list = []\n for _ in range(length):\n dist, state = self.dynamic_prior(sample, state)\n sample = dist.sample()\n locs.append(dist.parameters[\"loc\"])\n scale_diags.append(dist.parameters[\"scale_diag\"])\n sample_list.append(sample)\n\n sample = tf.stack(sample_list, axis=2)\n loc = tf.stack(locs, axis=2)\n scale_diag = tf.stack(scale_diags, axis=2)\n\n if fixed: # tile along the batch axis\n sample = sample + tf.zeros([batch_size, 1, 1])\n\n return sample, tfd.MultivariateNormalDiag(loc=loc, scale_diag=scale_diag)", "response": "Sample the dynamic prior."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nsamples the static latent variable from the given inputs.", "response": "def sample_dynamic_posterior(self, inputs, samples, static_sample=None):\n \"\"\"Sample the static latent posterior.\n\n Args:\n inputs: A batch of intermediate representations of image frames\n across all timesteps, of shape [..., batch_size, timesteps,\n hidden_size].\n samples: Number of samples to draw from the latent distribution.\n static_sample: A tensor sample of the static latent variable `f`\n of shape [..., batch_size, latent_size]. Only used\n for the full dynamic posterior formulation.\n\n Returns:\n A tuple of a sample tensor of shape [samples, batch_size, length\n latent_size], and a MultivariateNormalDiag distribution from which\n the tensor was sampled, with event shape [latent_size], and batch\n shape [broadcasted_shape, batch_size, length], where\n `broadcasted_shape` is the broadcasted sampled shape between the\n inputs and static sample.\n\n Raises:\n ValueError: If the \"full\" latent posterior formulation is being\n used, yet a static latent sample was not provided.\n \"\"\"\n if self.latent_posterior == \"factorized\":\n dist = self.dynamic_encoder(inputs)\n samples = dist.sample(samples) # (s, N, T, lat)\n else: # full\n if static_sample is None:\n raise ValueError(\n \"The full dynamic posterior requires a static latent sample\")\n dist = self.dynamic_encoder((inputs, static_sample))\n samples = dist.sample() # (samples, N, latent)\n return samples, dist"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef _canonicalize_param_vals_as_map(self, param_vals):\n if hasattr(param_vals, 'keys'):\n param_map = param_vals\n else:\n param_map = {p.name: v for (p, v) in zip(self.parameters, param_vals)}\n\n return param_map", "response": "This method takes a list of parameter values and converts them to a canonical dict that maps from the names given in self. parameters to the specified parameter values."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nsamples from the joint prior over model parameters and trajectories.", "response": "def prior_sample(self,\n num_timesteps,\n initial_step=0,\n params_sample_shape=(),\n trajectories_sample_shape=(),\n seed=None):\n \"\"\"Sample from the joint prior over model parameters and trajectories.\n\n Args:\n num_timesteps: Scalar `int` `Tensor` number of timesteps to model.\n initial_step: Optional scalar `int` `Tensor` specifying the starting\n timestep.\n Default value: 0.\n params_sample_shape: Number of possible worlds to sample iid from the\n parameter prior, or more generally, `Tensor` `int` shape to fill with\n iid samples.\n Default value: [] (i.e., draw a single sample and don't expand the\n shape).\n trajectories_sample_shape: For each sampled set of parameters, number\n of trajectories to sample, or more generally, `Tensor` `int` shape to\n fill with iid samples.\n Default value: [] (i.e., draw a single sample and don't expand the\n shape).\n seed: Python `int` random seed.\n\n Returns:\n trajectories: `float` `Tensor` of shape\n `trajectories_sample_shape + params_sample_shape + [num_timesteps, 1]`\n containing all sampled trajectories.\n param_samples: list of sampled parameter value `Tensor`s, in order\n corresponding to `self.parameters`, each of shape\n `params_sample_shape + prior.batch_shape + prior.event_shape`.\n \"\"\"\n\n seed = distributions.SeedStream(\n seed, salt='StructuralTimeSeries_prior_sample')\n\n with tf.compat.v1.name_scope(\n 'prior_sample',\n values=[num_timesteps, params_sample_shape, trajectories_sample_shape]):\n param_samples = [\n p.prior.sample(params_sample_shape, seed=seed(), name=p.name)\n for p in self.parameters\n ]\n model = self.make_state_space_model(\n num_timesteps=num_timesteps,\n initial_step=initial_step,\n param_vals=param_samples)\n return model.sample(trajectories_sample_shape, seed=seed()), param_samples"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef joint_log_prob(self, observed_time_series):\n\n with tf.compat.v1.name_scope(\n 'joint_log_prob', values=[observed_time_series]):\n [\n observed_time_series,\n mask\n ] = sts_util.canonicalize_observed_time_series_with_mask(\n observed_time_series)\n\n num_timesteps = distribution_util.prefer_static_value(\n tf.shape(input=observed_time_series))[-2]\n\n def log_joint_fn(*param_vals):\n \"\"\"Generated log-density function.\"\"\"\n\n # Sum the log_prob values from parameter priors.\n param_lp = sum([\n param.prior.log_prob(param_val)\n for (param, param_val) in zip(self.parameters, param_vals)\n ])\n\n # Build a linear Gaussian state space model and evaluate the marginal\n # log_prob on observations.\n lgssm = self.make_state_space_model(\n param_vals=param_vals, num_timesteps=num_timesteps)\n observation_lp = lgssm.log_prob(observed_time_series, mask=mask)\n\n # Sum over likelihoods from iid observations. Without this sum,\n # adding `param_lp + observation_lp` would broadcast the param priors\n # over the sample shape, which incorrectly multi-counts the param\n # priors.\n sample_ndims = tf.maximum(0,\n tf.rank(observation_lp) - tf.rank(param_lp))\n observation_lp = tf.reduce_sum(\n input_tensor=observation_lp, axis=tf.range(sample_ndims))\n\n return param_lp + observation_lp\n\n return log_joint_fn", "response": "Builds the joint log - probability function for the given model and time series."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _compute_min_event_ndims(bijector_list, compute_forward=True):\n min_event_ndims = 0\n # This is a mouthful, but what this encapsulates is that if not for rank\n # changing bijectors, we'd only need to compute the largest of the min\n # required ndims. Hence \"max_min\". Due to rank changing bijectors, we need to\n # account for synthetic rank growth / synthetic rank decrease from a rank\n # changing bijector.\n rank_changed_adjusted_max_min_event_ndims = 0\n\n if compute_forward:\n bijector_list = reversed(bijector_list)\n\n for b in bijector_list:\n if compute_forward:\n current_min_event_ndims = b.forward_min_event_ndims\n current_inverse_min_event_ndims = b.inverse_min_event_ndims\n else:\n current_min_event_ndims = b.inverse_min_event_ndims\n current_inverse_min_event_ndims = b.forward_min_event_ndims\n\n # New dimensions were touched.\n if rank_changed_adjusted_max_min_event_ndims < current_min_event_ndims:\n min_event_ndims += (\n current_min_event_ndims - rank_changed_adjusted_max_min_event_ndims)\n rank_changed_adjusted_max_min_event_ndims = max(\n current_min_event_ndims, rank_changed_adjusted_max_min_event_ndims)\n\n # If the number of dimensions has increased via forward, then\n # inverse_min_event_ndims > forward_min_event_ndims, and hence the\n # dimensions we computed on, have moved left (so we have operated\n # on additional dimensions).\n # Conversely, if the number of dimensions has decreased via forward,\n # then we have inverse_min_event_ndims < forward_min_event_ndims,\n # and so we will have operated on fewer right most dimensions.\n\n number_of_changed_dimensions = (\n current_min_event_ndims - current_inverse_min_event_ndims)\n rank_changed_adjusted_max_min_event_ndims -= number_of_changed_dimensions\n return min_event_ndims", "response": "This function computes the minimum event ndims for a given list of bijectors."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nconverting a vector size to a matrix size.", "response": "def vector_size_to_square_matrix_size(d, validate_args, name=None):\n \"\"\"Convert a vector size to a matrix size.\"\"\"\n if isinstance(d, (float, int, np.generic, np.ndarray)):\n n = (-1 + np.sqrt(1 + 8 * d)) / 2.\n if float(int(n)) != n:\n raise ValueError(\"Vector length is not a triangular number.\")\n return int(n)\n else:\n with tf.name_scope(name or \"vector_size_to_square_matrix_size\") as name:\n n = (-1. + tf.sqrt(1 + 8. * tf.cast(d, dtype=tf.float32))) / 2.\n if validate_args:\n with tf.control_dependencies([\n assert_util.assert_equal(\n tf.cast(tf.cast(n, dtype=tf.int32), dtype=tf.float32),\n n,\n message=\"Vector length is not a triangular number\")\n ]):\n n = tf.identity(n)\n return tf.cast(n, d.dtype)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef _kl_gumbel_gumbel(a, b, name=None):\n with tf.name_scope(name or \"kl_gumbel_gumbel\"):\n # Consistent with\n # http://www.mast.queensu.ca/~communications/Papers/gil-msc11.pdf, page 64\n # The paper uses beta to refer to scale and mu to refer to loc.\n # There is actually an error in the solution as printed; this is based on\n # the second-to-last step of the derivation. The value as printed would be\n # off by (a.loc - b.loc) / b.scale.\n return (tf.math.log(b.scale) - tf.math.log(a.scale) + np.euler_gamma *\n (a.scale / b.scale - 1.) +\n tf.math.expm1((b.loc - a.loc) / b.scale +\n tf.math.lgamma(a.scale / b.scale + 1.)) +\n (a.loc - b.loc) / b.scale)", "response": "Calculate the batched KL divergence of a Gumbel distribution with a and b Gumbel distribution."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef _ndtr(x):\n half_sqrt_2 = tf.constant(\n 0.5 * np.sqrt(2.), dtype=x.dtype, name=\"half_sqrt_2\")\n w = x * half_sqrt_2\n z = tf.abs(w)\n y = tf.where(\n tf.less(z, half_sqrt_2), 1. + tf.math.erf(w),\n tf.where(tf.greater(w, 0.), 2. - tf.math.erfc(z), tf.math.erfc(z)))\n return 0.5 * y", "response": "Implements ndtr core logic."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\ninversing of the CDF of the Normal distribution function.", "response": "def ndtri(p, name=\"ndtri\"):\n \"\"\"The inverse of the CDF of the Normal distribution function.\n\n Returns x such that the area under the pdf from minus infinity to x is equal\n to p.\n\n A piece-wise rational approximation is done for the function.\n This is a port of the implementation in netlib.\n\n Args:\n p: `Tensor` of type `float32`, `float64`.\n name: Python string. A name for the operation (default=\"ndtri\").\n\n Returns:\n x: `Tensor` with `dtype=p.dtype`.\n\n Raises:\n TypeError: if `p` is not floating-type.\n \"\"\"\n\n with tf.name_scope(name):\n p = tf.convert_to_tensor(value=p, name=\"p\")\n if dtype_util.as_numpy_dtype(p.dtype) not in [np.float32, np.float64]:\n raise TypeError(\n \"p.dtype=%s is not handled, see docstring for supported types.\"\n % p.dtype)\n return _ndtri(p)"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef _ndtri(p):\n\n # Constants used in piece-wise rational approximations. Taken from the cephes\n # library:\n # https://root.cern.ch/doc/v608/SpecFuncCephesInv_8cxx_source.html\n p0 = list(reversed([-5.99633501014107895267E1,\n 9.80010754185999661536E1,\n -5.66762857469070293439E1,\n 1.39312609387279679503E1,\n -1.23916583867381258016E0]))\n q0 = list(reversed([1.0,\n 1.95448858338141759834E0,\n 4.67627912898881538453E0,\n 8.63602421390890590575E1,\n -2.25462687854119370527E2,\n 2.00260212380060660359E2,\n -8.20372256168333339912E1,\n 1.59056225126211695515E1,\n -1.18331621121330003142E0]))\n p1 = list(reversed([4.05544892305962419923E0,\n 3.15251094599893866154E1,\n 5.71628192246421288162E1,\n 4.40805073893200834700E1,\n 1.46849561928858024014E1,\n 2.18663306850790267539E0,\n -1.40256079171354495875E-1,\n -3.50424626827848203418E-2,\n -8.57456785154685413611E-4]))\n q1 = list(reversed([1.0,\n 1.57799883256466749731E1,\n 4.53907635128879210584E1,\n 4.13172038254672030440E1,\n 1.50425385692907503408E1,\n 2.50464946208309415979E0,\n -1.42182922854787788574E-1,\n -3.80806407691578277194E-2,\n -9.33259480895457427372E-4]))\n p2 = list(reversed([3.23774891776946035970E0,\n 6.91522889068984211695E0,\n 3.93881025292474443415E0,\n 1.33303460815807542389E0,\n 2.01485389549179081538E-1,\n 1.23716634817820021358E-2,\n 3.01581553508235416007E-4,\n 2.65806974686737550832E-6,\n 6.23974539184983293730E-9]))\n q2 = list(reversed([1.0,\n 6.02427039364742014255E0,\n 3.67983563856160859403E0,\n 1.37702099489081330271E0,\n 2.16236993594496635890E-1,\n 1.34204006088543189037E-2,\n 3.28014464682127739104E-4,\n 2.89247864745380683936E-6,\n 6.79019408009981274425E-9]))\n\n def _create_polynomial(var, coeffs):\n \"\"\"Compute n_th order polynomial via Horner's method.\"\"\"\n coeffs = np.array(coeffs, dtype_util.as_numpy_dtype(var.dtype))\n if not coeffs.size:\n return tf.zeros_like(var)\n return coeffs[0] + _create_polynomial(var, coeffs[1:]) * var\n\n maybe_complement_p = tf.where(p > -np.expm1(-2.), 1. - p, p)\n # Write in an arbitrary value in place of 0 for p since 0 will cause NaNs\n # later on. The result from the computation when p == 0 is not used so any\n # number that doesn't result in NaNs is fine.\n sanitized_mcp = tf.where(\n maybe_complement_p <= 0.,\n tf.fill(tf.shape(input=p),\n dtype_util.as_numpy_dtype(p.dtype)(0.5)), maybe_complement_p)\n\n # Compute x for p > exp(-2): x/sqrt(2pi) = w + w**3 P0(w**2)/Q0(w**2).\n w = sanitized_mcp - 0.5\n ww = w ** 2\n x_for_big_p = w + w * ww * (_create_polynomial(ww, p0)\n / _create_polynomial(ww, q0))\n x_for_big_p *= -np.sqrt(2. * np.pi)\n\n # Compute x for p <= exp(-2): x = z - log(z)/z - (1/z) P(1/z) / Q(1/z),\n # where z = sqrt(-2. * log(p)), and P/Q are chosen between two different\n # arrays based on whether p < exp(-32).\n z = tf.sqrt(-2. * tf.math.log(sanitized_mcp))\n first_term = z - tf.math.log(z) / z\n second_term_small_p = (\n _create_polynomial(1. / z, p2) /\n _create_polynomial(1. / z, q2) / z)\n second_term_otherwise = (\n _create_polynomial(1. / z, p1) /\n _create_polynomial(1. / z, q1) / z)\n x_for_small_p = first_term - second_term_small_p\n x_otherwise = first_term - second_term_otherwise\n\n x = tf.where(sanitized_mcp > np.exp(-2.), x_for_big_p,\n tf.where(z >= 8.0, x_for_small_p, x_otherwise))\n\n x = tf.where(p > 1. - np.exp(-2.), x, -x)\n infinity_scalar = tf.constant(np.inf, dtype=p.dtype)\n infinity = tf.fill(tf.shape(input=p), infinity_scalar)\n x_nan_replaced = tf.where(\n p <= 0.0, -infinity, tf.where(p >= 1.0, infinity, x))\n return x_nan_replaced", "response": "Implements ndtri core logic."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nlog Normal distribution function.", "response": "def log_ndtr(x, series_order=3, name=\"log_ndtr\"):\n \"\"\"Log Normal distribution function.\n\n For details of the Normal distribution function see `ndtr`.\n\n This function calculates `(log o ndtr)(x)` by either calling `log(ndtr(x))` or\n using an asymptotic series. Specifically:\n - For `x > upper_segment`, use the approximation `-ndtr(-x)` based on\n `log(1-x) ~= -x, x << 1`.\n - For `lower_segment < x <= upper_segment`, use the existing `ndtr` technique\n and take a log.\n - For `x <= lower_segment`, we use the series approximation of erf to compute\n the log CDF directly.\n\n The `lower_segment` is set based on the precision of the input:\n\n ```\n lower_segment = { -20, x.dtype=float64\n { -10, x.dtype=float32\n upper_segment = { 8, x.dtype=float64\n { 5, x.dtype=float32\n ```\n\n When `x < lower_segment`, the `ndtr` asymptotic series approximation is:\n\n ```\n ndtr(x) = scale * (1 + sum) + R_N\n scale = exp(-0.5 x**2) / (-x sqrt(2 pi))\n sum = Sum{(-1)^n (2n-1)!! / (x**2)^n, n=1:N}\n R_N = O(exp(-0.5 x**2) (2N+1)!! / |x|^{2N+3})\n ```\n\n where `(2n-1)!! = (2n-1) (2n-3) (2n-5) ... (3) (1)` is a\n [double-factorial](https://en.wikipedia.org/wiki/Double_factorial).\n\n\n Args:\n x: `Tensor` of type `float32`, `float64`.\n series_order: Positive Python `integer`. Maximum depth to\n evaluate the asymptotic expansion. This is the `N` above.\n name: Python string. A name for the operation (default=\"log_ndtr\").\n\n Returns:\n log_ndtr: `Tensor` with `dtype=x.dtype`.\n\n Raises:\n TypeError: if `x.dtype` is not handled.\n TypeError: if `series_order` is a not Python `integer.`\n ValueError: if `series_order` is not in `[0, 30]`.\n \"\"\"\n if not isinstance(series_order, int):\n raise TypeError(\"series_order must be a Python integer.\")\n if series_order < 0:\n raise ValueError(\"series_order must be non-negative.\")\n if series_order > 30:\n raise ValueError(\"series_order must be <= 30.\")\n\n with tf.name_scope(name):\n x = tf.convert_to_tensor(value=x, name=\"x\")\n\n if dtype_util.base_equal(x.dtype, tf.float64):\n lower_segment = LOGNDTR_FLOAT64_LOWER\n upper_segment = LOGNDTR_FLOAT64_UPPER\n elif dtype_util.base_equal(x.dtype, tf.float32):\n lower_segment = LOGNDTR_FLOAT32_LOWER\n upper_segment = LOGNDTR_FLOAT32_UPPER\n else:\n raise TypeError(\"x.dtype=%s is not supported.\" % x.dtype)\n\n # The basic idea here was ported from:\n # https://root.cern.ch/doc/v608/SpecFuncCephesInv_8cxx_source.html\n # We copy the main idea, with a few changes\n # * For x >> 1, and X ~ Normal(0, 1),\n # Log[P[X < x]] = Log[1 - P[X < -x]] approx -P[X < -x],\n # which extends the range of validity of this function.\n # * We use one fixed series_order for all of 'x', rather than adaptive.\n # * Our docstring properly reflects that this is an asymptotic series, not a\n # Taylor series. We also provided a correct bound on the remainder.\n # * We need to use the max/min in the _log_ndtr_lower arg to avoid nan when\n # x=0. This happens even though the branch is unchosen because when x=0\n # the gradient of a select involves the calculation 1*dy+0*(-inf)=nan\n # regardless of whether dy is finite. Note that the minimum is a NOP if\n # the branch is chosen.\n return tf.where(\n tf.greater(x, upper_segment),\n -_ndtr(-x), # log(1-x) ~= -x, x << 1\n tf.where(\n tf.greater(x, lower_segment),\n tf.math.log(_ndtr(tf.maximum(x, lower_segment))),\n _log_ndtr_lower(tf.minimum(x, lower_segment), series_order)))"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _log_ndtr_lower(x, series_order):\n x_2 = tf.square(x)\n # Log of the term multiplying (1 + sum)\n log_scale = -0.5 * x_2 - tf.math.log(-x) - 0.5 * np.log(2. * np.pi)\n return log_scale + tf.math.log(_log_ndtr_asymptotic_series(x, series_order))", "response": "Asymptotic expansion version of Log [ cdf x ] appropriate for x << - 1."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\ncalculates the asymptotic series used in log_ndtr.", "response": "def _log_ndtr_asymptotic_series(x, series_order):\n \"\"\"Calculates the asymptotic series used in log_ndtr.\"\"\"\n npdt = dtype_util.as_numpy_dtype(x.dtype)\n if series_order <= 0:\n return npdt(1)\n x_2 = tf.square(x)\n even_sum = tf.zeros_like(x)\n odd_sum = tf.zeros_like(x)\n x_2n = x_2 # Start with x^{2*1} = x^{2*n} with n = 1.\n for n in range(1, series_order + 1):\n y = npdt(_double_factorial(2 * n - 1)) / x_2n\n if n % 2:\n odd_sum += y\n else:\n even_sum += y\n x_2n *= x_2\n return 1. + even_sum - odd_sum"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\ninverses function for erf error function.", "response": "def erfinv(x, name=\"erfinv\"):\n \"\"\"The inverse function for erf, the error function.\n\n Args:\n x: `Tensor` of type `float32`, `float64`.\n name: Python string. A name for the operation (default=\"erfinv\").\n\n Returns:\n x: `Tensor` with `dtype=x.dtype`.\n\n Raises:\n TypeError: if `x` is not floating-type.\n \"\"\"\n\n with tf.name_scope(name):\n x = tf.convert_to_tensor(value=x, name=\"x\")\n if dtype_util.as_numpy_dtype(x.dtype) not in [np.float32, np.float64]:\n raise TypeError(\"x.dtype={} is not handled, see docstring for supported \"\n \"types.\".format(dtype_util.name(x.dtype)))\n return ndtri((x + 1.) / 2.) / np.sqrt(2.)"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef log_cdf_laplace(x, name=\"log_cdf_laplace\"):\n\n with tf.name_scope(name):\n x = tf.convert_to_tensor(value=x, name=\"x\")\n\n # For x < 0, L(x) = 0.5 * exp{x} exactly, so Log[L(x)] = log(0.5) + x.\n lower_solution = -np.log(2.) + x\n\n # safe_exp_neg_x = exp{-x} for x > 0, but is\n # bounded above by 1, which avoids\n # log[1 - 1] = -inf for x = log(1/2), AND\n # exp{-x} --> inf, for x << -1\n safe_exp_neg_x = tf.exp(-tf.abs(x))\n\n # log1p(z) = log(1 + z) approx z for |z| << 1. This approxmation is used\n # internally by log1p, rather than being done explicitly here.\n upper_solution = tf.math.log1p(-0.5 * safe_exp_neg_x)\n\n return tf.where(x < 0., lower_solution, upper_solution)", "response": "Log Laplace distribution function."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef text_messages_joint_log_prob(count_data, lambda_1, lambda_2, tau):\n alpha = (1. / tf.reduce_mean(input_tensor=count_data))\n rv_lambda = tfd.Exponential(rate=alpha)\n\n rv_tau = tfd.Uniform()\n\n lambda_ = tf.gather(\n [lambda_1, lambda_2],\n indices=tf.cast(\n tau * tf.cast(tf.size(input=count_data), dtype=tf.float32) <= tf.cast(\n tf.range(tf.size(input=count_data)), dtype=tf.float32),\n dtype=tf.int32))\n rv_observation = tfd.Poisson(rate=lambda_)\n\n return (rv_lambda.log_prob(lambda_1) + rv_lambda.log_prob(lambda_2) +\n rv_tau.log_prob(tau) +\n tf.reduce_sum(input_tensor=rv_observation.log_prob(count_data)))", "response": "Joint log probability function."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nruns HMC on the text - messages unnormalized posterior.", "response": "def benchmark_text_messages_hmc(\n num_results=int(3e3),\n num_burnin_steps=int(3e3),\n num_leapfrog_steps=3):\n \"\"\"Runs HMC on the text-messages unnormalized posterior.\"\"\"\n\n if not tf.executing_eagerly():\n tf.compat.v1.reset_default_graph()\n\n # Build a static, pretend dataset.\n count_data = tf.cast(\n tf.concat(\n [tfd.Poisson(rate=15.).sample(43),\n tfd.Poisson(rate=25.).sample(31)],\n axis=0),\n dtype=tf.float32)\n if tf.executing_eagerly():\n count_data = count_data.numpy()\n else:\n with tf.compat.v1.Session():\n count_data = count_data.eval()\n\n # Define a closure over our joint_log_prob.\n def unnormalized_log_posterior(lambda1, lambda2, tau):\n return text_messages_joint_log_prob(count_data, lambda1, lambda2, tau)\n\n if tf.executing_eagerly():\n sample_chain = tf.function(tfp.mcmc.sample_chain)\n else:\n sample_chain = tfp.mcmc.sample_chain\n\n # Initialize the step_size. (It will be automatically adapted.)\n step_size = tf.compat.v2.Variable(\n name='step_size',\n initial_value=tf.constant(0.05, dtype=tf.float32),\n trainable=False)\n\n def computation():\n \"\"\"The benchmark computation.\"\"\"\n\n initial_chain_state = [\n tf.constant(count_data.mean(), name='init_lambda1'),\n tf.constant(count_data.mean(), name='init_lambda2'),\n tf.constant(0.5, name='init_tau'),\n ]\n\n unconstraining_bijectors = [\n tfp.bijectors.Exp(), # Maps a positive real to R.\n tfp.bijectors.Exp(), # Maps a positive real to R.\n tfp.bijectors.Sigmoid(), # Maps [0,1] to R.\n ]\n\n _, kernel_results = sample_chain(\n num_results=num_results,\n num_burnin_steps=num_burnin_steps,\n current_state=initial_chain_state,\n kernel=tfp.mcmc.TransformedTransitionKernel(\n inner_kernel=tfp.mcmc.HamiltonianMonteCarlo(\n target_log_prob_fn=unnormalized_log_posterior,\n num_leapfrog_steps=num_leapfrog_steps,\n step_size=step_size,\n step_size_update_fn=\n tfp.mcmc.make_simple_step_size_update_policy(num_burnin_steps),\n state_gradients_are_stopped=True),\n bijector=unconstraining_bijectors))\n\n return kernel_results.inner_results.is_accepted\n\n # Let's force evaluation of graph to ensure build time is not part of our time\n # trial.\n is_accepted_tensor = computation()\n if not tf.executing_eagerly():\n session = tf.compat.v1.Session()\n session.run(tf.compat.v1.global_variables_initializer())\n session.run(is_accepted_tensor)\n\n start_time = time.time()\n if tf.executing_eagerly():\n is_accepted = computation()\n else:\n is_accepted = session.run(is_accepted_tensor)\n wall_time = time.time() - start_time\n\n num_accepted = np.sum(is_accepted)\n acceptance_rate = np.float32(num_accepted) / np.float32(num_results)\n\n return dict(\n iters=(num_results + num_burnin_steps) * num_leapfrog_steps,\n extras={'acceptance_rate': acceptance_rate},\n wall_time=wall_time)"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nreturn True if the given index_points would yield a univariate marginal.", "response": "def _is_univariate_marginal(self, index_points):\n \"\"\"True if the given index_points would yield a univariate marginal.\n\n Args:\n index_points: the set of index set locations at which to compute the\n marginal Gaussian distribution. If this set is of size 1, the marginal is\n univariate.\n\n Returns:\n is_univariate: Boolean indicating whether the marginal is univariate or\n multivariate. In the case of dynamic shape in the number of index points,\n defaults to \"multivariate\" since that's the best we can do.\n \"\"\"\n num_index_points = tf.compat.dimension_value(\n index_points.shape[-(self.kernel.feature_ndims + 1)])\n if num_index_points is None:\n warnings.warn(\n 'Unable to detect statically whether the number of index_points is '\n '1. As a result, defaulting to treating the marginal GP at '\n '`index_points` as a multivariate Gaussian. This makes some methods, '\n 'like `cdf` unavailable.')\n return num_index_points == 1"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\ncompute the marginal distribution over function values at index_points.", "response": "def get_marginal_distribution(self, index_points=None):\n \"\"\"Compute the marginal of this GP over function values at `index_points`.\n\n Args:\n index_points: `float` `Tensor` representing finite (batch of) vector(s) of\n points in the index set over which the GP is defined. Shape has the form\n `[b1, ..., bB, e, f1, ..., fF]` where `F` is the number of feature\n dimensions and must equal `kernel.feature_ndims` and `e` is the number\n (size) of index points in each batch. Ultimately this distribution\n corresponds to a `e`-dimensional multivariate normal. The batch shape\n must be broadcastable with `kernel.batch_shape` and any batch dims\n yielded by `mean_fn`.\n\n Returns:\n marginal: a `Normal` or `MultivariateNormalLinearOperator` distribution,\n according to whether `index_points` consists of one or many index\n points, respectively.\n \"\"\"\n with self._name_scope('get_marginal_distribution'):\n # TODO(cgs): consider caching the result here, keyed on `index_points`.\n index_points = self._get_index_points(index_points)\n covariance = self._compute_covariance(index_points)\n loc = self._mean_fn(index_points)\n # If we're sure the number of index points is 1, we can just construct a\n # scalar Normal. This has computational benefits and supports things like\n # CDF that aren't otherwise straightforward to provide.\n if self._is_univariate_marginal(index_points):\n scale = tf.sqrt(covariance)\n # `loc` has a trailing 1 in the shape; squeeze it.\n loc = tf.squeeze(loc, axis=-1)\n return normal.Normal(\n loc=loc,\n scale=scale,\n validate_args=self._validate_args,\n allow_nan_stats=self._allow_nan_stats,\n name='marginal_distribution')\n else:\n scale = tf.linalg.LinearOperatorLowerTriangular(\n tf.linalg.cholesky(_add_diagonal_shift(covariance, self.jitter)),\n is_non_singular=True,\n name='GaussianProcessScaleLinearOperator')\n return mvn_linear_operator.MultivariateNormalLinearOperator(\n loc=loc,\n scale=scale,\n validate_args=self._validate_args,\n allow_nan_stats=self._allow_nan_stats,\n name='marginal_distribution')"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nreturns the index_points argument if given otherwise the class member .", "response": "def _get_index_points(self, index_points=None):\n \"\"\"Return `index_points` if not None, else `self._index_points`.\n\n Args:\n index_points: if given, this is what is returned; else,\n `self._index_points`\n\n Returns:\n index_points: the given arg, if not None, else the class member\n `self._index_points`.\n\n Rases:\n ValueError: if `index_points` and `self._index_points` are both `None`.\n \"\"\"\n if self._index_points is None and index_points is None:\n raise ValueError(\n 'This GaussianProcess instance was not instantiated with a value for '\n 'index_points. One must therefore be provided when calling sample, '\n 'log_prob, and other such methods. In particular, one can\\'t compute '\n 'KL divergences to/from an instance of `GaussianProccess` with '\n 'unspecified `index_points` directly. Instead, use the '\n '`get_marginal_distribution` function, which takes `index_points` as '\n 'an argument and returns a `Normal` or '\n '`MultivariateNormalLinearOperator` instance, whose KL can be '\n 'computed.')\n return index_points if index_points is not None else self._index_points"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _logsum_expbig_minus_expsmall(big, small):\n with tf.name_scope(\"logsum_expbig_minus_expsmall\"):\n return tf.math.log1p(-tf.exp(small - big)) + big", "response": "Stable evaluation of log [ expbig - expsmall )."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\ncomputing log_prob using log survival_function and cdf together.", "response": "def _log_prob_with_logsf_and_logcdf(self, y):\n \"\"\"Compute log_prob(y) using log survival_function and cdf together.\"\"\"\n # There are two options that would be equal if we had infinite precision:\n # Log[ sf(y - 1) - sf(y) ]\n # = Log[ exp{logsf(y - 1)} - exp{logsf(y)} ]\n # Log[ cdf(y) - cdf(y - 1) ]\n # = Log[ exp{logcdf(y)} - exp{logcdf(y - 1)} ]\n logsf_y = self.log_survival_function(y)\n logsf_y_minus_1 = self.log_survival_function(y - 1)\n logcdf_y = self.log_cdf(y)\n logcdf_y_minus_1 = self.log_cdf(y - 1)\n\n # Important: Here we use select in a way such that no input is inf, this\n # prevents the troublesome case where the output of select can be finite,\n # but the output of grad(select) will be NaN.\n\n # In either case, we are doing Log[ exp{big} - exp{small} ]\n # We want to use the sf items precisely when we are on the right side of the\n # median, which occurs when logsf_y < logcdf_y.\n big = tf.where(logsf_y < logcdf_y, logsf_y_minus_1, logcdf_y)\n small = tf.where(logsf_y < logcdf_y, logsf_y, logcdf_y_minus_1)\n\n return _logsum_expbig_minus_expsmall(big, small)"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\ncreate an IAF bijector.", "response": "def make_iaf_stack(total_event_size,\n num_hidden_layers=2,\n seed=None,\n dtype=tf.float32):\n \"\"\"Creates an stacked IAF bijector.\n\n This bijector operates on vector-valued events.\n\n Args:\n total_event_size: Number of dimensions to operate over.\n num_hidden_layers: How many hidden layers to use in each IAF.\n seed: Random seed for the initializers.\n dtype: DType for the variables.\n\n Returns:\n bijector: The created bijector.\n \"\"\"\n\n seed = tfd.SeedStream(seed, 'make_iaf_stack')\n\n def make_iaf():\n \"\"\"Create an IAF.\"\"\"\n initializer = tf.compat.v2.keras.initializers.VarianceScaling(\n 2 * 0.01, seed=seed() % (2**31 - 1))\n\n made = tfb.AutoregressiveLayer(\n params=2,\n event_shape=[total_event_size],\n hidden_units=[total_event_size] * num_hidden_layers,\n activation=tf.nn.elu,\n kernel_initializer=initializer,\n dtype=dtype)\n\n def shift_and_scale(x):\n # TODO(siege): Something is losing the static shape.\n x.set_shape(\n x.shape.merge_with([None] * (x.shape.ndims - 1) + [total_event_size]))\n return tf.unstack(made(x), num=2, axis=-1)\n\n return tfb.Invert(tfb.MaskedAutoregressiveFlow(shift_and_scale))\n\n def make_swap():\n \"\"\"Create an swap.\"\"\"\n permutation = list(reversed(range(total_event_size)))\n return tfb.Permute(permutation)\n\n bijector = make_iaf()\n bijector = make_swap()(bijector)\n bijector = make_iaf()(bijector)\n bijector = make_swap()(bijector)\n bijector = make_iaf()(bijector)\n bijector = make_swap()(bijector)\n\n return bijector"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nrun one iteration of NeuTra.", "response": "def one_step(self, current_state, previous_kernel_results):\n \"\"\"Runs one iteration of NeuTra.\n\n Args:\n current_state: `Tensor` or Python `list` of `Tensor`s representing the\n current state(s) of the Markov chain(s). The first `r` dimensions index\n independent chains, `r = tf.rank(target_log_prob_fn(*current_state))`.\n previous_kernel_results: `collections.namedtuple` containing `Tensor`s\n representing values from previous calls to this function (or from the\n `bootstrap_results` function.)\n\n Returns:\n next_state: Tensor or Python list of `Tensor`s representing the state(s)\n of the Markov chain(s) after taking exactly one step. Has same type and\n shape as `current_state`.\n kernel_results: `collections.namedtuple` of internal calculations used to\n advance the chain.\n \"\"\"\n\n @tfp.mcmc.internal.util.make_innermost_setter\n def set_num_leapfrog_steps(kernel_results, num_leapfrog_steps):\n return kernel_results._replace(\n accepted_results=kernel_results.accepted_results._replace(\n num_leapfrog_steps=num_leapfrog_steps))\n\n step_size = previous_kernel_results.new_step_size\n previous_kernel_results = set_num_leapfrog_steps(\n previous_kernel_results, self._num_leapfrog_steps(step_size))\n\n new_state, kernel_results = self._kernel.one_step(\n self._flatten_state(current_state), previous_kernel_results)\n return self._unflatten_state(new_state), kernel_results"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\ntrain the bijector and creates initial kernel_results.", "response": "def bootstrap_results(self, state):\n \"\"\"Trains the bijector and creates initial `previous_kernel_results`.\n\n The supplied `state` is only used to determine the number of chains to run\n in parallel_iterations\n\n Args:\n state: `Tensor` or Python `list` of `Tensor`s representing the initial\n state(s) of the Markov chain(s). The first `r` dimensions index\n independent chains, `r = tf.rank(target_log_prob_fn(*state))`.\n\n Returns:\n kernel_results: Instance of\n `UncalibratedHamiltonianMonteCarloKernelResults` inside\n `MetropolisHastingsResults` inside `TransformedTransitionKernelResults`\n inside `SimpleStepSizeAdaptationResults`.\n \"\"\"\n\n def loss():\n q = self._flattened_variational_distribution()\n # TODO(siege): How to seed this?\n samples = q.sample(self.train_batch_size)\n return tf.reduce_mean(\n input_tensor=q.log_prob(samples) -\n self._flattened_target_log_prob(samples),\n axis=-1)\n\n lr = tf.convert_to_tensor(value=self.learning_rate, dtype=self._dtype)\n dtype = lr.dtype\n\n learning_rate = tf.compat.v2.optimizers.schedules.PiecewiseConstantDecay(\n list(self.num_train_steps *\n np.array([0.2, 0.8]).astype(dtype.as_numpy_dtype())),\n [lr, lr * 0.1, lr * 0.01])\n\n opt = tf.compat.v2.optimizers.Adam(learning_rate)\n\n @tf.function(autograph=False)\n def train_step():\n with tf.GradientTape() as tape:\n loss_val = loss()\n vals = tape.watched_variables()\n grads = tape.gradient(loss_val, vals)\n grads_and_vals = list(zip(grads, vals))\n opt.apply_gradients(grads_and_vals)\n return loss_val\n\n for step in range(self.num_train_steps):\n loss_val = train_step()\n tf.debugging.assert_all_finite(\n loss_val, 'NeuTra loss is NaN at step {}'.format(step))\n if self.train_debug_fn:\n # pylint: disable=not-callable\n self.train_debug_fn(self, step, loss_val)\n\n state_parts = tf.nest.flatten(state)\n flat_state_shapes = tf.nest.flatten(self.state_shape)\n batch_shape = tf.shape(input=state_parts[0])[:-flat_state_shapes[0].ndims]\n\n return self._kernel.bootstrap_results(\n self._flattened_variational_distribution().sample(\n batch_shape, seed=self.seed))"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef mean_log_det(self, name=\"mean_log_det\"):\n with self._name_scope(name):\n return (self._multi_digamma(0.5 * self.df, self.dimension) +\n self.dimension * math.log(2.) +\n 2 * self.scale_operator.log_abs_determinant())", "response": "Computes the mean log det of X under this Wishart distribution."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\ncomputing the log normalizing constant log ( Z.", "response": "def log_normalization(self, name=\"log_normalization\"):\n \"\"\"Computes the log normalizing constant, log(Z).\"\"\"\n with self._name_scope(name):\n return (self.df * self.scale_operator.log_abs_determinant() +\n 0.5 * self.df * self.dimension * math.log(2.) +\n self._multi_lgamma(0.5 * self.df, self.dimension))"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\ncreates a sequence used in multivariate gamma ; shape = shape a + p.", "response": "def _multi_gamma_sequence(self, a, p, name=\"multi_gamma_sequence\"):\n \"\"\"Creates sequence used in multivariate (di)gamma; shape = shape(a)+[p].\"\"\"\n with self._name_scope(name):\n # Linspace only takes scalars, so we'll add in the offset afterwards.\n seq = tf.linspace(\n tf.constant(0., dtype=self.dtype), 0.5 - 0.5 * p, tf.cast(\n p, tf.int32))\n return seq + tf.expand_dims(a, [-1])"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _multi_lgamma(self, a, p, name=\"multi_lgamma\"):\n with self._name_scope(name):\n seq = self._multi_gamma_sequence(a, p)\n return (0.25 * p * (p - 1.) * math.log(math.pi) +\n tf.reduce_sum(input_tensor=tf.math.lgamma(seq), axis=[-1]))", "response": "Computes the log multivariate gamma function ; log ( Gamma_p a )."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _multi_digamma(self, a, p, name=\"multi_digamma\"):\n with self._name_scope(name):\n seq = self._multi_gamma_sequence(a, p)\n return tf.reduce_sum(input_tensor=tf.math.digamma(seq), axis=[-1])", "response": "Computes the multivariate digamma function ; Psi_p a."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _outer_squared_difference(x, y):\n z = x - y\n return z[..., tf.newaxis, :] * z[..., tf.newaxis]", "response": "Convenience function analogous to tf. squared_difference."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef _value_and_batch_jacobian(f, x):\n if tf.executing_eagerly():\n with tf.GradientTape() as tape:\n tape.watch(x)\n value = f(x)\n batch_jacobian = tape.batch_jacobian(value, x)\n else:\n value = f(x)\n batch_jacobian = gradients.batch_jacobian(value, x)\n return value, batch_jacobian", "response": "Enables uniform interface to value and batch jacobian calculation."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\ndisabling computation of the second derivatives for a tensor.", "response": "def _prevent_2nd_derivative(x):\n \"\"\"Disables computation of the second derivatives for a tensor.\n\n NB: you need to apply a non-identity function to the output tensor for the\n exception to be raised.\n\n Arguments:\n x: A tensor.\n\n Returns:\n A tensor with the same value and the same derivative as x, but that raises\n LookupError when trying to compute the second derivatives.\n \"\"\"\n def grad(dy):\n return array_ops.prevent_gradient(\n dy, message=\"Second derivative is not implemented.\")\n\n return tf.identity(x), grad"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nadding reparameterization (pathwise) gradients to samples of the mixture. Implicit reparameterization gradients are dx/dphi = -(d transform(x, phi) / dx)^-1 * d transform(x, phi) / dphi, where transform(x, phi) is distributional transform that removes all parameters from samples x. We implement them by replacing x with -stop_gradient(d transform(x, phi) / dx)^-1 * transform(x, phi)] for the backward pass (gradient computation). The derivative of this quantity w.r.t. phi is then the implicit reparameterization gradient. Note that this replaces the gradients w.r.t. both the mixture distribution parameters and components distributions parameters. Limitations: 1. Fundamental: components must be fully reparameterized. 2. Distributional transform is currently only implemented for factorized components. 3. Distributional transform currently only works for known rank of the batch tensor. Arguments: x: Sample of mixture distribution Returns: Tensor with same value as x, but with reparameterization gradients", "response": "def _reparameterize_sample(self, x):\n \"\"\"Adds reparameterization (pathwise) gradients to samples of the mixture.\n\n Implicit reparameterization gradients are\n dx/dphi = -(d transform(x, phi) / dx)^-1 * d transform(x, phi) / dphi,\n where transform(x, phi) is distributional transform that removes all\n parameters from samples x.\n\n We implement them by replacing x with\n -stop_gradient(d transform(x, phi) / dx)^-1 * transform(x, phi)]\n for the backward pass (gradient computation).\n The derivative of this quantity w.r.t. phi is then the implicit\n reparameterization gradient.\n Note that this replaces the gradients w.r.t. both the mixture\n distribution parameters and components distributions parameters.\n\n Limitations:\n 1. Fundamental: components must be fully reparameterized.\n 2. Distributional transform is currently only implemented for\n factorized components.\n 3. Distributional transform currently only works for known rank of the\n batch tensor.\n\n Arguments:\n x: Sample of mixture distribution\n\n Returns:\n Tensor with same value as x, but with reparameterization gradients\n \"\"\"\n # Remove the existing gradients of x wrt parameters of the components.\n x = tf.stop_gradient(x)\n\n x_2d_shape = [-1, self._event_size] # [S*prod(B), prod(E)]\n\n # Perform distributional transform of x in [S, B, E] shape,\n # but have Jacobian of size [S*prod(B), prod(E), prod(E)].\n def reshaped_distributional_transform(x_2d):\n return tf.reshape(\n self._distributional_transform(tf.reshape(x_2d, tf.shape(input=x))),\n x_2d_shape)\n\n # transform_2d: [S*prod(B), prod(E)]\n # jacobian: [S*prod(B), prod(E), prod(E)]\n transform_2d, jacobian = _value_and_batch_jacobian(\n reshaped_distributional_transform, tf.reshape(x, x_2d_shape))\n\n # We only provide the first derivative; the second derivative computed by\n # autodiff would be incorrect, so we raise an error if it is requested.\n transform_2d = _prevent_2nd_derivative(transform_2d)\n\n # Compute [- stop_gradient(jacobian)^-1 * transform] by solving a linear\n # system. The Jacobian is lower triangular because the distributional\n # transform for i-th event dimension does not depend on the next\n # dimensions.\n surrogate_x_2d = -tf.linalg.triangular_solve(\n tf.stop_gradient(jacobian), tf.expand_dims(transform_2d, axis=-1),\n lower=True) # [S*prod(B), prod(E), 1]\n surrogate_x = tf.reshape(surrogate_x_2d, tf.shape(input=x))\n\n # Replace gradients of x with gradients of surrogate_x, but keep the value.\n return x + (surrogate_x - tf.stop_gradient(surrogate_x))"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _distributional_transform(self, x):\n\n if tensorshape_util.rank(x.shape) is None:\n # tf.nn.softmax raises an error when applied to inputs of undefined rank.\n raise ValueError(\"Distributional transform does not support inputs of \"\n \"undefined rank.\")\n\n # Obtain factorized components distribution and assert that it's\n # a scalar distribution.\n if isinstance(self._components_distribution, independent.Independent):\n univariate_components = self._components_distribution.distribution\n else:\n univariate_components = self._components_distribution\n\n with tf.control_dependencies([\n assert_util.assert_equal(\n univariate_components.is_scalar_event(),\n True,\n message=\"`univariate_components` must have scalar event\")\n ]):\n x_padded = self._pad_sample_dims(x) # [S, B, 1, E]\n log_prob_x = univariate_components.log_prob(x_padded) # [S, B, k, E]\n cdf_x = univariate_components.cdf(x_padded) # [S, B, k, E]\n\n # log prob_k (x_1, ..., x_i-1)\n cumsum_log_prob_x = tf.reshape(\n tf.math.cumsum(\n # [S*prod(B)*k, prod(E)]\n tf.reshape(log_prob_x, [-1, self._event_size]),\n exclusive=True,\n axis=-1),\n tf.shape(input=log_prob_x)) # [S, B, k, E]\n\n logits_mix_prob = distribution_utils.pad_mixture_dimensions(\n self.mixture_distribution.logits, self, self.mixture_distribution,\n self._event_ndims) # [B, k, 1]\n\n # Logits of the posterior weights: log w_k + log prob_k (x_1, ..., x_i-1)\n log_posterior_weights_x = logits_mix_prob + cumsum_log_prob_x\n\n component_axis = tensorshape_util.rank(x.shape) - self._event_ndims\n posterior_weights_x = tf.nn.softmax(log_posterior_weights_x,\n axis=component_axis)\n return tf.reduce_sum(\n input_tensor=posterior_weights_x * cdf_x, axis=component_axis)", "response": "Performs the distributional transform of the mixture samples."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _split_covariance_into_marginals(covariance, block_sizes):\n start_dim = 0\n marginals = []\n for size in block_sizes:\n end_dim = start_dim + size\n marginals.append(covariance[..., start_dim:end_dim, start_dim:end_dim])\n start_dim = end_dim\n return marginals", "response": "Split a covariance matrix into block - diagonal marginals of given sizes."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\ndecompose an observed time series into contributions from each component. This method decomposes a time series according to the posterior represention of a structural time series model. In particular, it: - Computes the posterior marginal mean and covariances over the additive model's latent space. - Decomposes the latent posterior into the marginal blocks for each model component. - Maps the per-component latent posteriors back through each component's observation model, to generate the time series modeled by that component. Args: model: An instance of `tfp.sts.Sum` representing a structural time series model. observed_time_series: `float` `Tensor` of shape `batch_shape + [num_timesteps, 1]` (omitting the trailing unit dimension is also supported when `num_timesteps > 1`), specifying an observed time series. May optionally be an instance of `tfp.sts.MaskedTimeSeries`, which includes a mask `Tensor` to specify timesteps with missing observations. parameter_samples: Python `list` of `Tensors` representing posterior samples of model parameters, with shapes `[concat([ [num_posterior_draws], param.prior.batch_shape, param.prior.event_shape]) for param in model.parameters]`. This may optionally also be a map (Python `dict`) of parameter names to `Tensor` values. Returns: component_dists: A `collections.OrderedDict` instance mapping component StructuralTimeSeries instances (elements of `model.components`) to `tfd.Distribution` instances representing the posterior marginal distributions on the process modeled by each component. Each distribution has batch shape matching that of `posterior_means`/`posterior_covs`, and event shape of `[num_timesteps]`. #### Examples Suppose we've built a model and fit it to data: ```python day_of_week = tfp.sts.Seasonal( num_seasons=7, observed_time_series=observed_time_series, name='day_of_week') local_linear_trend = tfp.sts.LocalLinearTrend( observed_time_series=observed_time_series, name='local_linear_trend') model = tfp.sts.Sum(components=[day_of_week, local_linear_trend], observed_time_series=observed_time_series) num_steps_forecast = 50 samples, kernel_results = tfp.sts.fit_with_hmc(model, observed_time_series) ``` To extract the contributions of individual components, pass the time series and sampled parameters into `decompose_by_component`: ```python component_dists = decompose_by_component( model, observed_time_series=observed_time_series, parameter_samples=samples) # Component mean and stddev have shape `[len(observed_time_series)]`. day_of_week_effect_mean = component_dists[day_of_week].mean() day_of_week_effect_stddev = component_dists[day_of_week].stddev() ``` Using the component distributions, we can visualize the uncertainty for each component: ``` from matplotlib import pylab as plt num_components = len(component_dists) xs = np.arange(len(observed_time_series)) fig = plt.figure(figsize=(12, 3 * num_components)) for i, (component, component_dist) in enumerate(component_dists.items()): # If in graph mode, replace `.numpy()` with `.eval()` or `sess.run()`. component_mean = component_dist.mean().numpy() component_stddev = component_dist.stddev().numpy() ax = fig.add_subplot(num_components, 1, 1 + i) ax.plot(xs, component_mean, lw=2) ax.fill_between(xs, component_mean - 2 * component_stddev, component_mean + 2 * component_stddev, alpha=0.5) ax.set_title(component.name) ```", "response": "def decompose_by_component(model, observed_time_series, parameter_samples):\n \"\"\"Decompose an observed time series into contributions from each component.\n\n This method decomposes a time series according to the posterior represention\n of a structural time series model. In particular, it:\n - Computes the posterior marginal mean and covariances over the additive\n model's latent space.\n - Decomposes the latent posterior into the marginal blocks for each\n model component.\n - Maps the per-component latent posteriors back through each component's\n observation model, to generate the time series modeled by that component.\n\n Args:\n model: An instance of `tfp.sts.Sum` representing a structural time series\n model.\n observed_time_series: `float` `Tensor` of shape\n `batch_shape + [num_timesteps, 1]` (omitting the trailing unit dimension\n is also supported when `num_timesteps > 1`), specifying an observed time\n series. May optionally be an instance of `tfp.sts.MaskedTimeSeries`, which\n includes a mask `Tensor` to specify timesteps with missing observations.\n parameter_samples: Python `list` of `Tensors` representing posterior\n samples of model parameters, with shapes `[concat([\n [num_posterior_draws], param.prior.batch_shape,\n param.prior.event_shape]) for param in model.parameters]`. This may\n optionally also be a map (Python `dict`) of parameter names to\n `Tensor` values.\n Returns:\n component_dists: A `collections.OrderedDict` instance mapping\n component StructuralTimeSeries instances (elements of `model.components`)\n to `tfd.Distribution` instances representing the posterior marginal\n distributions on the process modeled by each component. Each distribution\n has batch shape matching that of `posterior_means`/`posterior_covs`, and\n event shape of `[num_timesteps]`.\n\n #### Examples\n\n Suppose we've built a model and fit it to data:\n\n ```python\n day_of_week = tfp.sts.Seasonal(\n num_seasons=7,\n observed_time_series=observed_time_series,\n name='day_of_week')\n local_linear_trend = tfp.sts.LocalLinearTrend(\n observed_time_series=observed_time_series,\n name='local_linear_trend')\n model = tfp.sts.Sum(components=[day_of_week, local_linear_trend],\n observed_time_series=observed_time_series)\n\n num_steps_forecast = 50\n samples, kernel_results = tfp.sts.fit_with_hmc(model, observed_time_series)\n ```\n\n To extract the contributions of individual components, pass the time series\n and sampled parameters into `decompose_by_component`:\n\n ```python\n component_dists = decompose_by_component(\n model,\n observed_time_series=observed_time_series,\n parameter_samples=samples)\n\n # Component mean and stddev have shape `[len(observed_time_series)]`.\n day_of_week_effect_mean = component_dists[day_of_week].mean()\n day_of_week_effect_stddev = component_dists[day_of_week].stddev()\n ```\n\n Using the component distributions, we can visualize the uncertainty for\n each component:\n\n ```\n from matplotlib import pylab as plt\n num_components = len(component_dists)\n xs = np.arange(len(observed_time_series))\n fig = plt.figure(figsize=(12, 3 * num_components))\n for i, (component, component_dist) in enumerate(component_dists.items()):\n\n # If in graph mode, replace `.numpy()` with `.eval()` or `sess.run()`.\n component_mean = component_dist.mean().numpy()\n component_stddev = component_dist.stddev().numpy()\n\n ax = fig.add_subplot(num_components, 1, 1 + i)\n ax.plot(xs, component_mean, lw=2)\n ax.fill_between(xs,\n component_mean - 2 * component_stddev,\n component_mean + 2 * component_stddev,\n alpha=0.5)\n ax.set_title(component.name)\n ```\n\n \"\"\"\n\n with tf.compat.v1.name_scope('decompose_by_component',\n values=[observed_time_series]):\n [\n observed_time_series,\n is_missing\n ] = sts_util.canonicalize_observed_time_series_with_mask(\n observed_time_series)\n\n # Run smoothing over the training timesteps to extract the\n # posterior on latents.\n num_timesteps = dist_util.prefer_static_value(\n tf.shape(input=observed_time_series))[-2]\n ssm = model.make_state_space_model(num_timesteps=num_timesteps,\n param_vals=parameter_samples)\n posterior_means, posterior_covs = ssm.posterior_marginals(\n observed_time_series, mask=is_missing)\n\n return _decompose_from_posterior_marginals(\n model, posterior_means, posterior_covs, parameter_samples)"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef decompose_forecast_by_component(model, forecast_dist, parameter_samples):\n\n with tf.compat.v1.name_scope('decompose_forecast_by_component'):\n try:\n forecast_lgssm = forecast_dist.components_distribution\n forecast_latent_mean, _ = forecast_lgssm._joint_mean() # pylint: disable=protected-access\n forecast_latent_covs, _ = forecast_lgssm._joint_covariances() # pylint: disable=protected-access\n except AttributeError as e:\n raise ValueError(\n 'Forecast distribution must be a MixtureSameFamily of'\n 'LinearGaussianStateSpaceModel distributions, such as returned by'\n '`tfp.sts.forecast()`. (saw exception: {})'.format(e))\n\n # Since `parameter_samples` will have sample shape `[num_posterior_draws]`,\n # we need to move the `num_posterior_draws` dimension of the forecast\n # moments from the trailing batch dimension, where it's currently put by\n # `sts.forecast`, back to the leading (sample shape) dimension.\n forecast_latent_mean = dist_util.move_dimension(\n forecast_latent_mean, source_idx=-3, dest_idx=0)\n forecast_latent_covs = dist_util.move_dimension(\n forecast_latent_covs, source_idx=-4, dest_idx=0)\n\n return _decompose_from_posterior_marginals(\n model, forecast_latent_mean, forecast_latent_covs, parameter_samples)", "response": "Decompose a forecast distribution into contributions from each component."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nconvert dense tensor x to SparseTensor.", "response": "def dense_to_sparse(x, ignore_value=None, name=None):\n \"\"\"Converts dense `Tensor` to `SparseTensor`, dropping `ignore_value` cells.\n\n Args:\n x: A `Tensor`.\n ignore_value: Entries in `x` equal to this value will be\n absent from the return `SparseTensor`. If `None`, default value of\n `x` dtype will be used (e.g. '' for `str`, 0 for `int`).\n name: Python `str` prefix for ops created by this function.\n\n Returns:\n sparse_x: A `tf.SparseTensor` with the same shape as `x`.\n\n Raises:\n ValueError: when `x`'s rank is `None`.\n \"\"\"\n # Copied (with modifications) from:\n # tensorflow/contrib/layers/python/ops/sparse_ops.py.\n with tf.compat.v1.name_scope(name, 'dense_to_sparse', [x, ignore_value]):\n x = tf.convert_to_tensor(value=x, name='x')\n if ignore_value is None:\n if x.dtype.base_dtype == tf.string:\n # Exception due to TF strings are converted to numpy objects by default.\n ignore_value = ''\n else:\n ignore_value = x.dtype.as_numpy_dtype(0)\n ignore_value = tf.cast(ignore_value, x.dtype, name='ignore_value')\n indices = tf.where(tf.not_equal(x, ignore_value), name='indices')\n return tf.SparseTensor(\n indices=indices,\n values=tf.gather_nd(x, indices, name='values'),\n dense_shape=tf.shape(input=x, out_type=tf.int64, name='dense_shape'))"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef _operator(attr):\n @functools.wraps(attr)\n def func(a, *args):\n return attr(a.value, *args)\n return func", "response": "Defers an operator overload to attr.\n\n"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef sample_shape(self):\n if isinstance(self._sample_shape, tf.Tensor):\n return tf.TensorShape(tf.get_static_value(self._sample_shape))\n return tf.TensorShape(self._sample_shape)", "response": "Sample shape of random variable as a TensorShape."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nreturns the sample_shape of the random variable as a 1 - D Tensor.", "response": "def sample_shape_tensor(self, name=\"sample_shape_tensor\"):\n \"\"\"Sample shape of random variable as a 1-D `Tensor`.\n\n Args:\n name: name to give to the op\n\n Returns:\n sample_shape: `Tensor`.\n \"\"\"\n with tf.compat.v1.name_scope(name):\n if isinstance(self._sample_shape, tf.Tensor):\n return self._sample_shape\n return tf.convert_to_tensor(\n value=self.sample_shape.as_list(), dtype=tf.int32)"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\ngets tensor that the random variable corresponds to.", "response": "def value(self):\n \"\"\"Get tensor that the random variable corresponds to.\"\"\"\n if self._value is None:\n try:\n self._value = self.distribution.sample(self.sample_shape_tensor())\n except NotImplementedError:\n raise NotImplementedError(\n \"sample is not implemented for {0}. You must either pass in the \"\n \"value argument or implement sample for {0}.\"\n .format(self.distribution.__class__.__name__))\n return self._value"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef numpy(self):\n if not isinstance(self.value, ops.EagerTensor):\n raise NotImplementedError(\"value argument must be a EagerTensor.\")\n\n return self.value.numpy()", "response": "Value as NumPy array only available for TF Eager."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nbuilding a scale-and-shift function using a multi-layer neural network. This will be wrapped in a make_template to ensure the variables are only created once. It takes the `d`-dimensional input x[0:d] and returns the `D-d` dimensional outputs `loc` (\"mu\") and `log_scale` (\"alpha\"). The default template does not support conditioning and will raise an exception if `condition_kwargs` are passed to it. To use conditioning in real nvp bijector, implement a conditioned shift/scale template that handles the `condition_kwargs`. Arguments: hidden_layers: Python `list`-like of non-negative integer, scalars indicating the number of units in each hidden layer. Default: `[512, 512]. shift_only: Python `bool` indicating if only the `shift` term shall be computed (i.e. NICE bijector). Default: `False`. activation: Activation function (callable). Explicitly setting to `None` implies a linear activation. name: A name for ops managed by this function. Default: \"real_nvp_default_template\". *args: `tf.layers.dense` arguments. **kwargs: `tf.layers.dense` keyword arguments. Returns: shift: `Float`-like `Tensor` of shift terms (\"mu\" in [Papamakarios et al. (2016)][1]). log_scale: `Float`-like `Tensor` of log(scale) terms (\"alpha\" in [Papamakarios et al. (2016)][1]). Raises: NotImplementedError: if rightmost dimension of `inputs` is unknown prior to graph execution, or if `condition_kwargs` is not empty. #### References [1]: George Papamakarios, Theo Pavlakou, and Iain Murray. Masked Autoregressive Flow for Density Estimation. In _Neural Information Processing Systems_, 2017. https://arxiv.org/abs/1705.07057", "response": "def real_nvp_default_template(hidden_layers,\n shift_only=False,\n activation=tf.nn.relu,\n name=None,\n *args, # pylint: disable=keyword-arg-before-vararg\n **kwargs):\n \"\"\"Build a scale-and-shift function using a multi-layer neural network.\n\n This will be wrapped in a make_template to ensure the variables are only\n created once. It takes the `d`-dimensional input x[0:d] and returns the `D-d`\n dimensional outputs `loc` (\"mu\") and `log_scale` (\"alpha\").\n\n The default template does not support conditioning and will raise an\n exception if `condition_kwargs` are passed to it. To use conditioning in\n real nvp bijector, implement a conditioned shift/scale template that\n handles the `condition_kwargs`.\n\n Arguments:\n hidden_layers: Python `list`-like of non-negative integer, scalars\n indicating the number of units in each hidden layer. Default: `[512, 512].\n shift_only: Python `bool` indicating if only the `shift` term shall be\n computed (i.e. NICE bijector). Default: `False`.\n activation: Activation function (callable). Explicitly setting to `None`\n implies a linear activation.\n name: A name for ops managed by this function. Default:\n \"real_nvp_default_template\".\n *args: `tf.layers.dense` arguments.\n **kwargs: `tf.layers.dense` keyword arguments.\n\n Returns:\n shift: `Float`-like `Tensor` of shift terms (\"mu\" in\n [Papamakarios et al. (2016)][1]).\n log_scale: `Float`-like `Tensor` of log(scale) terms (\"alpha\" in\n [Papamakarios et al. (2016)][1]).\n\n Raises:\n NotImplementedError: if rightmost dimension of `inputs` is unknown prior to\n graph execution, or if `condition_kwargs` is not empty.\n\n #### References\n\n [1]: George Papamakarios, Theo Pavlakou, and Iain Murray. Masked\n Autoregressive Flow for Density Estimation. In _Neural Information\n Processing Systems_, 2017. https://arxiv.org/abs/1705.07057\n \"\"\"\n\n with tf.compat.v2.name_scope(name or \"real_nvp_default_template\"):\n\n def _fn(x, output_units, **condition_kwargs):\n \"\"\"Fully connected MLP parameterized via `real_nvp_template`.\"\"\"\n if condition_kwargs:\n raise NotImplementedError(\n \"Conditioning not implemented in the default template.\")\n\n if tensorshape_util.rank(x.shape) == 1:\n x = x[tf.newaxis, ...]\n reshape_output = lambda x: x[0]\n else:\n reshape_output = lambda x: x\n for units in hidden_layers:\n x = tf.compat.v1.layers.dense(\n inputs=x,\n units=units,\n activation=activation,\n *args, # pylint: disable=keyword-arg-before-vararg\n **kwargs)\n x = tf.compat.v1.layers.dense(\n inputs=x,\n units=(1 if shift_only else 2) * output_units,\n activation=None,\n *args, # pylint: disable=keyword-arg-before-vararg\n **kwargs)\n if shift_only:\n return reshape_output(x), None\n shift, log_scale = tf.split(x, 2, axis=-1)\n return reshape_output(shift), reshape_output(log_scale)\n\n return tf.compat.v1.make_template(\"real_nvp_default_template\", _fn)"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _uniform_unit_norm(dimension, shape, dtype, seed):\n # This works because the Gaussian distribution is spherically symmetric.\n # raw shape: shape + [dimension]\n raw = normal.Normal(\n loc=dtype_util.as_numpy_dtype(dtype)(0),\n scale=dtype_util.as_numpy_dtype(dtype)(1)).sample(\n tf.concat([shape, [dimension]], axis=0), seed=seed())\n unit_norm = raw / tf.norm(tensor=raw, ord=2, axis=-1)[..., tf.newaxis]\n return unit_norm", "response": "Returns a batch of points chosen uniformly from the unit hypersphere."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nreplicate the input tensor n times along a new major dimension.", "response": "def _replicate(n, tensor):\n \"\"\"Replicate the input tensor n times along a new (major) dimension.\"\"\"\n # TODO(axch) Does this already exist somewhere? Should it get contributed?\n multiples = tf.concat([[n], tf.ones_like(tensor.shape)], axis=0)\n return tf.tile(tf.expand_dims(tensor, axis=0), multiples)"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _sample_n(self, num_samples, seed=None, name=None):\n if self.dimension < 0:\n raise ValueError(\n 'Cannot sample negative-dimension correlation matrices.')\n # Notation below: B is the batch shape, i.e., tf.shape(concentration)\n seed = seed_stream.SeedStream(seed, 'sample_lkj')\n with tf.name_scope('sample_lkj' or name):\n if not dtype_util.is_floating(self.concentration.dtype):\n raise TypeError(\n 'The concentration argument should have floating type, not '\n '{}'.format(dtype_util.name(self.concentration.dtype)))\n\n concentration = _replicate(num_samples, self.concentration)\n concentration_shape = tf.shape(input=concentration)\n if self.dimension <= 1:\n # For any dimension <= 1, there is only one possible correlation matrix.\n shape = tf.concat([\n concentration_shape, [self.dimension, self.dimension]], axis=0)\n return tf.ones(shape=shape, dtype=self.concentration.dtype)\n beta_conc = concentration + (self.dimension - 2.) / 2.\n beta_dist = beta.Beta(concentration1=beta_conc, concentration0=beta_conc)\n\n # Note that the sampler below deviates from [1], by doing the sampling in\n # cholesky space. This does not change the fundamental logic of the\n # sampler, but does speed up the sampling.\n\n # This is the correlation coefficient between the first two dimensions.\n # This is also `r` in reference [1].\n corr12 = 2. * beta_dist.sample(seed=seed()) - 1.\n\n # Below we construct the Cholesky of the initial 2x2 correlation matrix,\n # which is of the form:\n # [[1, 0], [r, sqrt(1 - r**2)]], where r is the correlation between the\n # first two dimensions.\n # This is the top-left corner of the cholesky of the final sample.\n first_row = tf.concat([\n tf.ones_like(corr12)[..., tf.newaxis],\n tf.zeros_like(corr12)[..., tf.newaxis]], axis=-1)\n second_row = tf.concat([\n corr12[..., tf.newaxis],\n tf.sqrt(1 - corr12**2)[..., tf.newaxis]], axis=-1)\n\n chol_result = tf.concat([\n first_row[..., tf.newaxis, :],\n second_row[..., tf.newaxis, :]], axis=-2)\n\n for n in range(2, self.dimension):\n # Loop invariant: on entry, result has shape B + [n, n]\n beta_conc -= 0.5\n # norm is y in reference [1].\n norm = beta.Beta(\n concentration1=n/2.,\n concentration0=beta_conc\n ).sample(seed=seed())\n # distance shape: B + [1] for broadcast\n distance = tf.sqrt(norm)[..., tf.newaxis]\n # direction is u in reference [1].\n # direction shape: B + [n]\n direction = _uniform_unit_norm(\n n, concentration_shape, self.concentration.dtype, seed)\n # raw_correlation is w in reference [1].\n raw_correlation = distance * direction # shape: B + [n]\n\n # This is the next row in the cholesky of the result,\n # which differs from the construction in reference [1].\n # In the reference, the new row `z` = chol_result @ raw_correlation^T\n # = C @ raw_correlation^T (where as short hand we use C = chol_result).\n # We prove that the below equation is the right row to add to the\n # cholesky, by showing equality with reference [1].\n # Let S be the sample constructed so far, and let `z` be as in\n # reference [1]. Then at this iteration, the new sample S' will be\n # [[S z^T]\n # [z 1]]\n # In our case we have the cholesky decomposition factor C, so\n # we want our new row x (same size as z) to satisfy:\n # [[S z^T] [[C 0] [[C^T x^T] [[CC^T Cx^T]\n # [z 1]] = [x k]] [0 k]] = [xC^t xx^T + k**2]]\n # Since C @ raw_correlation^T = z = C @ x^T, and C is invertible,\n # we have that x = raw_correlation. Also 1 = xx^T + k**2, so k\n # = sqrt(1 - xx^T) = sqrt(1 - |raw_correlation|**2) = sqrt(1 -\n # distance**2).\n new_row = tf.concat(\n [raw_correlation, tf.sqrt(1. - norm[..., tf.newaxis])], axis=-1)\n\n # Finally add this new row, by growing the cholesky of the result.\n chol_result = tf.concat([\n chol_result,\n tf.zeros_like(chol_result[..., 0][..., tf.newaxis])], axis=-1)\n\n chol_result = tf.concat(\n [chol_result, new_row[..., tf.newaxis, :]], axis=-2)\n\n if self.input_output_cholesky:\n return chol_result\n\n result = tf.matmul(chol_result, chol_result, transpose_b=True)\n # The diagonal for a correlation matrix should always be ones. Due to\n # numerical instability the matmul might not achieve that, so manually set\n # these to ones.\n result = tf.linalg.set_diag(\n result,\n tf.ones(shape=tf.shape(input=result)[:-1], dtype=result.dtype))\n # This sampling algorithm can produce near-PSD matrices on which standard\n # algorithms such as `tf.cholesky` or `tf.linalg.self_adjoint_eigvals`\n # fail. Specifically, as documented in b/116828694, around 2% of trials\n # of 900,000 5x5 matrices (distributed according to 9 different\n # concentration parameter values) contained at least one matrix on which\n # the Cholesky decomposition failed.\n return result", "response": "Returns a Tensor of samples from an LKJ distribution."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _log_unnorm_prob(self, x, name=None):\n with tf.name_scope(name or 'log_unnorm_prob_lkj'):\n x = tf.convert_to_tensor(value=x, name='x')\n # The density is det(matrix) ** (concentration - 1).\n # Computing the determinant with `logdet` is usually fine, since\n # correlation matrices are Hermitian and PSD. But in some cases, for a\n # PSD matrix whose eigenvalues are close to zero, `logdet` raises an error\n # complaining that it is not PSD. The root cause is the computation of the\n # cholesky decomposition in `logdet`. Hence, we use the less efficient but\n # more robust `slogdet` which does not use `cholesky`.\n #\n # An alternative would have been to check allow_nan_stats and use\n # eigenvalues = tf.linalg.self_adjoint_eigvals(x)\n # psd_mask = tf.cast(\n # tf.reduce_min(eigenvalues, axis=-1) >= 0, dtype=x.dtype)\n # tf.where(psd_mask, answer, float('-inf'))\n # to emit probability 0 for inputs that are not PSD, without ever raising\n # an error. More care must be taken, as due to numerical stability issues,\n # self_adjoint_eigvals can return slightly negative eigenvalues even for\n # a PSD matrix.\n if self.input_output_cholesky:\n logdet = 2.0 * tf.reduce_sum(\n input_tensor=tf.math.log(tf.linalg.diag_part(x)), axis=[-1])\n else:\n _, logdet = tf.linalg.slogdet(x)\n answer = (self.concentration - 1.) * logdet\n return answer", "response": "Returns the unnormalized log density of a LKJ distribution."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nreturn the log normalization of an LKJ distribution.", "response": "def _log_normalization(self, name='log_normalization'):\n \"\"\"Returns the log normalization of an LKJ distribution.\n\n Args:\n name: Python `str` name prefixed to Ops created by this function.\n\n Returns:\n log_z: A Tensor of the same shape and dtype as `concentration`, containing\n the corresponding log normalizers.\n \"\"\"\n # The formula is from D. Lewandowski et al [1], p. 1999, from the\n # proof that eqs 16 and 17 are equivalent.\n with tf.name_scope(name or 'log_normalization_lkj'):\n logpi = np.log(np.pi)\n ans = tf.zeros_like(self.concentration)\n for k in range(1, self.dimension):\n ans += logpi * (k / 2.)\n ans += tf.math.lgamma(self.concentration +\n (self.dimension - 1 - k) / 2.)\n ans -= tf.math.lgamma(self.concentration + (self.dimension - 1) / 2.)\n return ans"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef common_dtype(args_list, preferred_dtype=None):\n dtype = None\n preferred_dtype = (None if preferred_dtype is None\n else tf.as_dtype(preferred_dtype))\n for a in tf.nest.flatten(args_list):\n if hasattr(a, 'dtype'):\n dt = tf.as_dtype(a.dtype)\n else:\n continue\n if dtype is None:\n dtype = dt\n elif dtype != dt:\n raise TypeError('Found incompatible dtypes, {} and {}.'.format(dtype, dt))\n if dtype is None and preferred_dtype is None:\n return None\n return (preferred_dtype if dtype is None else dtype).as_numpy_dtype", "response": "Returns explict dtype from args_list if exists else preferred_dtype."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _make_summary_statistic(attr):\n def _fn(self, **kwargs):\n \"\"\"Implements summary statistic, eg, mean, stddev, mode.\"\"\"\n x = getattr(self.distribution, attr)(**kwargs)\n shape = prefer_static.concat([\n self.distribution.batch_shape_tensor(),\n prefer_static.ones(prefer_static.rank_from_shape(self.sample_shape),\n dtype=self.sample_shape.dtype),\n self.distribution.event_shape_tensor(),\n ], axis=0)\n x = tf.reshape(x, shape=shape)\n shape = prefer_static.concat([\n self.distribution.batch_shape_tensor(),\n self.sample_shape,\n self.distribution.event_shape_tensor(),\n ], axis=0)\n return tf.broadcast_to(x, shape)\n return _fn", "response": "Factory for implementing summary statistics eg mean stddev mode."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef _kl_sample(a, b, name='kl_sample'):\n assertions = []\n a_ss = tf.get_static_value(a.sample_shape)\n b_ss = tf.get_static_value(b.sample_shape)\n msg = '`a.sample_shape` must be identical to `b.sample_shape`.'\n if a_ss is not None and b_ss is not None:\n if not np.array_equal(a_ss, b_ss):\n raise ValueError(msg)\n elif a.validate_args or b.validate_args:\n assertions.append(assert_util.assert_equal(\n a.sample_shape, b.sample_shape, message=msg))\n with tf.control_dependencies(assertions):\n kl = kullback_leibler.kl_divergence(\n a.distribution, b.distribution, name=name)\n n = prefer_static.reduce_prod(a.sample_shape)\n return tf.cast(x=n, dtype=kl.dtype) * kl", "response": "Batchwise KL divergence of a and b."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _broadcast_to(tensor_to_broadcast, target_tensors):\n output = tensor_to_broadcast\n for tensor in target_tensors:\n output += tf.zeros_like(tensor)\n return output", "response": "Helper to broadcast a tensor to a list of target tensors."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nevaluating the pdf at the peak.", "response": "def _pdf_at_peak(self):\n \"\"\"Pdf evaluated at the peak.\"\"\"\n return (self.peak - self.low) / (self.high - self.low)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef effective_sample_size(states,\n filter_threshold=0.,\n filter_beyond_lag=None,\n name=None):\n \"\"\"Estimate a lower bound on effective sample size for each independent chain.\n\n Roughly speaking, \"effective sample size\" (ESS) is the size of an iid sample\n with the same variance as `state`.\n\n More precisely, given a stationary sequence of possibly correlated random\n variables `X_1, X_2,...,X_N`, each identically distributed ESS is the number\n such that\n\n ```Variance{ N**-1 * Sum{X_i} } = ESS**-1 * Variance{ X_1 }.```\n\n If the sequence is uncorrelated, `ESS = N`. In general, one should expect\n `ESS <= N`, with more highly correlated sequences having smaller `ESS`.\n\n Args:\n states: `Tensor` or list of `Tensor` objects. Dimension zero should index\n identically distributed states.\n filter_threshold: `Tensor` or list of `Tensor` objects.\n Must broadcast with `state`. The auto-correlation sequence is truncated\n after the first appearance of a term less than `filter_threshold`.\n Setting to `None` means we use no threshold filter. Since `|R_k| <= 1`,\n setting to any number less than `-1` has the same effect.\n filter_beyond_lag: `Tensor` or list of `Tensor` objects. Must be\n `int`-like and scalar valued. The auto-correlation sequence is truncated\n to this length. Setting to `None` means we do not filter based on number\n of lags.\n name: `String` name to prepend to created ops.\n\n Returns:\n ess: `Tensor` or list of `Tensor` objects. The effective sample size of\n each component of `states`. Shape will be `states.shape[1:]`.\n\n Raises:\n ValueError: If `states` and `filter_threshold` or `states` and\n `filter_beyond_lag` are both lists with different lengths.\n\n #### Examples\n\n We use ESS to estimate standard error.\n\n ```\n import tensorflow as tf\n import tensorflow_probability as tfp\n tfd = tfp.distributions\n\n target = tfd.MultivariateNormalDiag(scale_diag=[1., 2.])\n\n # Get 1000 states from one chain.\n states = tfp.mcmc.sample_chain(\n num_burnin_steps=200,\n num_results=1000,\n current_state=tf.constant([0., 0.]),\n kernel=tfp.mcmc.HamiltonianMonteCarlo(\n target_log_prob_fn=target.log_prob,\n step_size=0.05,\n num_leapfrog_steps=20))\n states.shape\n ==> (1000, 2)\n\n ess = effective_sample_size(states)\n ==> Shape (2,) Tensor\n\n mean, variance = tf.nn.moments(states, axis=0)\n standard_error = tf.sqrt(variance / ess)\n ```\n\n Some math shows that, with `R_k` the auto-correlation sequence,\n `R_k := Covariance{X_1, X_{1+k}} / Variance{X_1}`, we have\n\n ```ESS(N) = N / [ 1 + 2 * ( (N - 1) / N * R_1 + ... + 1 / N * R_{N-1} ) ]```\n\n This function estimates the above by first estimating the auto-correlation.\n Since `R_k` must be estimated using only `N - k` samples, it becomes\n progressively noisier for larger `k`. For this reason, the summation over\n `R_k` should be truncated at some number `filter_beyond_lag < N`. Since many\n MCMC methods generate chains where `R_k > 0`, a reasonable criteria is to\n truncate at the first index where the estimated auto-correlation becomes\n negative.\n\n The arguments `filter_beyond_lag`, `filter_threshold` are filters intended to\n remove noisy tail terms from `R_k`. They combine in an \"OR\" manner meaning\n terms are removed if they were to be filtered under the `filter_beyond_lag` OR\n `filter_threshold` criteria.\n \"\"\"\n states_was_list = _is_list_like(states)\n\n # Convert all args to lists.\n if not states_was_list:\n states = [states]\n\n filter_beyond_lag = _broadcast_maybelist_arg(states, filter_beyond_lag,\n 'filter_beyond_lag')\n filter_threshold = _broadcast_maybelist_arg(states, filter_threshold,\n 'filter_threshold')\n\n # Process items, one at a time.\n with tf.compat.v1.name_scope(name, 'effective_sample_size'):\n ess_list = [\n _effective_sample_size_single_state(s, ml, mlt)\n for (s, ml, mlt) in zip(states, filter_beyond_lag, filter_threshold)\n ]\n\n if states_was_list:\n return ess_list\n return ess_list[0]", "response": "Estimate a lower bound on effective sample size for each independent chain."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef potential_scale_reduction(chains_states,\n independent_chain_ndims=1,\n name=None):\n \"\"\"Gelman and Rubin (1992)'s potential scale reduction for chain convergence.\n\n Given `N > 1` states from each of `C > 1` independent chains, the potential\n scale reduction factor, commonly referred to as R-hat, measures convergence of\n the chains (to the same target) by testing for equality of means.\n Specifically, R-hat measures the degree to which variance (of the means)\n between chains exceeds what one would expect if the chains were identically\n distributed. See [Gelman and Rubin (1992)][1]; [Brooks and Gelman (1998)][2].\n\n Some guidelines:\n\n * The initial state of the chains should be drawn from a distribution\n overdispersed with respect to the target.\n * If all chains converge to the target, then as `N --> infinity`, R-hat --> 1.\n Before that, R-hat > 1 (except in pathological cases, e.g. if the chain\n paths were identical).\n * The above holds for any number of chains `C > 1`. Increasing `C` does\n improves effectiveness of the diagnostic.\n * Sometimes, R-hat < 1.2 is used to indicate approximate convergence, but of\n course this is problem dependent. See [Brooks and Gelman (1998)][2].\n * R-hat only measures non-convergence of the mean. If higher moments, or\n other statistics are desired, a different diagnostic should be used. See\n [Brooks and Gelman (1998)][2].\n\n Args:\n chains_states: `Tensor` or Python `list` of `Tensor`s representing the\n state(s) of a Markov Chain at each result step. The `ith` state is\n assumed to have shape `[Ni, Ci1, Ci2,...,CiD] + A`.\n Dimension `0` indexes the `Ni > 1` result steps of the Markov Chain.\n Dimensions `1` through `D` index the `Ci1 x ... x CiD` independent\n chains to be tested for convergence to the same target.\n The remaining dimensions, `A`, can have any shape (even empty).\n independent_chain_ndims: Integer type `Tensor` with value `>= 1` giving the\n number of giving the number of dimensions, from `dim = 1` to `dim = D`,\n holding independent chain results to be tested for convergence.\n name: `String` name to prepend to created tf. Default:\n `potential_scale_reduction`.\n\n Returns:\n `Tensor` or Python `list` of `Tensor`s representing the R-hat statistic for\n the state(s). Same `dtype` as `state`, and shape equal to\n `state.shape[1 + independent_chain_ndims:]`.\n\n Raises:\n ValueError: If `independent_chain_ndims < 1`.\n\n #### Examples\n\n Diagnosing convergence by monitoring 10 chains that each attempt to\n sample from a 2-variate normal.\n\n ```python\n import tensorflow as tf\n import tensorflow_probability as tfp\n tfd = tfp.distributions\n\n target = tfd.MultivariateNormalDiag(scale_diag=[1., 2.])\n\n # Get 10 (2x) overdispersed initial states.\n initial_state = target.sample(10) * 2.\n ==> (10, 2)\n\n # Get 1000 samples from the 10 independent chains.\n chains_states, _ = tfp.mcmc.sample_chain(\n num_burnin_steps=200,\n num_results=1000,\n current_state=initial_state,\n kernel=tfp.mcmc.HamiltonianMonteCarlo(\n target_log_prob_fn=target.log_prob,\n step_size=0.05,\n num_leapfrog_steps=20))\n chains_states.shape\n ==> (1000, 10, 2)\n\n rhat = tfp.mcmc.diagnostic.potential_scale_reduction(\n chains_states, independent_chain_ndims=1)\n\n # The second dimension needed a longer burn-in.\n rhat.eval()\n ==> [1.05, 1.3]\n ```\n\n To see why R-hat is reasonable, let `X` be a random variable drawn uniformly\n from the combined states (combined over all chains). Then, in the limit\n `N, C --> infinity`, with `E`, `Var` denoting expectation and variance,\n\n ```R-hat = ( E[Var[X | chain]] + Var[E[X | chain]] ) / E[Var[X | chain]].```\n\n Using the law of total variance, the numerator is the variance of the combined\n states, and the denominator is the total variance minus the variance of the\n the individual chain means. If the chains are all drawing from the same\n distribution, they will have the same mean, and thus the ratio should be one.\n\n #### References\n\n [1]: Stephen P. Brooks and Andrew Gelman. General Methods for Monitoring\n Convergence of Iterative Simulations. _Journal of Computational and\n Graphical Statistics_, 7(4), 1998.\n\n [2]: Andrew Gelman and Donald B. Rubin. Inference from Iterative Simulation\n Using Multiple Sequences. _Statistical Science_, 7(4):457-472, 1992.\n \"\"\"\n chains_states_was_list = _is_list_like(chains_states)\n if not chains_states_was_list:\n chains_states = [chains_states]\n\n # tf.get_static_value returns None iff a constant value (as a numpy\n # array) is not efficiently computable. Therefore, we try constant_value then\n # check for None.\n icn_const_ = tf.get_static_value(\n tf.convert_to_tensor(value=independent_chain_ndims))\n if icn_const_ is not None:\n independent_chain_ndims = icn_const_\n if icn_const_ < 1:\n raise ValueError(\n 'Argument `independent_chain_ndims` must be `>= 1`, found: {}'.format(\n independent_chain_ndims))\n\n with tf.compat.v1.name_scope(name, 'potential_scale_reduction'):\n rhat_list = [\n _potential_scale_reduction_single_state(s, independent_chain_ndims)\n for s in chains_states\n ]\n\n if chains_states_was_list:\n return rhat_list\n return rhat_list[0]", "response": "Gelman and Rubin 2002 s potential scale reduction for chains."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef _potential_scale_reduction_single_state(state, independent_chain_ndims):\n with tf.compat.v1.name_scope(\n 'potential_scale_reduction_single_state',\n values=[state, independent_chain_ndims]):\n # We assume exactly one leading dimension indexes e.g. correlated samples\n # from each Markov chain.\n state = tf.convert_to_tensor(value=state, name='state')\n sample_ndims = 1\n\n sample_axis = tf.range(0, sample_ndims)\n chain_axis = tf.range(sample_ndims,\n sample_ndims + independent_chain_ndims)\n sample_and_chain_axis = tf.range(\n 0, sample_ndims + independent_chain_ndims)\n\n n = _axis_size(state, sample_axis)\n m = _axis_size(state, chain_axis)\n\n # In the language of Brooks and Gelman (1998),\n # B / n is the between chain variance, the variance of the chain means.\n # W is the within sequence variance, the mean of the chain variances.\n b_div_n = _reduce_variance(\n tf.reduce_mean(input_tensor=state, axis=sample_axis, keepdims=True),\n sample_and_chain_axis,\n biased=False)\n w = tf.reduce_mean(\n input_tensor=_reduce_variance(\n state, sample_axis, keepdims=True, biased=True),\n axis=sample_and_chain_axis)\n\n # sigma^2_+ is an estimate of the true variance, which would be unbiased if\n # each chain was drawn from the target. c.f. \"law of total variance.\"\n sigma_2_plus = w + b_div_n\n\n return ((m + 1.) / m) * sigma_2_plus / w - (n - 1.) / (m * n)", "response": "Internal function for the potential scale reduction for one single state."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\ngets number of elements of x in axis as type x. dtype.", "response": "def _axis_size(x, axis=None):\n \"\"\"Get number of elements of `x` in `axis`, as type `x.dtype`.\"\"\"\n if axis is None:\n return tf.cast(tf.size(input=x), x.dtype)\n return tf.cast(\n tf.reduce_prod(input_tensor=tf.gather(tf.shape(input=x), axis)), x.dtype)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef _broadcast_maybelist_arg(states, secondary_arg, name):\n if _is_list_like(secondary_arg):\n if len(secondary_arg) != len(states):\n raise ValueError('Argument `%s` was a list of different length ({}) than '\n '`states` ({})'.format(name, len(states)))\n else:\n secondary_arg = [secondary_arg] * len(states)\n\n return secondary_arg", "response": "Broadcast a listable secondary_arg to that of states."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef quadrature_scheme_lognormal_gauss_hermite(\n loc, scale, quadrature_size,\n validate_args=False, name=None): # pylint: disable=unused-argument\n \"\"\"Use Gauss-Hermite quadrature to form quadrature on positive-reals.\n\n Note: for a given `quadrature_size`, this method is generally less accurate\n than `quadrature_scheme_lognormal_quantiles`.\n\n Args:\n loc: `float`-like (batch of) scalar `Tensor`; the location parameter of\n the LogNormal prior.\n scale: `float`-like (batch of) scalar `Tensor`; the scale parameter of\n the LogNormal prior.\n quadrature_size: Python `int` scalar representing the number of quadrature\n points.\n validate_args: Python `bool`, default `False`. When `True` distribution\n parameters are checked for validity despite possibly degrading runtime\n performance. When `False` invalid inputs may silently render incorrect\n outputs.\n name: Python `str` name prefixed to Ops created by this class.\n\n Returns:\n grid: (Batch of) length-`quadrature_size` vectors representing the\n `log_rate` parameters of a `Poisson`.\n probs: (Batch of) length-`quadrature_size` vectors representing the\n weight associate with each `grid` value.\n \"\"\"\n with tf.name_scope(\n name or \"vector_diffeomixture_quadrature_gauss_hermite\"):\n grid, probs = np.polynomial.hermite.hermgauss(deg=quadrature_size)\n npdt = dtype_util.as_numpy_dtype(loc.dtype)\n grid = grid.astype(npdt)\n probs = probs.astype(npdt)\n probs /= np.linalg.norm(probs, ord=1, keepdims=True)\n probs = tf.convert_to_tensor(value=probs, name=\"probs\", dtype=loc.dtype)\n # The following maps the broadcast of `loc` and `scale` to each grid\n # point, i.e., we are creating several log-rates that correspond to the\n # different Gauss-Hermite quadrature points and (possible) batches of\n # `loc` and `scale`.\n grid = (loc[..., tf.newaxis] + np.sqrt(2.) * scale[..., tf.newaxis] * grid)\n return grid, probs", "response": "Use Gauss - Hermite quadrature to form a lognormal graph on positive - reals."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nuses LogNormal quantiles to form quadrature on positive - reals.", "response": "def quadrature_scheme_lognormal_quantiles(\n loc, scale, quadrature_size,\n validate_args=False, name=None):\n \"\"\"Use LogNormal quantiles to form quadrature on positive-reals.\n\n Args:\n loc: `float`-like (batch of) scalar `Tensor`; the location parameter of\n the LogNormal prior.\n scale: `float`-like (batch of) scalar `Tensor`; the scale parameter of\n the LogNormal prior.\n quadrature_size: Python `int` scalar representing the number of quadrature\n points.\n validate_args: Python `bool`, default `False`. When `True` distribution\n parameters are checked for validity despite possibly degrading runtime\n performance. When `False` invalid inputs may silently render incorrect\n outputs.\n name: Python `str` name prefixed to Ops created by this class.\n\n Returns:\n grid: (Batch of) length-`quadrature_size` vectors representing the\n `log_rate` parameters of a `Poisson`.\n probs: (Batch of) length-`quadrature_size` vectors representing the\n weight associate with each `grid` value.\n \"\"\"\n with tf.name_scope(name or \"quadrature_scheme_lognormal_quantiles\"):\n # Create a LogNormal distribution.\n dist = transformed_distribution.TransformedDistribution(\n distribution=normal.Normal(loc=loc, scale=scale),\n bijector=exp_bijector.Exp(),\n validate_args=validate_args)\n batch_ndims = tensorshape_util.rank(dist.batch_shape)\n if batch_ndims is None:\n batch_ndims = tf.shape(input=dist.batch_shape_tensor())[0]\n\n def _compute_quantiles():\n \"\"\"Helper to build quantiles.\"\"\"\n # Omit {0, 1} since they might lead to Inf/NaN.\n zero = tf.zeros([], dtype=dist.dtype)\n edges = tf.linspace(zero, 1., quadrature_size + 3)[1:-1]\n # Expand edges so its broadcast across batch dims.\n edges = tf.reshape(\n edges,\n shape=tf.concat(\n [[-1], tf.ones([batch_ndims], dtype=tf.int32)], axis=0))\n quantiles = dist.quantile(edges)\n # Cyclically permute left by one.\n perm = tf.concat([tf.range(1, 1 + batch_ndims), [0]], axis=0)\n quantiles = tf.transpose(a=quantiles, perm=perm)\n return quantiles\n quantiles = _compute_quantiles()\n\n # Compute grid as quantile midpoints.\n grid = (quantiles[..., :-1] + quantiles[..., 1:]) / 2.\n # Set shape hints.\n new_shape = tensorshape_util.concatenate(dist.batch_shape,\n [quadrature_size])\n tensorshape_util.set_shape(grid, new_shape)\n\n # By construction probs is constant, i.e., `1 / quadrature_size`. This is\n # important, because non-constant probs leads to non-reparameterizable\n # samples.\n probs = tf.fill(\n dims=[quadrature_size], value=1. / tf.cast(quadrature_size, dist.dtype))\n\n return grid, probs"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef merge(self, x=None, y=None, ildj=None, kwargs=None, mapping=None):\n if mapping is None:\n mapping = _Mapping(x=x, y=y, ildj=ildj, kwargs=kwargs)\n elif any(arg is not None for arg in [x, y, ildj, kwargs]):\n raise ValueError(\"Cannot simultaneously specify mapping and individual \"\n \"arguments.\")\n\n return _Mapping(\n x=self._merge(self.x, mapping.x),\n y=self._merge(self.y, mapping.y),\n ildj=self._merge(self.ildj, mapping.ildj),\n kwargs=self._merge(self.kwargs, mapping.kwargs, use_equals=True))", "response": "Returns a new _Mapping with args merged with self."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nremove the given field from the cache value.", "response": "def remove(self, field):\n \"\"\"To support weak referencing, removes cache key from the cache value.\"\"\"\n return _Mapping(\n x=None if field == \"x\" else self.x,\n y=None if field == \"y\" else self.y,\n ildj=self.ildj,\n kwargs=self.kwargs)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _merge(self, old, new, use_equals=False):\n if old is None:\n return new\n if new is None:\n return old\n if (old == new) if use_equals else (old is new):\n return old\n raise ValueError(\"Incompatible values: %s != %s\" % (old, new))", "response": "Helper to merge two values."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nconverting nested tuple list or dict to nested tuple.", "response": "def _deep_tuple(self, x):\n \"\"\"Converts nested `tuple`, `list`, or `dict` to nested `tuple`.\"\"\"\n if isinstance(x, dict):\n return self._deep_tuple(tuple(sorted(x.items())))\n elif isinstance(x, (list, tuple)):\n return tuple(map(self._deep_tuple, x))\n\n return x"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _left_doubling_increments(batch_shape, max_doublings, step_size, seed=None,\n name=None):\n \"\"\"Computes the doubling increments for the left end point.\n\n The doubling procedure expands an initial interval to find a superset of the\n true slice. At each doubling iteration, the interval width is doubled to\n either the left or the right hand side with equal probability.\n If, initially, the left end point is at `L(0)` and the width of the\n interval is `w(0)`, then the left end point and the width at the\n k-th iteration (denoted L(k) and w(k) respectively) are given by the following\n recursions:\n\n ```none\n w(k) = 2 * w(k-1)\n L(k) = L(k-1) - w(k-1) * X_k, X_k ~ Bernoulli(0.5)\n or, L(0) - L(k) = w(0) Sum(2^i * X(i+1), 0 <= i < k)\n ```\n\n This function computes the sequence of `L(0)-L(k)` and `w(k)` for k between 0\n and `max_doublings` independently for each chain.\n\n Args:\n batch_shape: Positive int32 `tf.Tensor`. The batch shape.\n max_doublings: Scalar positive int32 `tf.Tensor`. The maximum number of\n doublings to consider.\n step_size: A real `tf.Tensor` with shape compatible with [num_chains].\n The size of the initial interval.\n seed: (Optional) positive int. The random seed. If None, no seed is set.\n name: Python `str` name prefixed to Ops created by this function.\n Default value: `None` (i.e., 'find_slice_bounds').\n\n Returns:\n left_increments: A tensor of shape (max_doublings+1, batch_shape). The\n relative position of the left end point after the doublings.\n widths: A tensor of shape (max_doublings+1, ones_like(batch_shape)). The\n widths of the intervals at each stage of the doubling.\n \"\"\"\n with tf.compat.v1.name_scope(name, 'left_doubling_increments',\n [batch_shape, max_doublings, step_size]):\n\n step_size = tf.convert_to_tensor(value=step_size)\n dtype = step_size.dtype.base_dtype\n # Output shape of the left increments tensor.\n output_shape = tf.concat(([max_doublings + 1], batch_shape), axis=0)\n # A sample realization of X_k.\n expand_left = distributions.Bernoulli(0.5, dtype=dtype).sample(\n sample_shape=output_shape, seed=seed)\n\n # The widths of the successive intervals. Starts with 1.0 and ends with\n # 2^max_doublings.\n width_multipliers = tf.cast(2 ** tf.range(0, max_doublings+1), dtype=dtype)\n # Output shape of the `widths` tensor.\n widths_shape = tf.concat(([max_doublings + 1],\n tf.ones_like(batch_shape)), axis=0)\n width_multipliers = tf.reshape(width_multipliers, shape=widths_shape)\n # Widths shape is [max_doublings + 1, 1, 1, 1...].\n widths = width_multipliers * step_size\n\n # Take the cumulative sum of the left side increments in slice width to give\n # the resulting distance from the inital lower bound.\n left_increments = tf.cumsum(widths * expand_left, exclusive=True, axis=0)\n return left_increments, widths", "response": "This function computes the doubling increments for the left end point."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _find_best_interval_idx(x, name=None):\n with tf.compat.v1.name_scope(name, 'find_best_interval_idx', [x]):\n # Returns max_doublings + 1. Positive int32.\n k = tf.shape(input=x)[0]\n dtype = x.dtype.base_dtype\n # Factors by which to multiply the flag. Corresponds to (2 * k - i) above.\n mults = tf.range(2 * k, k, -1, dtype=dtype)[:, tf.newaxis]\n # Factors by which to shift the flag. Corresponds to i above. Ensures the\n # widest bounds are selected if there are no bounds outside the slice.\n shifts = tf.range(k, dtype=dtype)[:, tf.newaxis]\n indices = tf.argmax(input=mults * x + shifts, axis=0, output_type=dtype)\n return indices", "response": "Internal function for finding the index of the optimal set of bounds for each chain."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef slice_bounds_by_doubling(x_initial,\n target_log_prob,\n log_slice_heights,\n max_doublings,\n step_size,\n seed=None,\n name=None):\n \"\"\"Returns the bounds of the slice at each stage of doubling procedure.\n\n Precomputes the x coordinates of the left (L) and right (R) endpoints of the\n interval `I` produced in the \"doubling\" algorithm [Neal 2003][1] P713. Note\n that we simultaneously compute all possible doubling values for each chain,\n for the reason that at small-medium densities, the gains from parallel\n evaluation might cause a speed-up, but this will be benchmarked against the\n while loop implementation.\n\n Args:\n x_initial: `tf.Tensor` of any shape and any real dtype consumable by\n `target_log_prob`. The initial points.\n target_log_prob: A callable taking a `tf.Tensor` of shape and dtype as\n `x_initial` and returning a tensor of the same shape. The log density of\n the target distribution.\n log_slice_heights: `tf.Tensor` with the same shape as `x_initial` and the\n same dtype as returned by `target_log_prob`. The log of the height of the\n slice for each chain. The values must be bounded above by\n `target_log_prob(x_initial)`.\n max_doublings: Scalar positive int32 `tf.Tensor`. The maximum number of\n doublings to consider.\n step_size: `tf.Tensor` with same dtype as and shape compatible with\n `x_initial`. The size of the initial interval.\n seed: (Optional) positive int. The random seed. If None, no seed is set.\n name: Python `str` name prefixed to Ops created by this function.\n Default value: `None` (i.e., 'find_slice_bounds').\n\n Returns:\n upper_bounds: A tensor of same shape and dtype as `x_initial`. Slice upper\n bounds for each chain.\n lower_bounds: A tensor of same shape and dtype as `x_initial`. Slice lower\n bounds for each chain.\n both_ok: A tensor of shape `x_initial` and boolean dtype. Indicates if both\n the chosen upper and lower bound lie outside of the slice.\n\n #### References\n\n [1]: Radford M. Neal. Slice Sampling. The Annals of Statistics. 2003, Vol 31,\n No. 3 , 705-767.\n https://projecteuclid.org/download/pdf_1/euclid.aos/1056562461\n \"\"\"\n with tf.compat.v1.name_scope(\n name, 'slice_bounds_by_doubling',\n [x_initial, log_slice_heights, max_doublings, step_size]):\n seed_gen = distributions.SeedStream(seed, salt='slice_bounds_by_doubling')\n x_initial = tf.convert_to_tensor(value=x_initial)\n batch_shape = tf.shape(input=x_initial)\n dtype = step_size.dtype.base_dtype\n left_endpoints = x_initial + step_size * tf.random.uniform(\n batch_shape, minval=-1.0, maxval=0.0, dtype=dtype, seed=seed_gen())\n\n # Compute the increments by which we need to step the upper and lower bounds\n # part of the doubling procedure.\n left_increments, widths = _left_doubling_increments(\n batch_shape, max_doublings, step_size, seed=seed_gen())\n # The left and right end points. Shape (max_doublings+1,) + batch_shape.\n left_endpoints -= left_increments\n right_endpoints = left_endpoints + widths\n\n # Test if these end points lie outside of the slice.\n # Checks if the end points of the slice are outside the graph of the pdf.\n left_ep_values = tf.map_fn(target_log_prob, left_endpoints)\n right_ep_values = tf.map_fn(target_log_prob, right_endpoints)\n left_ok = left_ep_values < log_slice_heights\n right_ok = right_ep_values < log_slice_heights\n both_ok = left_ok & right_ok\n\n both_ok_f = tf.reshape(both_ok, [max_doublings + 1, -1])\n\n best_interval_idx = _find_best_interval_idx(\n tf.cast(both_ok_f, dtype=tf.int32))\n\n # Formats the above index as required to use with gather_nd.\n point_index_gather = tf.stack(\n [best_interval_idx,\n tf.range(tf.size(input=best_interval_idx))],\n axis=1,\n name='point_index_gather')\n left_ep_f = tf.reshape(left_endpoints, [max_doublings + 1, -1])\n right_ep_f = tf.reshape(right_endpoints, [max_doublings + 1, -1])\n # The x values of the uppper and lower bounds of the slices for each chain.\n lower_bounds = tf.reshape(tf.gather_nd(left_ep_f, point_index_gather),\n batch_shape)\n upper_bounds = tf.reshape(tf.gather_nd(right_ep_f, point_index_gather),\n batch_shape)\n both_ok = tf.reduce_any(input_tensor=both_ok, axis=0)\n return upper_bounds, lower_bounds, both_ok", "response": "Returns the upper and lower bounds of a slice at each stage of doubling procedure."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _sample_with_shrinkage(x_initial, target_log_prob, log_slice_heights,\n step_size, lower_bounds, upper_bounds, seed=None,\n name=None):\n \"\"\"Samples from the slice by applying shrinkage for rejected points.\n\n Implements the one dimensional slice sampling algorithm of Neal (2003), with a\n doubling algorithm (Neal 2003 P715 Fig. 4), which doubles the size of the\n interval at each iteration and shrinkage (Neal 2003 P716 Fig. 5), which\n reduces the width of the slice when a selected point is rejected, by setting\n the relevant bound that that value. Randomly sampled points are checked for\n two criteria: that they lie within the slice and that they pass the\n acceptability check (Neal 2003 P717 Fig. 6), which tests that the new state\n could have generated the previous one.\n\n Args:\n x_initial: A tensor of any shape. The initial positions of the chains. This\n function assumes that all the dimensions of `x_initial` are batch\n dimensions (i.e. the event shape is `[]`).\n target_log_prob: Callable accepting a tensor like `x_initial` and returning\n a tensor containing the log density at that point of the same shape.\n log_slice_heights: Tensor of the same shape and dtype as the return value\n of `target_log_prob` when applied to `x_initial`. The log of the height of\n the chosen slice.\n step_size: A tensor of shape and dtype compatible with `x_initial`. The min\n interval size in the doubling algorithm.\n lower_bounds: Tensor of same shape and dtype as `x_initial`. Slice lower\n bounds for each chain.\n upper_bounds: Tensor of same shape and dtype as `x_initial`. Slice upper\n bounds for each chain.\n seed: (Optional) positive int. The random seed. If None, no seed is set.\n name: Python `str` name prefixed to Ops created by this function.\n Default value: `None` (i.e., 'find_slice_bounds').\n\n Returns:\n x_proposed: A tensor of the same shape and dtype as `x_initial`. The next\n proposed state of the chain.\n \"\"\"\n with tf.compat.v1.name_scope(\n name, 'sample_with_shrinkage',\n [x_initial, log_slice_heights, step_size, lower_bounds, upper_bounds]):\n seed_gen = distributions.SeedStream(seed, salt='_sample_with_shrinkage')\n # Keeps track of whether an acceptable sample has been found for the chain.\n found = tf.zeros_like(x_initial, dtype=tf.bool)\n cond = lambda found, *ignored_args: ~tf.reduce_all(input_tensor=found)\n x_next = tf.identity(x_initial)\n x_initial_shape = tf.shape(input=x_initial)\n x_initial_dtype = x_initial.dtype.base_dtype\n def _body(found, left, right, x_next):\n \"\"\"Iterates until every chain has found a suitable next state.\"\"\"\n proportions = tf.random.uniform(\n x_initial_shape, dtype=x_initial_dtype, seed=seed_gen())\n x_proposed = tf.where(~found, left + proportions * (right - left), x_next)\n accept_res = _test_acceptance(x_initial, target_log_prob=target_log_prob,\n decided=found,\n log_slice_heights=log_slice_heights,\n x_proposed=x_proposed, step_size=step_size,\n lower_bounds=left, upper_bounds=right)\n boundary_test = log_slice_heights < target_log_prob(x_proposed)\n can_accept = boundary_test & accept_res\n next_found = found | can_accept\n # Note that it might seem that we are moving the left and right end points\n # even if the point has been accepted (which is contrary to the stated\n # algorithm in Neal). However, this does not matter because the endpoints\n # for points that have been already accepted are not used again so it\n # doesn't matter what we do with them.\n next_left = tf.where(x_proposed < x_initial, x_proposed, left)\n next_right = tf.where(x_proposed >= x_initial, x_proposed, right)\n return next_found, next_left, next_right, x_proposed\n\n return tf.while_loop(\n cond=cond,\n body=_body,\n loop_vars=(found, lower_bounds, upper_bounds, x_next))[-1]", "response": "This function is a helper function for the one dimensional slice sampling algorithm of Neal 2003."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef sample_annealed_importance_chain(\n num_steps,\n proposal_log_prob_fn,\n target_log_prob_fn,\n current_state,\n make_kernel_fn,\n parallel_iterations=10,\n name=None):\n \"\"\"Runs annealed importance sampling (AIS) to estimate normalizing constants.\n\n This function uses an MCMC transition operator (e.g., Hamiltonian Monte Carlo)\n to sample from a series of distributions that slowly interpolates between\n an initial \"proposal\" distribution:\n\n `exp(proposal_log_prob_fn(x) - proposal_log_normalizer)`\n\n and the target distribution:\n\n `exp(target_log_prob_fn(x) - target_log_normalizer)`,\n\n accumulating importance weights along the way. The product of these\n importance weights gives an unbiased estimate of the ratio of the\n normalizing constants of the initial distribution and the target\n distribution:\n\n `E[exp(ais_weights)] = exp(target_log_normalizer - proposal_log_normalizer)`.\n\n Note: When running in graph mode, `proposal_log_prob_fn` and\n `target_log_prob_fn` are called exactly three times (although this may be\n reduced to two times in the future).\n\n Args:\n num_steps: Integer number of Markov chain updates to run. More\n iterations means more expense, but smoother annealing between q\n and p, which in turn means exponentially lower variance for the\n normalizing constant estimator.\n proposal_log_prob_fn: Python callable that returns the log density of the\n initial distribution.\n target_log_prob_fn: Python callable which takes an argument like\n `current_state` (or `*current_state` if it's a list) and returns its\n (possibly unnormalized) log-density under the target distribution.\n current_state: `Tensor` or Python `list` of `Tensor`s representing the\n current state(s) of the Markov chain(s). The first `r` dimensions index\n independent chains, `r = tf.rank(target_log_prob_fn(*current_state))`.\n make_kernel_fn: Python `callable` which returns a `TransitionKernel`-like\n object. Must take one argument representing the `TransitionKernel`'s\n `target_log_prob_fn`. The `target_log_prob_fn` argument represents the\n `TransitionKernel`'s target log distribution. Note:\n `sample_annealed_importance_chain` creates a new `target_log_prob_fn`\n which is an interpolation between the supplied `target_log_prob_fn` and\n `proposal_log_prob_fn`; it is this interpolated function which is used as\n an argument to `make_kernel_fn`.\n parallel_iterations: The number of iterations allowed to run in parallel.\n It must be a positive integer. See `tf.while_loop` for more details.\n name: Python `str` name prefixed to Ops created by this function.\n Default value: `None` (i.e., \"sample_annealed_importance_chain\").\n\n Returns:\n next_state: `Tensor` or Python list of `Tensor`s representing the\n state(s) of the Markov chain(s) at the final iteration. Has same shape as\n input `current_state`.\n ais_weights: Tensor with the estimated weight(s). Has shape matching\n `target_log_prob_fn(current_state)`.\n kernel_results: `collections.namedtuple` of internal calculations used to\n advance the chain.\n\n #### Examples\n\n ##### Estimate the normalizing constant of a log-gamma distribution.\n\n ```python\n tfd = tfp.distributions\n\n # Run 100 AIS chains in parallel\n num_chains = 100\n dims = 20\n dtype = np.float32\n\n proposal = tfd.MultivatiateNormalDiag(\n loc=tf.zeros([dims], dtype=dtype))\n\n target = tfd.TransformedDistribution(\n distribution=tfd.Gamma(concentration=dtype(2),\n rate=dtype(3)),\n bijector=tfp.bijectors.Invert(tfp.bijectors.Exp()),\n event_shape=[dims])\n\n chains_state, ais_weights, kernels_results = (\n tfp.mcmc.sample_annealed_importance_chain(\n num_steps=1000,\n proposal_log_prob_fn=proposal.log_prob,\n target_log_prob_fn=target.log_prob,\n current_state=proposal.sample(num_chains),\n make_kernel_fn=lambda tlp_fn: tfp.mcmc.HamiltonianMonteCarlo(\n target_log_prob_fn=tlp_fn,\n step_size=0.2,\n num_leapfrog_steps=2)))\n\n log_estimated_normalizer = (tf.reduce_logsumexp(ais_weights)\n - np.log(num_chains))\n log_true_normalizer = tf.lgamma(2.) - 2. * tf.log(3.)\n ```\n\n ##### Estimate marginal likelihood of a Bayesian regression model.\n\n ```python\n tfd = tfp.distributions\n\n def make_prior(dims, dtype):\n return tfd.MultivariateNormalDiag(\n loc=tf.zeros(dims, dtype))\n\n def make_likelihood(weights, x):\n return tfd.MultivariateNormalDiag(\n loc=tf.tensordot(weights, x, axes=[[0], [-1]]))\n\n # Run 100 AIS chains in parallel\n num_chains = 100\n dims = 10\n dtype = np.float32\n\n # Make training data.\n x = np.random.randn(num_chains, dims).astype(dtype)\n true_weights = np.random.randn(dims).astype(dtype)\n y = np.dot(x, true_weights) + np.random.randn(num_chains)\n\n # Setup model.\n prior = make_prior(dims, dtype)\n def target_log_prob_fn(weights):\n return prior.log_prob(weights) + make_likelihood(weights, x).log_prob(y)\n\n proposal = tfd.MultivariateNormalDiag(\n loc=tf.zeros(dims, dtype))\n\n weight_samples, ais_weights, kernel_results = (\n tfp.mcmc.sample_annealed_importance_chain(\n num_steps=1000,\n proposal_log_prob_fn=proposal.log_prob,\n target_log_prob_fn=target_log_prob_fn\n current_state=tf.zeros([num_chains, dims], dtype),\n make_kernel_fn=lambda tlp_fn: tfp.mcmc.HamiltonianMonteCarlo(\n target_log_prob_fn=tlp_fn,\n step_size=0.1,\n num_leapfrog_steps=2)))\n log_normalizer_estimate = (tf.reduce_logsumexp(ais_weights)\n - np.log(num_chains))\n ```\n\n \"\"\"\n with tf.compat.v1.name_scope(name, \"sample_annealed_importance_chain\",\n [num_steps, current_state]):\n num_steps = tf.convert_to_tensor(\n value=num_steps, dtype=tf.int32, name=\"num_steps\")\n if mcmc_util.is_list_like(current_state):\n current_state = [\n tf.convert_to_tensor(value=s, name=\"current_state\")\n for s in current_state\n ]\n else:\n current_state = tf.convert_to_tensor(\n value=current_state, name=\"current_state\")\n\n def _make_convex_combined_log_prob_fn(iter_):\n def _fn(*args):\n p = tf.identity(proposal_log_prob_fn(*args), name=\"proposal_log_prob\")\n t = tf.identity(target_log_prob_fn(*args), name=\"target_log_prob\")\n dtype = p.dtype.base_dtype\n beta = tf.cast(iter_ + 1, dtype) / tf.cast(num_steps, dtype)\n return tf.identity(beta * t + (1. - beta) * p,\n name=\"convex_combined_log_prob\")\n return _fn\n\n def _loop_body(iter_, ais_weights, current_state, kernel_results):\n \"\"\"Closure which implements `tf.while_loop` body.\"\"\"\n x = (current_state if mcmc_util.is_list_like(current_state)\n else [current_state])\n proposal_log_prob = proposal_log_prob_fn(*x)\n target_log_prob = target_log_prob_fn(*x)\n ais_weights += ((target_log_prob - proposal_log_prob) /\n tf.cast(num_steps, ais_weights.dtype))\n kernel = make_kernel_fn(_make_convex_combined_log_prob_fn(iter_))\n next_state, inner_results = kernel.one_step(\n current_state, kernel_results.inner_results)\n kernel_results = AISResults(\n proposal_log_prob=proposal_log_prob,\n target_log_prob=target_log_prob,\n inner_results=inner_results,\n )\n return [iter_ + 1, ais_weights, next_state, kernel_results]\n\n def _bootstrap_results(init_state):\n \"\"\"Creates first version of `previous_kernel_results`.\"\"\"\n kernel = make_kernel_fn(_make_convex_combined_log_prob_fn(iter_=0))\n inner_results = kernel.bootstrap_results(init_state)\n\n convex_combined_log_prob = inner_results.accepted_results.target_log_prob\n dtype = convex_combined_log_prob.dtype.as_numpy_dtype\n shape = tf.shape(input=convex_combined_log_prob)\n proposal_log_prob = tf.fill(shape, dtype(np.nan),\n name=\"bootstrap_proposal_log_prob\")\n target_log_prob = tf.fill(shape, dtype(np.nan),\n name=\"target_target_log_prob\")\n\n return AISResults(\n proposal_log_prob=proposal_log_prob,\n target_log_prob=target_log_prob,\n inner_results=inner_results,\n )\n\n previous_kernel_results = _bootstrap_results(current_state)\n inner_results = previous_kernel_results.inner_results\n\n ais_weights = tf.zeros(\n shape=tf.broadcast_dynamic_shape(\n tf.shape(input=inner_results.proposed_results.target_log_prob),\n tf.shape(input=inner_results.accepted_results.target_log_prob)),\n dtype=inner_results.proposed_results.target_log_prob.dtype.base_dtype)\n\n [_, ais_weights, current_state, kernel_results] = tf.while_loop(\n cond=lambda iter_, *args: iter_ < num_steps,\n body=_loop_body,\n loop_vars=[\n np.int32(0), # iter_\n ais_weights,\n current_state,\n previous_kernel_results,\n ],\n parallel_iterations=parallel_iterations)\n\n return [current_state, ais_weights, kernel_results]", "response": "Runs an annealing from the given proposal_log_prob_fn target_log_prob_fn and current_state and makes a kernel function that returns the unbiased estimate of the ratio of the importance weights of the target distribution."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\ncreate a value - setting interceptor that sets the value of the Uniform and Uniform Un", "response": "def make_value_setter(**model_kwargs):\n \"\"\"Creates a value-setting interceptor.\n\n This function creates an interceptor that sets values of Edward2 random\n variable objects. This is useful for a range of tasks, including conditioning\n on observed data, sampling from posterior predictive distributions, and as a\n building block of inference primitives such as computing log joint\n probabilities (see examples below).\n\n Args:\n **model_kwargs: dict of str to Tensor. Keys are the names of random\n variables in the model to which this interceptor is being applied. Values\n are Tensors to set their value to. Variables not included in this dict\n will not be set and will maintain their existing value semantics (by\n default, a sample from the parent-conditional distribution).\n\n Returns:\n set_values: function that sets the value of intercepted ops.\n\n #### Examples\n\n Consider for illustration a model with latent `z` and\n observed `x`, and a corresponding trainable posterior model:\n\n ```python\n num_observations = 10\n def model():\n z = ed.Normal(loc=0, scale=1., name='z') # log rate\n x = ed.Poisson(rate=tf.exp(z) * tf.ones(num_observations), name='x')\n return x\n\n def variational_model():\n return ed.Normal(loc=tf.Variable(0.),\n scale=tf.nn.softplus(tf.Variable(-4.)),\n name='z') # for simplicity, match name of the model RV.\n ```\n\n We can use a value-setting interceptor to condition the model on observed\n data. This approach is slightly more cumbersome than that of partially\n evaluating the complete log-joint function, but has the potential advantage\n that it returns a new model callable, which may be used to sample downstream\n variables, passed into additional transformations, etc.\n\n ```python\n x_observed = np.array([6, 3, 1, 8, 7, 0, 6, 4, 7, 5])\n def observed_model():\n with ed.interception(make_value_setter(x=x_observed)):\n model()\n observed_log_joint_fn = ed.make_log_joint_fn(observed_model)\n\n # After fixing 'x', the observed log joint is now only a function of 'z'.\n # This enables us to define a variational lower bound,\n # `E_q[ log p(x, z) - log q(z)]`, simply by evaluating the observed and\n # variational log joints at variational samples.\n variational_log_joint_fn = ed.make_log_joint_fn(variational_model)\n with ed.tape() as variational_sample: # Sample trace from variational model.\n variational_model()\n elbo_loss = -(observed_log_joint_fn(**variational_sample) -\n variational_log_joint_fn(**variational_sample))\n ```\n\n After performing inference by minimizing the variational loss, a value-setting\n interceptor enables simulation from the posterior predictive distribution:\n\n ```python\n with ed.tape() as posterior_samples: # tape is a map {rv.name : rv}\n variational_model()\n with ed.interception(ed.make_value_setter(**posterior_samples)):\n x = model()\n # x is a sample from p(X | Z = z') where z' ~ q(z) (the variational model)\n ```\n\n As another example, using a value setter inside of `ed.tape` enables\n computing the log joint probability, by setting all variables to\n posterior values and then accumulating the log probs of those values under\n the induced parent-conditional distributions. This is one way that we could\n have implemented `ed.make_log_joint_fn`:\n\n ```python\n def make_log_joint_fn_demo(model):\n def log_joint_fn(**model_kwargs):\n with ed.tape() as model_tape:\n with ed.make_value_setter(**model_kwargs):\n model()\n\n # accumulate sum_i log p(X_i = x_i | X_{:i-1} = x_{:i-1})\n log_prob = 0.\n for rv in model_tape.values():\n log_prob += tf.reduce_sum(rv.log_prob(rv.value))\n\n return log_prob\n return log_joint_fn\n ```\n\n \"\"\"\n def set_values(f, *args, **kwargs):\n \"\"\"Sets random variable values to its aligned value.\"\"\"\n name = kwargs.get(\"name\")\n if name in model_kwargs:\n kwargs[\"value\"] = model_kwargs[name]\n return interceptable(f)(*args, **kwargs)\n return set_values"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\ntaking Edward probabilistic program and returns its log - joint function.", "response": "def make_log_joint_fn(model):\n \"\"\"Takes Edward probabilistic program and returns its log joint function.\n\n Args:\n model: Python callable which executes the generative process of a\n computable probability distribution using `ed.RandomVariable`s.\n\n Returns:\n A log-joint probability function. Its inputs are `model`'s original inputs\n and random variables which appear during the program execution. Its output\n is a scalar tf.Tensor.\n\n #### Examples\n\n Below we define Bayesian logistic regression as an Edward program,\n representing the model's generative process. We apply `make_log_joint_fn` in\n order to represent the model in terms of its joint probability function.\n\n ```python\n from tensorflow_probability import edward2 as ed\n\n def logistic_regression(features):\n coeffs = ed.Normal(loc=0., scale=1.,\n sample_shape=features.shape[1], name=\"coeffs\")\n outcomes = ed.Bernoulli(logits=tf.tensordot(features, coeffs, [[1], [0]]),\n name=\"outcomes\")\n return outcomes\n\n log_joint = ed.make_log_joint_fn(logistic_regression)\n\n features = tf.random_normal([3, 2])\n coeffs_value = tf.random_normal([2])\n outcomes_value = tf.round(tf.random_uniform([3]))\n output = log_joint(features, coeffs=coeffs_value, outcomes=outcomes_value)\n ```\n\n \"\"\"\n def log_joint_fn(*args, **kwargs):\n \"\"\"Log-probability of inputs according to a joint probability distribution.\n\n Args:\n *args: Positional arguments. They are the model's original inputs and can\n alternatively be specified as part of `kwargs`.\n **kwargs: Keyword arguments, where for each key-value pair `k` and `v`,\n `v` is passed as a `value` to the random variable(s) whose keyword\n argument `name` during construction is equal to `k`.\n\n Returns:\n Scalar tf.Tensor, which represents the model's log-probability summed\n over all Edward random variables and their dimensions.\n\n Raises:\n TypeError: If a random variable in the model has no specified value in\n `**kwargs`.\n \"\"\"\n log_probs = []\n\n def interceptor(rv_constructor, *rv_args, **rv_kwargs):\n \"\"\"Overrides a random variable's `value` and accumulates its log-prob.\"\"\"\n # Set value to keyword argument indexed by `name` (an input tensor).\n rv_name = rv_kwargs.get(\"name\")\n if rv_name is None:\n raise KeyError(\"Random variable constructor {} has no name \"\n \"in its arguments.\".format(rv_constructor.__name__))\n\n # If no value is explicitly passed in for an RV, default to the value\n # from the RV constructor. This may have been set explicitly by the user\n # or forwarded from a lower-level interceptor.\n previously_specified_value = rv_kwargs.get(\"value\")\n value = kwargs.get(rv_name, previously_specified_value)\n if value is None:\n raise LookupError(\"Keyword argument specifying value for {} is \"\n \"missing.\".format(rv_name))\n rv_kwargs[\"value\"] = value\n\n rv = rv_constructor(*rv_args, **rv_kwargs)\n log_prob = tf.reduce_sum(input_tensor=rv.distribution.log_prob(rv.value))\n log_probs.append(log_prob)\n return rv\n\n model_kwargs = _get_function_inputs(model, kwargs)\n with interception(interceptor):\n model(*args, **model_kwargs)\n log_prob = sum(log_probs)\n return log_prob\n return log_joint_fn"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _get_function_inputs(f, src_kwargs):\n if hasattr(f, \"_func\"): # functions returned by tf.make_template\n f = f._func # pylint: disable=protected-access\n\n try: # getargspec was deprecated in Python 3.6\n argspec = inspect.getfullargspec(f)\n except AttributeError:\n argspec = inspect.getargspec(f)\n\n fkwargs = {k: v for k, v in six.iteritems(src_kwargs) if k in argspec.args}\n return fkwargs", "response": "Filters inputs to be compatible with function f s signature."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nnetworking block for VGG.", "response": "def _vggconv_block(x, filters, kernel, stride, kernel_posterior_fn):\n \"\"\"Network block for VGG.\"\"\"\n out = tfp.layers.Convolution2DFlipout(\n filters,\n kernel,\n padding='same',\n kernel_posterior_fn=kernel_posterior_fn)(x)\n out = tf.keras.layers.BatchNormalization()(out)\n out = tf.keras.layers.Activation('relu')(out)\n\n out = tfp.layers.Convolution2DFlipout(\n filters,\n kernel,\n padding='same',\n kernel_posterior_fn=kernel_posterior_fn)(out)\n out = tf.keras.layers.BatchNormalization()(out)\n out = tf.keras.layers.Activation('relu')(out)\n\n out = tf.keras.layers.MaxPooling2D(\n pool_size=(2, 2), strides=stride)(out)\n return out"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef compute_output_shape(self, input_shape):\n input_shape = tf.TensorShape(input_shape)\n input_shape = input_shape.with_rank_at_least(2)\n if tf.compat.dimension_value(input_shape[-1]) is None:\n raise ValueError(\n 'The innermost dimension of `input_shape` must be defined, '\n 'but saw: {}'.format(input_shape))\n return input_shape[:-1].concatenate(self.units)", "response": "Computes the output shape of the layer."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef get_config(self):\n config = {\n 'units': self.units,\n 'activation': (tf.keras.activations.serialize(self.activation)\n if self.activation else None),\n 'activity_regularizer':\n tf.keras.initializers.serialize(self.activity_regularizer),\n }\n function_keys = [\n 'kernel_posterior_fn',\n 'kernel_posterior_tensor_fn',\n 'kernel_prior_fn',\n 'kernel_divergence_fn',\n 'bias_posterior_fn',\n 'bias_posterior_tensor_fn',\n 'bias_prior_fn',\n 'bias_divergence_fn',\n ]\n for function_key in function_keys:\n function = getattr(self, function_key)\n if function is None:\n function_name = None\n function_type = None\n else:\n function_name, function_type = tfp_layers_util.serialize_function(\n function)\n config[function_key] = function_name\n config[function_key + '_type'] = function_type\n base_config = super(_DenseVariational, self).get_config()\n return dict(list(base_config.items()) + list(config.items()))", "response": "Returns the config of the current layer."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nsimulate a No - U - Turn Sampler.", "response": "def kernel(target_log_prob_fn,\n current_state,\n step_size,\n seed=None,\n current_target_log_prob=None,\n current_grads_target_log_prob=None,\n name=None):\n \"\"\"Simulates a No-U-Turn Sampler (NUTS) trajectory.\n\n Args:\n target_log_prob_fn: Python callable which takes an argument like\n `*current_state` and returns its (possibly unnormalized) log-density under\n the target distribution.\n current_state: List of `Tensor`s representing the states to simulate from.\n step_size: List of `Tensor`s representing the step sizes for the leapfrog\n integrator. Must have same shape as `current_state`.\n seed: Integer to seed the random number generator.\n current_target_log_prob: Scalar `Tensor` representing the value of\n `target_log_prob_fn` at the `current_state`.\n current_grads_target_log_prob: List of `Tensor`s representing gradient of\n `current_target_log_prob` with respect to `current_state`. Must have same\n shape as `current_state`.\n name: A name for the operation.\n\n Returns:\n next_state: List of `Tensor`s representing the next states of the NUTS\n trajectory. Has same shape as `current_state`.\n next_target_log_prob: Scalar `Tensor` representing the value of\n `target_log_prob_fn` at `next_state`.\n next_grads_target_log_prob: List of `Tensor`s representing the gradient of\n `next_target_log_prob` with respect to `next_state`.\n\n Raises:\n NotImplementedError: If the execution mode is not eager.\n \"\"\"\n if not tf.executing_eagerly():\n raise NotImplementedError(\"`kernel` is only available in Eager mode.\")\n\n with tf.compat.v1.name_scope(\n name,\n default_name=\"nuts_kernel\",\n values=[\n current_state, step_size, seed, current_target_log_prob,\n current_grads_target_log_prob\n ]):\n with tf.compat.v1.name_scope(\"initialize\"):\n current_state = [tf.convert_to_tensor(value=s) for s in current_state]\n step_size = [tf.convert_to_tensor(value=s) for s in step_size]\n value_and_gradients_fn = lambda *args: tfp.math.value_and_gradient( # pylint: disable=g-long-lambda\n target_log_prob_fn, args)\n value_and_gradients_fn = _embed_no_none_gradient_check(\n value_and_gradients_fn)\n if (current_target_log_prob is None or\n current_grads_target_log_prob is None):\n (current_target_log_prob,\n current_grads_target_log_prob) = value_and_gradients_fn(*current_state)\n\n seed_stream = tfd.SeedStream(seed, \"nuts_kernel\")\n current_momentum = []\n for state_tensor in current_state:\n momentum_tensor = tf.random.normal(\n shape=tf.shape(input=state_tensor),\n dtype=state_tensor.dtype,\n seed=seed_stream())\n current_momentum.append(momentum_tensor)\n\n # Draw a slice variable u ~ Uniform(0, p(initial state, initial\n # momentum)) and compute log u. For numerical stability, we perform this\n # in log space where log u = log (u' * p(...)) = log u' + log\n # p(...) and u' ~ Uniform(0, 1).\n log_slice_sample = tf.math.log(tf.random.uniform([], seed=seed_stream()))\n log_slice_sample += _log_joint(current_target_log_prob,\n current_momentum)\n\n # Initialize loop variables. It comprises a collection of information\n # about a \"reverse\" state, a collection of information about a \"forward\"\n # state, a collection of information about the next state,\n # the trajectory's tree depth, the number of candidate states, and\n # whether to continue the trajectory.\n reverse_state = current_state\n reverse_target_log_prob = current_target_log_prob\n reverse_grads_target_log_prob = current_grads_target_log_prob\n reverse_momentum = current_momentum\n forward_state = current_state\n forward_target_log_prob = current_target_log_prob\n forward_grads_target_log_prob = current_grads_target_log_prob\n forward_momentum = current_momentum\n next_state = current_state\n next_target_log_prob = current_target_log_prob\n next_grads_target_log_prob = current_grads_target_log_prob\n depth = 0\n num_states = 1\n continue_trajectory = True\n\n while continue_trajectory:\n # Grow the No-U-Turn Sampler trajectory by choosing a random direction and\n # simulating Hamiltonian dynamics in that direction. This extends either\n # the forward or reverse state.\n direction = tfp.math.random_rademacher([], seed=seed_stream())\n if direction < 0:\n [\n reverse_state,\n reverse_target_log_prob,\n reverse_grads_target_log_prob,\n reverse_momentum,\n _,\n _,\n _,\n _,\n next_state_in_subtree,\n next_target_log_prob_in_subtree,\n next_grads_target_log_prob_in_subtree,\n num_states_in_subtree,\n continue_trajectory,\n ] = _build_tree(\n value_and_gradients_fn=value_and_gradients_fn,\n current_state=reverse_state,\n current_target_log_prob=reverse_target_log_prob,\n current_grads_target_log_prob=reverse_grads_target_log_prob,\n current_momentum=reverse_momentum,\n direction=direction,\n depth=depth,\n step_size=step_size,\n log_slice_sample=log_slice_sample,\n seed=seed_stream())\n else:\n [\n _,\n _,\n _,\n _,\n forward_state,\n forward_target_log_prob,\n forward_grads_target_log_prob,\n forward_momentum,\n next_state_in_subtree,\n next_target_log_prob_in_subtree,\n next_grads_target_log_prob_in_subtree,\n num_states_in_subtree,\n continue_trajectory,\n ] = _build_tree(\n value_and_gradients_fn=value_and_gradients_fn,\n current_state=forward_state,\n current_target_log_prob=forward_target_log_prob,\n current_grads_target_log_prob=forward_grads_target_log_prob,\n current_momentum=forward_momentum,\n direction=direction,\n depth=depth,\n step_size=step_size,\n log_slice_sample=log_slice_sample,\n seed=seed_stream())\n\n if continue_trajectory:\n # If the built tree did not terminate, accept the tree's next state\n # with a certain probability.\n accept_state_in_subtree = _random_bernoulli(\n [],\n probs=tf.minimum(1., num_states_in_subtree / num_states),\n dtype=tf.bool,\n seed=seed_stream())\n if accept_state_in_subtree:\n next_state = next_state_in_subtree\n next_target_log_prob = next_target_log_prob_in_subtree\n next_grads_target_log_prob = next_grads_target_log_prob_in_subtree\n\n # Continue the NUTS trajectory if the tree-building did not terminate, and\n # if the reverse-most and forward-most states do not exhibit a U-turn.\n has_no_u_turn = tf.logical_and(\n _has_no_u_turn(forward_state, reverse_state, forward_momentum),\n _has_no_u_turn(forward_state, reverse_state, reverse_momentum))\n continue_trajectory = continue_trajectory and has_no_u_turn\n num_states += num_states_in_subtree\n depth += 1\n\n return next_state, next_target_log_prob, next_grads_target_log_prob"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nbuild a tree at a given depth.", "response": "def _build_tree(value_and_gradients_fn,\n current_state,\n current_target_log_prob,\n current_grads_target_log_prob,\n current_momentum,\n direction,\n depth,\n step_size,\n log_slice_sample,\n max_simulation_error=1000.,\n seed=None):\n \"\"\"Builds a tree at a given tree depth and at a given state.\n\n The `current` state is immediately adjacent to, but outside of,\n the subtrajectory spanned by the returned `forward` and `reverse` states.\n\n Args:\n value_and_gradients_fn: Python callable which takes an argument like\n `*current_state` and returns a tuple of its (possibly unnormalized)\n log-density under the target distribution and its gradient with respect to\n each state.\n current_state: List of `Tensor`s representing the current states of the\n NUTS trajectory.\n current_target_log_prob: Scalar `Tensor` representing the value of\n `target_log_prob_fn` at the `current_state`.\n current_grads_target_log_prob: List of `Tensor`s representing gradient of\n `current_target_log_prob` with respect to `current_state`. Must have same\n shape as `current_state`.\n current_momentum: List of `Tensor`s representing the momentums of\n `current_state`. Must have same shape as `current_state`.\n direction: int that is either -1 or 1. It determines whether to perform\n leapfrog integration backwards (reverse) or forward in time respectively.\n depth: non-negative int that indicates how deep of a tree to build.\n Each call to `_build_tree` takes `2**depth` leapfrog steps.\n step_size: List of `Tensor`s representing the step sizes for the leapfrog\n integrator. Must have same shape as `current_state`.\n log_slice_sample: The log of an auxiliary slice variable. It is used\n together with `max_simulation_error` to avoid simulating trajectories with\n too much numerical error.\n max_simulation_error: Maximum simulation error to tolerate before\n terminating the trajectory. Simulation error is the\n `log_slice_sample` minus the log-joint probability at the simulated state.\n seed: Integer to seed the random number generator.\n\n Returns:\n reverse_state: List of `Tensor`s representing the \"reverse\" states of the\n NUTS trajectory. Has same shape as `current_state`.\n reverse_target_log_prob: Scalar `Tensor` representing the value of\n `target_log_prob_fn` at the `reverse_state`.\n reverse_grads_target_log_prob: List of `Tensor`s representing gradient of\n `reverse_target_log_prob` with respect to `reverse_state`. Has same shape\n as `reverse_state`.\n reverse_momentum: List of `Tensor`s representing the momentums of\n `reverse_state`. Has same shape as `reverse_state`.\n forward_state: List of `Tensor`s representing the \"forward\" states of the\n NUTS trajectory. Has same shape as `current_state`.\n forward_target_log_prob: Scalar `Tensor` representing the value of\n `target_log_prob_fn` at the `forward_state`.\n forward_grads_target_log_prob: List of `Tensor`s representing gradient of\n `forward_target_log_prob` with respect to `forward_state`. Has same shape\n as `forward_state`.\n forward_momentum: List of `Tensor`s representing the momentums of\n `forward_state`. Has same shape as `forward_state`.\n next_state: List of `Tensor`s representing the next states of the NUTS\n trajectory. Has same shape as `current_state`.\n next_target_log_prob: Scalar `Tensor` representing the value of\n `target_log_prob_fn` at `next_state`.\n next_grads_target_log_prob: List of `Tensor`s representing the gradient of\n `next_target_log_prob` with respect to `next_state`.\n num_states: Number of acceptable candidate states in the subtree. A state is\n acceptable if it is \"in the slice\", that is, if its log-joint probability\n with its momentum is greater than `log_slice_sample`.\n continue_trajectory: bool determining whether to continue the simulation\n trajectory. The trajectory is continued if no U-turns are encountered\n within the built subtree, and if the log-probability accumulation due to\n integration error does not exceed `max_simulation_error`.\n \"\"\"\n if depth == 0: # base case\n # Take a leapfrog step. Terminate the tree-building if the simulation\n # error from the leapfrog integrator is too large. States discovered by\n # continuing the simulation are likely to have very low probability.\n [\n next_state,\n next_target_log_prob,\n next_grads_target_log_prob,\n next_momentum,\n ] = _leapfrog(\n value_and_gradients_fn=value_and_gradients_fn,\n current_state=current_state,\n current_grads_target_log_prob=current_grads_target_log_prob,\n current_momentum=current_momentum,\n step_size=direction * step_size)\n next_log_joint = _log_joint(next_target_log_prob, next_momentum)\n num_states = tf.cast(next_log_joint > log_slice_sample, dtype=tf.int32)\n continue_trajectory = (next_log_joint >\n log_slice_sample - max_simulation_error)\n return [\n next_state,\n next_target_log_prob,\n next_grads_target_log_prob,\n next_momentum,\n next_state,\n next_target_log_prob,\n next_grads_target_log_prob,\n next_momentum,\n next_state,\n next_target_log_prob,\n next_grads_target_log_prob,\n num_states,\n continue_trajectory,\n ]\n\n # Build a tree at the current state.\n seed_stream = tfd.SeedStream(seed, \"build_tree\")\n [\n reverse_state,\n reverse_target_log_prob,\n reverse_grads_target_log_prob,\n reverse_momentum,\n forward_state,\n forward_target_log_prob,\n forward_grads_target_log_prob,\n forward_momentum,\n next_state,\n next_target_log_prob,\n next_grads_target_log_prob,\n num_states,\n continue_trajectory,\n ] = _build_tree(value_and_gradients_fn=value_and_gradients_fn,\n current_state=current_state,\n current_target_log_prob=current_target_log_prob,\n current_grads_target_log_prob=current_grads_target_log_prob,\n current_momentum=current_momentum,\n direction=direction,\n depth=depth - 1,\n step_size=step_size,\n log_slice_sample=log_slice_sample,\n seed=seed_stream())\n if continue_trajectory:\n # If the just-built subtree did not terminate, build a second subtree at\n # the forward or reverse state, as appropriate.\n if direction < 0:\n [\n reverse_state,\n reverse_target_log_prob,\n reverse_grads_target_log_prob,\n reverse_momentum,\n _,\n _,\n _,\n _,\n far_state,\n far_target_log_prob,\n far_grads_target_log_prob,\n far_num_states,\n far_continue_trajectory,\n ] = _build_tree(\n value_and_gradients_fn=value_and_gradients_fn,\n current_state=reverse_state,\n current_target_log_prob=reverse_target_log_prob,\n current_grads_target_log_prob=reverse_grads_target_log_prob,\n current_momentum=reverse_momentum,\n direction=direction,\n depth=depth - 1,\n step_size=step_size,\n log_slice_sample=log_slice_sample,\n seed=seed_stream())\n else:\n [\n _,\n _,\n _,\n _,\n forward_state,\n forward_target_log_prob,\n forward_grads_target_log_prob,\n forward_momentum,\n far_state,\n far_target_log_prob,\n far_grads_target_log_prob,\n far_num_states,\n far_continue_trajectory,\n ] = _build_tree(\n value_and_gradients_fn=value_and_gradients_fn,\n current_state=forward_state,\n current_target_log_prob=forward_target_log_prob,\n current_grads_target_log_prob=forward_grads_target_log_prob,\n current_momentum=forward_momentum,\n direction=direction,\n depth=depth - 1,\n step_size=step_size,\n log_slice_sample=log_slice_sample,\n seed=seed_stream())\n\n # Propose either `next_state` (which came from the first subtree and so is\n # nearby) or the new forward/reverse state (which came from the second\n # subtree and so is far away).\n num_states += far_num_states\n accept_far_state = _random_bernoulli(\n [],\n probs=far_num_states / num_states,\n dtype=tf.bool,\n seed=seed_stream())\n if accept_far_state:\n next_state = far_state\n next_target_log_prob = far_target_log_prob\n next_grads_target_log_prob = far_grads_target_log_prob\n\n # Continue the NUTS trajectory if the far subtree did not terminate either,\n # and if the reverse-most and forward-most states do not exhibit a U-turn.\n has_no_u_turn = tf.logical_and(\n _has_no_u_turn(forward_state, reverse_state, forward_momentum),\n _has_no_u_turn(forward_state, reverse_state, reverse_momentum))\n continue_trajectory = far_continue_trajectory and has_no_u_turn\n\n return [\n reverse_state,\n reverse_target_log_prob,\n reverse_grads_target_log_prob,\n reverse_momentum,\n forward_state,\n forward_target_log_prob,\n forward_grads_target_log_prob,\n forward_momentum,\n next_state,\n next_target_log_prob,\n next_grads_target_log_prob,\n num_states,\n continue_trajectory,\n ]"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nwrap value and gradients function to assist with None gradients.", "response": "def _embed_no_none_gradient_check(value_and_gradients_fn):\n \"\"\"Wraps value and gradients function to assist with None gradients.\"\"\"\n @functools.wraps(value_and_gradients_fn)\n def func_wrapped(*args, **kwargs):\n \"\"\"Wrapped function which checks for None gradients.\"\"\"\n value, grads = value_and_gradients_fn(*args, **kwargs)\n if any(grad is None for grad in grads):\n raise ValueError(\"Gradient is None for a state.\")\n return value, grads\n return func_wrapped"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nreturns True if two given states and momentum do not exhibit a U - turn pattern.", "response": "def _has_no_u_turn(state_one, state_two, momentum):\n \"\"\"If two given states and momentum do not exhibit a U-turn pattern.\"\"\"\n dot_product = sum([\n tf.reduce_sum(input_tensor=(s1 - s2) * m)\n for s1, s2, m in zip(state_one, state_two, momentum)\n ])\n return dot_product > 0"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nrunning one step of leapfrog integration.", "response": "def _leapfrog(value_and_gradients_fn,\n current_state,\n current_grads_target_log_prob,\n current_momentum,\n step_size):\n \"\"\"Runs one step of leapfrog integration.\"\"\"\n mid_momentum = [\n m + 0.5 * step * g for m, step, g in\n zip(current_momentum, step_size, current_grads_target_log_prob)]\n next_state = [\n s + step * m for s, step, m in\n zip(current_state, step_size, mid_momentum)]\n next_target_log_prob, next_grads_target_log_prob = value_and_gradients_fn(\n *next_state)\n next_momentum = [\n m + 0.5 * step * g for m, step, g in\n zip(mid_momentum, step_size, next_grads_target_log_prob)]\n return [\n next_state,\n next_target_log_prob,\n next_grads_target_log_prob,\n next_momentum,\n ]"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nlog - joint probability given a state s log - probability and momentum.", "response": "def _log_joint(current_target_log_prob, current_momentum):\n \"\"\"Log-joint probability given a state's log-probability and momentum.\"\"\"\n momentum_log_prob = -sum(\n [tf.reduce_sum(input_tensor=0.5 * (m**2.)) for m in current_momentum])\n return current_target_log_prob + momentum_log_prob"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nreturn samples from a Bernoulli distribution.", "response": "def _random_bernoulli(shape, probs, dtype=tf.int32, seed=None, name=None):\n \"\"\"Returns samples from a Bernoulli distribution.\"\"\"\n with tf.compat.v1.name_scope(name, \"random_bernoulli\", [shape, probs]):\n probs = tf.convert_to_tensor(value=probs)\n random_uniform = tf.random.uniform(shape, dtype=probs.dtype, seed=seed)\n return tf.cast(tf.less(random_uniform, probs), dtype)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef default_loc_scale_fn(\n is_singular=False,\n loc_initializer=tf.compat.v1.initializers.random_normal(stddev=0.1),\n untransformed_scale_initializer=tf.compat.v1.initializers.random_normal(\n mean=-3., stddev=0.1),\n loc_regularizer=None,\n untransformed_scale_regularizer=None,\n loc_constraint=None,\n untransformed_scale_constraint=None):\n \"\"\"Makes closure which creates `loc`, `scale` params from `tf.get_variable`.\n\n This function produces a closure which produces `loc`, `scale` using\n `tf.get_variable`. The closure accepts the following arguments:\n\n dtype: Type of parameter's event.\n shape: Python `list`-like representing the parameter's event shape.\n name: Python `str` name prepended to any created (or existing)\n `tf.Variable`s.\n trainable: Python `bool` indicating all created `tf.Variable`s should be\n added to the graph collection `GraphKeys.TRAINABLE_VARIABLES`.\n add_variable_fn: `tf.get_variable`-like `callable` used to create (or\n access existing) `tf.Variable`s.\n\n Args:\n is_singular: Python `bool` indicating if `scale is None`. Default: `False`.\n loc_initializer: Initializer function for the `loc` parameters.\n The default is `tf.random_normal_initializer(mean=0., stddev=0.1)`.\n untransformed_scale_initializer: Initializer function for the `scale`\n parameters. Default value: `tf.random_normal_initializer(mean=-3.,\n stddev=0.1)`. This implies the softplus transformed result is initialized\n near `0`. It allows a `Normal` distribution with `scale` parameter set to\n this value to approximately act like a point mass.\n loc_regularizer: Regularizer function for the `loc` parameters.\n The default (`None`) is to use the `tf.get_variable` default.\n untransformed_scale_regularizer: Regularizer function for the `scale`\n parameters. The default (`None`) is to use the `tf.get_variable` default.\n loc_constraint: An optional projection function to be applied to the\n loc after being updated by an `Optimizer`. The function must take as input\n the unprojected variable and must return the projected variable (which\n must have the same shape). Constraints are not safe to use when doing\n asynchronous distributed training.\n The default (`None`) is to use the `tf.get_variable` default.\n untransformed_scale_constraint: An optional projection function to be\n applied to the `scale` parameters after being updated by an `Optimizer`\n (e.g. used to implement norm constraints or value constraints). The\n function must take as input the unprojected variable and must return the\n projected variable (which must have the same shape). Constraints are not\n safe to use when doing asynchronous distributed training. The default\n (`None`) is to use the `tf.get_variable` default.\n\n Returns:\n default_loc_scale_fn: Python `callable` which instantiates `loc`, `scale`\n parameters from args: `dtype, shape, name, trainable, add_variable_fn`.\n \"\"\"\n def _fn(dtype, shape, name, trainable, add_variable_fn):\n \"\"\"Creates `loc`, `scale` parameters.\"\"\"\n loc = add_variable_fn(\n name=name + '_loc',\n shape=shape,\n initializer=loc_initializer,\n regularizer=loc_regularizer,\n constraint=loc_constraint,\n dtype=dtype,\n trainable=trainable)\n if is_singular:\n return loc, None\n untransformed_scale = add_variable_fn(\n name=name + '_untransformed_scale',\n shape=shape,\n initializer=untransformed_scale_initializer,\n regularizer=untransformed_scale_regularizer,\n constraint=untransformed_scale_constraint,\n dtype=dtype,\n trainable=trainable)\n scale = (np.finfo(dtype.as_numpy_dtype).eps +\n tf.nn.softplus(untransformed_scale))\n return loc, scale\n return _fn", "response": "Returns a closure which creates loc scale params from tf. get_variable."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef default_mean_field_normal_fn(\n is_singular=False,\n loc_initializer=tf.compat.v1.initializers.random_normal(stddev=0.1),\n untransformed_scale_initializer=tf.compat.v1.initializers.random_normal(\n mean=-3., stddev=0.1),\n loc_regularizer=None,\n untransformed_scale_regularizer=None,\n loc_constraint=None,\n untransformed_scale_constraint=None):\n \"\"\"Creates a function to build Normal distributions with trainable params.\n\n This function produces a closure which produces `tfd.Normal`\n parameterized by a loc` and `scale` each created using `tf.get_variable`.\n\n Args:\n is_singular: Python `bool` if `True`, forces the special case limit of\n `scale->0`, i.e., a `Deterministic` distribution.\n loc_initializer: Initializer function for the `loc` parameters.\n The default is `tf.random_normal_initializer(mean=0., stddev=0.1)`.\n untransformed_scale_initializer: Initializer function for the `scale`\n parameters. Default value: `tf.random_normal_initializer(mean=-3.,\n stddev=0.1)`. This implies the softplus transformed result is initialized\n near `0`. It allows a `Normal` distribution with `scale` parameter set to\n this value to approximately act like a point mass.\n loc_regularizer: Regularizer function for the `loc` parameters.\n untransformed_scale_regularizer: Regularizer function for the `scale`\n parameters.\n loc_constraint: An optional projection function to be applied to the\n loc after being updated by an `Optimizer`. The function must take as input\n the unprojected variable and must return the projected variable (which\n must have the same shape). Constraints are not safe to use when doing\n asynchronous distributed training.\n untransformed_scale_constraint: An optional projection function to be\n applied to the `scale` parameters after being updated by an `Optimizer`\n (e.g. used to implement norm constraints or value constraints). The\n function must take as input the unprojected variable and must return the\n projected variable (which must have the same shape). Constraints are not\n safe to use when doing asynchronous distributed training.\n\n Returns:\n make_normal_fn: Python `callable` which creates a `tfd.Normal`\n using from args: `dtype, shape, name, trainable, add_variable_fn`.\n \"\"\"\n loc_scale_fn = default_loc_scale_fn(\n is_singular=is_singular,\n loc_initializer=loc_initializer,\n untransformed_scale_initializer=untransformed_scale_initializer,\n loc_regularizer=loc_regularizer,\n untransformed_scale_regularizer=untransformed_scale_regularizer,\n loc_constraint=loc_constraint,\n untransformed_scale_constraint=untransformed_scale_constraint)\n def _fn(dtype, shape, name, trainable, add_variable_fn):\n \"\"\"Creates multivariate `Deterministic` or `Normal` distribution.\n\n Args:\n dtype: Type of parameter's event.\n shape: Python `list`-like representing the parameter's event shape.\n name: Python `str` name prepended to any created (or existing)\n `tf.Variable`s.\n trainable: Python `bool` indicating all created `tf.Variable`s should be\n added to the graph collection `GraphKeys.TRAINABLE_VARIABLES`.\n add_variable_fn: `tf.get_variable`-like `callable` used to create (or\n access existing) `tf.Variable`s.\n\n Returns:\n Multivariate `Deterministic` or `Normal` distribution.\n \"\"\"\n loc, scale = loc_scale_fn(dtype, shape, name, trainable, add_variable_fn)\n if scale is None:\n dist = tfd.Deterministic(loc=loc)\n else:\n dist = tfd.Normal(loc=loc, scale=scale)\n batch_ndims = tf.size(input=dist.batch_shape_tensor())\n return tfd.Independent(dist, reinterpreted_batch_ndims=batch_ndims)\n return _fn", "response": "Returns a function to build a normal distribution with trainable params."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\ndefaults multivariate standard normal distribution.", "response": "def default_multivariate_normal_fn(dtype, shape, name, trainable,\n add_variable_fn):\n \"\"\"Creates multivariate standard `Normal` distribution.\n\n Args:\n dtype: Type of parameter's event.\n shape: Python `list`-like representing the parameter's event shape.\n name: Python `str` name prepended to any created (or existing)\n `tf.Variable`s.\n trainable: Python `bool` indicating all created `tf.Variable`s should be\n added to the graph collection `GraphKeys.TRAINABLE_VARIABLES`.\n add_variable_fn: `tf.get_variable`-like `callable` used to create (or\n access existing) `tf.Variable`s.\n\n Returns:\n Multivariate standard `Normal` distribution.\n \"\"\"\n del name, trainable, add_variable_fn # unused\n dist = tfd.Normal(loc=tf.zeros(shape, dtype), scale=dtype.as_numpy_dtype(1))\n batch_ndims = tf.size(input=dist.batch_shape_tensor())\n return tfd.Independent(dist, reinterpreted_batch_ndims=batch_ndims)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef serialize_function(func):\n if isinstance(func, types.LambdaType):\n return generic_utils.func_dump(func), 'lambda'\n return func.__name__, 'function'", "response": "Serializes a Python function to be serialized by Keras."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef broadcast_structure(to_structure, from_structure):\n from_parts = tf.nest.flatten(from_structure)\n if len(from_parts) == 1:\n from_structure = tf.nest.map_structure(lambda _: from_parts[0],\n to_structure)\n return from_structure", "response": "Broadcasts from_structure to to_structure."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef expand_as_args(args):\n return (isinstance(args, collections.Sequence) and\n not _is_namedtuple(args) and not _force_leaf(args))", "response": "Returns True if args should be expanded as args."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nconvert the given arguments to Tensor s.", "response": "def convert_args_to_tensor(args, dtype=None, name=None):\n \"\"\"Converts `args` to `Tensor`s.\n\n Use this when it is necessary to convert user-provided arguments that will\n then be passed to user-provided callables.\n\n When `dtype` is `None` this function behaves as follows:\n\n 1A. If the top-level structure is a `list`/`tuple` but not a `namedtuple`,\n then it is left as is and only its elements are converted to `Tensor`s.\n\n 2A. The sub-structures are converted to `Tensor`s eagerly. E.g. if `args` is\n `{'arg': [[1], [2]]}` it is converted to\n `{'arg': tf.constant([[1], [2]])}`. If the conversion fails, it will\n attempt to recurse into its children.\n\n When `dtype` is specified, it acts as both a structural and numeric type\n constraint. `dtype` can be a single `DType`, `None` or a nested collection\n thereof. The conversion rule becomes as follows:\n\n 1B. The return value of this function will have the same structure as `dtype`.\n\n 2B. If the leaf of `dtype` is a concrete `DType`, then the corresponding\n sub-structure in `args` is converted to a `Tensor`.\n\n 3B. If the leaf of `dtype` is `None`, then the corresponding sub-structure is\n converted eagerly as described in the rule 2A above.\n\n Args:\n args: Arguments to convert to `Tensor`s.\n dtype: Optional structure/numeric type constraint.\n name: Optional name-scope to use.\n\n Returns:\n args: Converted `args`.\n\n #### Examples.\n\n This table shows some useful conversion cases. `T` means `Tensor`, `NT` means\n `namedtuple` and `CNT` means a `namedtuple` with a `Tensor`-conversion\n function registered.\n\n | args | dtype | output |\n |:------------:|:----------:|:------------------:|\n | `{\"a\": 1}` | `None` | `{\"a\": T(1)}` |\n | `T(1)` | `None` | `T(1)` |\n | `[1]` | `None` | `[T(1)]` |\n | `[1]` | `tf.int32` | `T([1])` |\n | `[[T(1)]]` | `None` | `[T([1])]` |\n | `[[T(1)]]` | `[[None]]` | `[[T(1)]]` |\n | `NT(1, 2)` | `None` | `NT(T(1), T(2))` |\n | `NT(1, 2)` | `tf.int32` | `T([1, 2])` |\n | `CNT(1, 2)` | `None` | `T(...)` |\n | `[[1, [2]]]` | `None` | `[[T(1), T([2])]]` |\n\n \"\"\"\n if dtype is None:\n if expand_as_args(args) or _expand_as_kwargs(args):\n shallow_args = _get_shallow_structure(args)\n return nest.map_structure_up_to(\n shallow_args, lambda s: _nested_convert_to_tensor(s, name=name), args)\n else:\n return _nested_convert_to_tensor(args, name=name)\n else:\n return nest.map_structure_up_to(\n dtype, lambda s, dtype: _nested_convert_to_tensor(s, dtype, name), args,\n dtype)"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef call_fn(fn, args):\n\n if expand_as_args(args):\n return fn(*args)\n elif _expand_as_kwargs(args):\n return fn(**args)\n else:\n return fn(args)", "response": "Calls fn with args possibly expanding args."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _wrap_method(cls, attr):\n fn = getattr(cls, attr)\n is_property = isinstance(fn, property)\n if is_property:\n fn = fn.fget\n @functools.wraps(fn)\n def wrapped(self, *args, **kwargs):\n return fn(self._value(), *args, **kwargs) # pylint: disable=protected-access\n return property(wrapped) if is_property else wrapped", "response": "Wraps a method in a dependency_injected_function that will be called when a Tensor member function is used to take existing\n."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef _get_tensor_like_attributes():\n # Enable \"Tensor semantics\" for distributions.\n # See tensorflow/python/framework/ops.py `class Tensor` for details.\n attrs = dict()\n # Setup overloadable operators and white-listed members / properties.\n attrs.update((attr, _wrap_method(tf.Tensor, attr))\n for attr in tf.Tensor.OVERLOADABLE_OPERATORS.union({'__iter__'}))\n # Copy some members straight-through.\n attrs.update((attr, getattr(tf.Tensor, attr))\n for attr in {'__nonzero__', '__bool__', '__array_priority__'})\n return attrs", "response": "Returns a dictionary of Tensor attributes related to shape and Python builtins."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\ngets the value of the current object in the cache.", "response": "def _value(self, dtype=None, name=None, as_ref=False): # pylint: disable=g-doc-args\n \"\"\"Get the value returned by `tf.convert_to_tensor(distribution)`.\n\n Note: this function may mutate the distribution instance state by caching\n the concretized `Tensor` value.\n\n Args:\n dtype: Must return a `Tensor` with the given `dtype` if specified.\n name: If the conversion function creates a new `Tensor`, it should use the\n given `name` if specified.\n as_ref: `as_ref` is true, the function must return a `Tensor` reference,\n such as a `Variable`.\n Returns:\n concretized_distribution_value: `Tensor` identical to\n `tf.convert_to_tensor(distribution)`.\n\n #### Examples\n\n ```python\n tfd = tfp.distributions\n x = tfd.Normal(0.5, 1).set_tensor_conversion(tfd.Distribution.mean)\n\n x._value()\n # ==> tf.convert_to_tensor(x) ==> 0.5\n\n x._value() + 2\n # ==> tf.convert_to_tensor(x) + 2. ==> 2.5\n\n x + 2\n # ==> tf.convert_to_tensor(x) + 2. ==> 2.5\n ```\n\n \"\"\"\n # pylint: disable=protected-access\n if as_ref:\n raise NotImplementedError(\n 'Cannot convert a `Distribution` to a reference '\n '(e.g., `tf.Variable`).')\n if self._concrete_value is None:\n if self._convert_to_tensor_fn is None:\n raise NotImplementedError(\n 'Failed to convert object of type {} to Tensor. Contents: {}. '\n 'Call `distribution.set_tensor_conversion(lambda self: ...)` to '\n 'enable `tf.convert_to_tensor` capability. For example: '\n '`x = tfd.Normal(0,1).set_tensor_conversion(tfd.Distribution.mean)`'\n ' results in `tf.convert_to_tensor(x)` being identical to '\n '`x.mean()`.'.format(type(self), self))\n with self._name_scope('value'):\n self._concrete_value = (self._convert_to_tensor_fn(self)\n if callable(self._convert_to_tensor_fn)\n else self._convert_to_tensor_fn)\n if not tf.is_tensor(self._concrete_value):\n self._concrete_value = tfd._convert_to_tensor(\n value=self._concrete_value,\n name=name or 'concrete_value',\n dtype=dtype,\n dtype_hint=self.dtype)\n return self._concrete_value"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\ncreate the encoder function.", "response": "def make_encoder(activation, latent_size, base_depth):\n \"\"\"Creates the encoder function.\n\n Args:\n activation: Activation function in hidden layers.\n latent_size: The dimensionality of the encoding.\n base_depth: The lowest depth for a layer.\n\n Returns:\n encoder: A `callable` mapping a `Tensor` of images to a\n `tfd.Distribution` instance over encodings.\n \"\"\"\n conv = functools.partial(\n tf.keras.layers.Conv2D, padding=\"SAME\", activation=activation)\n\n encoder_net = tf.keras.Sequential([\n conv(base_depth, 5, 1),\n conv(base_depth, 5, 2),\n conv(2 * base_depth, 5, 1),\n conv(2 * base_depth, 5, 2),\n conv(4 * latent_size, 7, padding=\"VALID\"),\n tf.keras.layers.Flatten(),\n tf.keras.layers.Dense(2 * latent_size, activation=None),\n ])\n\n def encoder(images):\n images = 2 * tf.cast(images, dtype=tf.float32) - 1\n net = encoder_net(images)\n return tfd.MultivariateNormalDiag(\n loc=net[..., :latent_size],\n scale_diag=tf.nn.softplus(net[..., latent_size:] +\n _softplus_inverse(1.0)),\n name=\"code\")\n\n return encoder"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef make_decoder(activation, latent_size, output_shape, base_depth):\n deconv = functools.partial(\n tf.keras.layers.Conv2DTranspose, padding=\"SAME\", activation=activation)\n conv = functools.partial(\n tf.keras.layers.Conv2D, padding=\"SAME\", activation=activation)\n\n decoder_net = tf.keras.Sequential([\n deconv(2 * base_depth, 7, padding=\"VALID\"),\n deconv(2 * base_depth, 5),\n deconv(2 * base_depth, 5, 2),\n deconv(base_depth, 5),\n deconv(base_depth, 5, 2),\n deconv(base_depth, 5),\n conv(output_shape[-1], 5, activation=None),\n ])\n\n def decoder(codes):\n original_shape = tf.shape(input=codes)\n # Collapse the sample and batch dimension and convert to rank-4 tensor for\n # use with a convolutional decoder network.\n codes = tf.reshape(codes, (-1, 1, 1, latent_size))\n logits = decoder_net(codes)\n logits = tf.reshape(\n logits, shape=tf.concat([original_shape[:-1], output_shape], axis=0))\n return tfd.Independent(tfd.Bernoulli(logits=logits),\n reinterpreted_batch_ndims=len(output_shape),\n name=\"image\")\n\n return decoder", "response": "Creates the decoder function."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\ncreate the mixture of Gaussians prior distribution.", "response": "def make_mixture_prior(latent_size, mixture_components):\n \"\"\"Creates the mixture of Gaussians prior distribution.\n\n Args:\n latent_size: The dimensionality of the latent representation.\n mixture_components: Number of elements of the mixture.\n\n Returns:\n random_prior: A `tfd.Distribution` instance representing the distribution\n over encodings in the absence of any evidence.\n \"\"\"\n if mixture_components == 1:\n # See the module docstring for why we don't learn the parameters here.\n return tfd.MultivariateNormalDiag(\n loc=tf.zeros([latent_size]),\n scale_identity_multiplier=1.0)\n\n loc = tf.compat.v1.get_variable(\n name=\"loc\", shape=[mixture_components, latent_size])\n raw_scale_diag = tf.compat.v1.get_variable(\n name=\"raw_scale_diag\", shape=[mixture_components, latent_size])\n mixture_logits = tf.compat.v1.get_variable(\n name=\"mixture_logits\", shape=[mixture_components])\n\n return tfd.MixtureSameFamily(\n components_distribution=tfd.MultivariateNormalDiag(\n loc=loc,\n scale_diag=tf.nn.softplus(raw_scale_diag)),\n mixture_distribution=tfd.Categorical(logits=mixture_logits),\n name=\"prior\")"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef pack_images(images, rows, cols):\n shape = tf.shape(input=images)\n width = shape[-3]\n height = shape[-2]\n depth = shape[-1]\n images = tf.reshape(images, (-1, width, height, depth))\n batch = tf.shape(input=images)[0]\n rows = tf.minimum(rows, batch)\n cols = tf.minimum(batch // rows, cols)\n images = images[:rows * cols]\n images = tf.reshape(images, (rows, cols, width, height, depth))\n images = tf.transpose(a=images, perm=[0, 2, 1, 3, 4])\n images = tf.reshape(images, [1, rows * width, cols * height, depth])\n return images", "response": "Helper utility to make a field of images."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef model_fn(features, labels, mode, params, config):\n del labels, config\n\n if params[\"analytic_kl\"] and params[\"mixture_components\"] != 1:\n raise NotImplementedError(\n \"Using `analytic_kl` is only supported when `mixture_components = 1` \"\n \"since there's no closed form otherwise.\")\n\n encoder = make_encoder(params[\"activation\"],\n params[\"latent_size\"],\n params[\"base_depth\"])\n decoder = make_decoder(params[\"activation\"],\n params[\"latent_size\"],\n IMAGE_SHAPE,\n params[\"base_depth\"])\n latent_prior = make_mixture_prior(params[\"latent_size\"],\n params[\"mixture_components\"])\n\n image_tile_summary(\n \"input\", tf.cast(features, dtype=tf.float32), rows=1, cols=16)\n\n approx_posterior = encoder(features)\n approx_posterior_sample = approx_posterior.sample(params[\"n_samples\"])\n decoder_likelihood = decoder(approx_posterior_sample)\n image_tile_summary(\n \"recon/sample\",\n tf.cast(decoder_likelihood.sample()[:3, :16], dtype=tf.float32),\n rows=3,\n cols=16)\n image_tile_summary(\n \"recon/mean\",\n decoder_likelihood.mean()[:3, :16],\n rows=3,\n cols=16)\n\n # `distortion` is just the negative log likelihood.\n distortion = -decoder_likelihood.log_prob(features)\n avg_distortion = tf.reduce_mean(input_tensor=distortion)\n tf.compat.v1.summary.scalar(\"distortion\", avg_distortion)\n\n if params[\"analytic_kl\"]:\n rate = tfd.kl_divergence(approx_posterior, latent_prior)\n else:\n rate = (approx_posterior.log_prob(approx_posterior_sample)\n - latent_prior.log_prob(approx_posterior_sample))\n avg_rate = tf.reduce_mean(input_tensor=rate)\n tf.compat.v1.summary.scalar(\"rate\", avg_rate)\n\n elbo_local = -(rate + distortion)\n\n elbo = tf.reduce_mean(input_tensor=elbo_local)\n loss = -elbo\n tf.compat.v1.summary.scalar(\"elbo\", elbo)\n\n importance_weighted_elbo = tf.reduce_mean(\n input_tensor=tf.reduce_logsumexp(input_tensor=elbo_local, axis=0) -\n tf.math.log(tf.cast(params[\"n_samples\"], dtype=tf.float32)))\n tf.compat.v1.summary.scalar(\"elbo/importance_weighted\",\n importance_weighted_elbo)\n\n # Decode samples from the prior for visualization.\n random_image = decoder(latent_prior.sample(16))\n image_tile_summary(\n \"random/sample\",\n tf.cast(random_image.sample(), dtype=tf.float32),\n rows=4,\n cols=4)\n image_tile_summary(\"random/mean\", random_image.mean(), rows=4, cols=4)\n\n # Perform variational inference by minimizing the -ELBO.\n global_step = tf.compat.v1.train.get_or_create_global_step()\n learning_rate = tf.compat.v1.train.cosine_decay(\n params[\"learning_rate\"], global_step, params[\"max_steps\"])\n tf.compat.v1.summary.scalar(\"learning_rate\", learning_rate)\n optimizer = tf.compat.v1.train.AdamOptimizer(learning_rate)\n train_op = optimizer.minimize(loss, global_step=global_step)\n\n return tf.estimator.EstimatorSpec(\n mode=mode,\n loss=loss,\n train_op=train_op,\n eval_metric_ops={\n \"elbo\":\n tf.compat.v1.metrics.mean(elbo),\n \"elbo/importance_weighted\":\n tf.compat.v1.metrics.mean(importance_weighted_elbo),\n \"rate\":\n tf.compat.v1.metrics.mean(avg_rate),\n \"distortion\":\n tf.compat.v1.metrics.mean(avg_distortion),\n },\n )", "response": "Builds the model function for use in an estimator."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\ndownloads a file from a directory.", "response": "def download(directory, filename):\n \"\"\"Downloads a file.\"\"\"\n filepath = os.path.join(directory, filename)\n if tf.io.gfile.exists(filepath):\n return filepath\n if not tf.io.gfile.exists(directory):\n tf.io.gfile.makedirs(directory)\n url = os.path.join(ROOT_PATH, filename)\n print(\"Downloading %s to %s\" % (url, filepath))\n urllib.request.urlretrieve(url, filepath)\n return filepath"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef build_fake_input_fns(batch_size):\n random_sample = np.random.rand(batch_size, *IMAGE_SHAPE).astype(\"float32\")\n\n def train_input_fn():\n dataset = tf.data.Dataset.from_tensor_slices(\n random_sample).map(lambda row: (row, 0)).batch(batch_size).repeat()\n return tf.compat.v1.data.make_one_shot_iterator(dataset).get_next()\n\n def eval_input_fn():\n dataset = tf.data.Dataset.from_tensor_slices(\n random_sample).map(lambda row: (row, 0)).batch(batch_size)\n return tf.compat.v1.data.make_one_shot_iterator(dataset).get_next()\n\n return train_input_fn, eval_input_fn", "response": "Builds fake MNIST - style data for unit testing."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef build_input_fns(data_dir, batch_size):\n\n # Build an iterator over training batches.\n def train_input_fn():\n dataset = static_mnist_dataset(data_dir, \"train\")\n dataset = dataset.shuffle(50000).repeat().batch(batch_size)\n return tf.compat.v1.data.make_one_shot_iterator(dataset).get_next()\n\n # Build an iterator over the heldout set.\n def eval_input_fn():\n eval_dataset = static_mnist_dataset(data_dir, \"valid\")\n eval_dataset = eval_dataset.batch(batch_size)\n return tf.compat.v1.data.make_one_shot_iterator(eval_dataset).get_next()\n\n return train_input_fn, eval_input_fn", "response": "Builds an iterator switching between train and heldout data."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nverifies that parts don t broadcast.", "response": "def maybe_check_wont_broadcast(flat_xs, validate_args):\n \"\"\"Verifies that `parts` don't broadcast.\"\"\"\n flat_xs = tuple(flat_xs) # So we can receive generators.\n if not validate_args:\n # Note: we don't try static validation because it is theoretically\n # possible that a user wants to take advantage of broadcasting.\n # Only when `validate_args` is `True` do we enforce the validation.\n return flat_xs\n msg = 'Broadcasting probably indicates an error in model specification.'\n s = tuple(x.shape for x in flat_xs)\n if all(tensorshape_util.is_fully_defined(s_) for s_ in s):\n if not all(a == b for a, b in zip(s[1:], s[:-1])):\n raise ValueError(msg)\n return flat_xs\n assertions = [assert_util.assert_equal(a, b, message=msg)\n for a, b in zip(s[1:], s[:-1])]\n with tf.control_dependencies(assertions):\n return tuple(tf.identity(x) for x in flat_xs)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef softplus_and_shift(x, shift=1e-5, name=None):\n with tf.compat.v1.name_scope(name, 'softplus_and_shift', [x, shift]):\n x = tf.convert_to_tensor(value=x, name='x')\n y = tf.nn.softplus(x)\n if shift is not None:\n y += shift\n return y", "response": "Converts a batch of float - like tensors to positive valued scalars."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nconverting x to batch of lower - triangular matrices with positive diagonal diagonal .", "response": "def tril_with_diag_softplus_and_shift(x, diag_shift=1e-5, name=None):\n \"\"\"Converts (batch of) vectors to (batch of) lower-triangular scale matrices.\n\n Args:\n x: (Batch of) `float`-like `Tensor` representing vectors which will be\n transformed into lower-triangular scale matrices with positive diagonal\n elements. Rightmost shape `n` must be such that\n `n = dims * (dims + 1) / 2` for some positive, integer `dims`.\n diag_shift: `Tensor` added to `softplus` transformation of diagonal\n elements.\n Default value: `1e-5`.\n name: A `name_scope` name for operations created by this function.\n Default value: `None` (i.e., \"tril_with_diag_softplus_and_shift\").\n\n Returns:\n scale_tril: (Batch of) lower-triangular `Tensor` with `x.dtype` and\n rightmost shape `[dims, dims]` where `n = dims * (dims + 1) / 2` where\n `n = x.shape[-1]`.\n \"\"\"\n with tf.compat.v1.name_scope(name, 'tril_with_diag_softplus_and_shift',\n [x, diag_shift]):\n x = tf.convert_to_tensor(value=x, name='x')\n x = tfd.fill_triangular(x)\n diag = softplus_and_shift(tf.linalg.diag_part(x), diag_shift)\n x = tf.linalg.set_diag(x, diag)\n return x"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nconstruct a trainable multivariate normal triangular distribution.", "response": "def multivariate_normal_tril(x,\n dims,\n layer_fn=tf.compat.v1.layers.dense,\n loc_fn=lambda x: x,\n scale_fn=tril_with_diag_softplus_and_shift,\n name=None):\n \"\"\"Constructs a trainable `tfd.MultivariateNormalTriL` distribution.\n\n This function creates a MultivariateNormal (MVN) with lower-triangular scale\n matrix. By default the MVN is parameterized via affine transformation of input\n tensor `x`. Using default args, this function is mathematically equivalent to:\n\n ```none\n Y = MVN(loc=matmul(W, x) + b,\n scale_tril=f(reshape_tril(matmul(M, x) + c)))\n\n where,\n W in R^[d, n]\n M in R^[d*(d+1)/2, n]\n b in R^d\n c in R^d\n f(S) = set_diag(S, softplus(matrix_diag_part(S)) + 1e-5)\n ```\n\n Observe that `f` makes the diagonal of the triangular-lower scale matrix be\n positive and no smaller than `1e-5`.\n\n #### Examples\n\n ```python\n # This example fits a multilinear regression loss.\n import tensorflow as tf\n import tensorflow_probability as tfp\n\n # Create fictitious training data.\n dtype = np.float32\n n = 3000 # number of samples\n x_size = 4 # size of single x\n y_size = 2 # size of single y\n def make_training_data():\n np.random.seed(142)\n x = np.random.randn(n, x_size).astype(dtype)\n w = np.random.randn(x_size, y_size).astype(dtype)\n b = np.random.randn(1, y_size).astype(dtype)\n true_mean = np.tensordot(x, w, axes=[[-1], [0]]) + b\n noise = np.random.randn(n, y_size).astype(dtype)\n y = true_mean + noise\n return y, x\n y, x = make_training_data()\n\n # Build TF graph for fitting MVNTriL maximum likelihood estimator.\n mvn = tfp.trainable_distributions.multivariate_normal_tril(x, dims=y_size)\n loss = -tf.reduce_mean(mvn.log_prob(y))\n train_op = tf.train.AdamOptimizer(learning_rate=2.**-3).minimize(loss)\n mse = tf.reduce_mean(tf.squared_difference(y, mvn.mean()))\n init_op = tf.global_variables_initializer()\n\n # Run graph 1000 times.\n num_steps = 1000\n loss_ = np.zeros(num_steps) # Style: `_` to indicate sess.run result.\n mse_ = np.zeros(num_steps)\n with tf.Session() as sess:\n sess.run(init_op)\n for it in xrange(loss_.size):\n _, loss_[it], mse_[it] = sess.run([train_op, loss, mse])\n if it % 200 == 0 or it == loss_.size - 1:\n print(\"iteration:{} loss:{} mse:{}\".format(it, loss_[it], mse_[it]))\n\n # ==> iteration:0 loss:38.2020797729 mse:4.17175960541\n # iteration:200 loss:2.90179634094 mse:0.990987896919\n # iteration:400 loss:2.82727336884 mse:0.990926623344\n # iteration:600 loss:2.82726788521 mse:0.990926682949\n # iteration:800 loss:2.82726788521 mse:0.990926682949\n # iteration:999 loss:2.82726788521 mse:0.990926682949\n ```\n\n Args:\n x: `Tensor` with floating type. Must have statically defined rank and\n statically known right-most dimension.\n dims: Scalar, `int`, `Tensor` indicated the MVN event size, i.e., the\n created MVN will be distribution over length-`dims` vectors.\n layer_fn: Python `callable` which takes input `x` and `int` scalar `d` and\n returns a transformation of `x` with shape\n `tf.concat([tf.shape(x)[:-1], [d]], axis=0)`.\n Default value: `tf.layers.dense`.\n loc_fn: Python `callable` which transforms the `loc` parameter. Takes a\n (batch of) length-`dims` vectors and returns a `Tensor` of same shape and\n `dtype`.\n Default value: `lambda x: x`.\n scale_fn: Python `callable` which transforms the `scale` parameters. Takes a\n (batch of) length-`dims * (dims + 1) / 2` vectors and returns a\n lower-triangular `Tensor` of same batch shape with rightmost dimensions\n having shape `[dims, dims]`.\n Default value: `tril_with_diag_softplus_and_shift`.\n name: A `name_scope` name for operations created by this function.\n Default value: `None` (i.e., \"multivariate_normal_tril\").\n\n Returns:\n mvntril: An instance of `tfd.MultivariateNormalTriL`.\n \"\"\"\n with tf.compat.v1.name_scope(name, 'multivariate_normal_tril', [x, dims]):\n x = tf.convert_to_tensor(value=x, name='x')\n x = layer_fn(x, dims + dims * (dims + 1) // 2)\n return tfd.MultivariateNormalTriL(\n loc=loc_fn(x[..., :dims]),\n scale_tril=scale_fn(x[..., dims:]))"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef bernoulli(x, layer_fn=tf.compat.v1.layers.dense, name=None):\n with tf.compat.v1.name_scope(name, 'bernoulli', [x]):\n x = tf.convert_to_tensor(value=x, name='x')\n logits = tf.squeeze(layer_fn(x, 1), axis=-1)\n return tfd.Bernoulli(logits=logits)", "response": "Constructs a trainable Bernoulli distribution."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nconstructs a trainable tfd. Normal distribution.", "response": "def normal(x,\n layer_fn=tf.compat.v1.layers.dense,\n loc_fn=lambda x: x,\n scale_fn=1.,\n name=None):\n \"\"\"Constructs a trainable `tfd.Normal` distribution.\n\n\n This function creates a Normal distribution parameterized by loc and scale.\n Using default args, this function is mathematically equivalent to:\n\n ```none\n Y = Normal(loc=matmul(W, x) + b, scale=1)\n\n where,\n W in R^[d, n]\n b in R^d\n ```\n\n #### Examples\n\n This function can be used as a [linear regression](\n https://en.wikipedia.org/wiki/Linear_regression) loss.\n\n ```python\n # This example fits a linear regression loss.\n import tensorflow as tf\n import tensorflow_probability as tfp\n\n # Create fictitious training data.\n dtype = np.float32\n n = 3000 # number of samples\n x_size = 4 # size of single x\n def make_training_data():\n np.random.seed(142)\n x = np.random.randn(n, x_size).astype(dtype)\n w = np.random.randn(x_size).astype(dtype)\n b = np.random.randn(1).astype(dtype)\n true_mean = np.tensordot(x, w, axes=[[-1], [-1]]) + b\n noise = np.random.randn(n).astype(dtype)\n y = true_mean + noise\n return y, x\n y, x = make_training_data()\n\n # Build TF graph for fitting Normal maximum likelihood estimator.\n normal = tfp.trainable_distributions.normal(x)\n loss = -tf.reduce_mean(normal.log_prob(y))\n train_op = tf.train.AdamOptimizer(learning_rate=2.**-5).minimize(loss)\n mse = tf.reduce_mean(tf.squared_difference(y, normal.mean()))\n init_op = tf.global_variables_initializer()\n\n # Run graph 1000 times.\n num_steps = 1000\n loss_ = np.zeros(num_steps) # Style: `_` to indicate sess.run result.\n mse_ = np.zeros(num_steps)\n with tf.Session() as sess:\n sess.run(init_op)\n for it in xrange(loss_.size):\n _, loss_[it], mse_[it] = sess.run([train_op, loss, mse])\n if it % 200 == 0 or it == loss_.size - 1:\n print(\"iteration:{} loss:{} mse:{}\".format(it, loss_[it], mse_[it]))\n\n # ==> iteration:0 loss:6.34114170074 mse:10.8444051743\n # iteration:200 loss:1.40146839619 mse:0.965059816837\n # iteration:400 loss:1.40052902699 mse:0.963181257248\n # iteration:600 loss:1.40052902699 mse:0.963181257248\n # iteration:800 loss:1.40052902699 mse:0.963181257248\n # iteration:999 loss:1.40052902699 mse:0.963181257248\n ```\n\n Args:\n x: `Tensor` with floating type. Must have statically defined rank and\n statically known right-most dimension.\n layer_fn: Python `callable` which takes input `x` and `int` scalar `d` and\n returns a transformation of `x` with shape\n `tf.concat([tf.shape(x)[:-1], [1]], axis=0)`.\n Default value: `tf.layers.dense`.\n loc_fn: Python `callable` which transforms the `loc` parameter. Takes a\n (batch of) length-`dims` vectors and returns a `Tensor` of same shape and\n `dtype`.\n Default value: `lambda x: x`.\n scale_fn: Python `callable` or `Tensor`. If a `callable` transforms the\n `scale` parameters; if `Tensor` is the `tfd.Normal` `scale` argument.\n Takes a (batch of) length-`dims` vectors and returns a `Tensor` of same\n size. (Taking a `callable` or `Tensor` is how `tf.Variable` intializers\n behave.)\n Default value: `1`.\n name: A `name_scope` name for operations created by this function.\n Default value: `None` (i.e., \"normal\").\n\n Returns:\n normal: An instance of `tfd.Normal`.\n \"\"\"\n with tf.compat.v1.name_scope(name, 'normal', [x]):\n x = tf.convert_to_tensor(value=x, name='x')\n if callable(scale_fn):\n y = layer_fn(x, 2)\n loc = loc_fn(y[..., 0])\n scale = scale_fn(y[..., 1])\n else:\n y = tf.squeeze(layer_fn(x, 1), axis=-1)\n loc = loc_fn(y)\n scale = tf.cast(scale_fn, loc.dtype.base_dtype)\n return tfd.Normal(loc=loc, scale=scale)"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef poisson(x,\n layer_fn=tf.compat.v1.layers.dense,\n log_rate_fn=lambda x: x,\n name=None):\n \"\"\"Constructs a trainable `tfd.Poisson` distribution.\n\n This function creates a Poisson distribution parameterized by log rate.\n Using default args, this function is mathematically equivalent to:\n\n ```none\n Y = Poisson(log_rate=matmul(W, x) + b)\n\n where,\n W in R^[d, n]\n b in R^d\n ```\n\n #### Examples\n\n This can be used as a [Poisson regression](\n https://en.wikipedia.org/wiki/Poisson_regression) loss.\n\n ```python\n # This example fits a poisson regression loss.\n import numpy as np\n import tensorflow as tf\n import tensorflow_probability as tfp\n\n # Create fictitious training data.\n dtype = np.float32\n n = 3000 # number of samples\n x_size = 4 # size of single x\n def make_training_data():\n np.random.seed(142)\n x = np.random.randn(n, x_size).astype(dtype)\n w = np.random.randn(x_size).astype(dtype)\n b = np.random.randn(1).astype(dtype)\n true_log_rate = np.tensordot(x, w, axes=[[-1], [-1]]) + b\n y = np.random.poisson(lam=np.exp(true_log_rate)).astype(dtype)\n return y, x\n y, x = make_training_data()\n\n # Build TF graph for fitting Poisson maximum likelihood estimator.\n poisson = tfp.trainable_distributions.poisson(x)\n loss = -tf.reduce_mean(poisson.log_prob(y))\n train_op = tf.train.AdamOptimizer(learning_rate=2.**-5).minimize(loss)\n mse = tf.reduce_mean(tf.squared_difference(y, poisson.mean()))\n init_op = tf.global_variables_initializer()\n\n # Run graph 1000 times.\n num_steps = 1000\n loss_ = np.zeros(num_steps) # Style: `_` to indicate sess.run result.\n mse_ = np.zeros(num_steps)\n with tf.Session() as sess:\n sess.run(init_op)\n for it in xrange(loss_.size):\n _, loss_[it], mse_[it] = sess.run([train_op, loss, mse])\n if it % 200 == 0 or it == loss_.size - 1:\n print(\"iteration:{} loss:{} mse:{}\".format(it, loss_[it], mse_[it]))\n\n # ==> iteration:0 loss:37.0814208984 mse:6359.41259766\n # iteration:200 loss:1.42010736465 mse:40.7654914856\n # iteration:400 loss:1.39027583599 mse:8.77660560608\n # iteration:600 loss:1.3902695179 mse:8.78443241119\n # iteration:800 loss:1.39026939869 mse:8.78443622589\n # iteration:999 loss:1.39026939869 mse:8.78444766998\n ```\n\n Args:\n x: `Tensor` with floating type. Must have statically defined rank and\n statically known right-most dimension.\n layer_fn: Python `callable` which takes input `x` and `int` scalar `d` and\n returns a transformation of `x` with shape\n `tf.concat([tf.shape(x)[:-1], [1]], axis=0)`.\n Default value: `tf.layers.dense`.\n log_rate_fn: Python `callable` which transforms the `log_rate` parameter.\n Takes a (batch of) length-`dims` vectors and returns a `Tensor` of same\n shape and `dtype`.\n Default value: `lambda x: x`.\n name: A `name_scope` name for operations created by this function.\n Default value: `None` (i.e., \"poisson\").\n\n Returns:\n poisson: An instance of `tfd.Poisson`.\n \"\"\"\n with tf.compat.v1.name_scope(name, 'poisson', [x]):\n x = tf.convert_to_tensor(value=x, name='x')\n log_rate = log_rate_fn(tf.squeeze(layer_fn(x, 1), axis=-1))\n return tfd.Poisson(log_rate=log_rate)", "response": "Constructs a trainable tfd. Poisson distribution."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _euler_method(random_draw_parts,\n state_parts,\n drift_parts,\n step_size_parts,\n volatility_parts,\n name=None):\n \"\"\"Applies one step of Euler-Maruyama method.\n\n Generates proposal of the form:\n\n ```python\n tfd.Normal(loc=state_parts + _get_drift(state_parts, ...),\n scale=tf.sqrt(step_size * volatility_fn(current_state)))\n ```\n\n `_get_drift(state_parts, ..)` is a diffusion drift value at `state_parts`.\n\n\n Args:\n random_draw_parts: Python `list` of `Tensor`s containing the value(s) of the\n random perturbation variable(s). Must broadcast with the shape of\n `state_parts`.\n state_parts: Python `list` of `Tensor`s representing the current\n state(s) of the Markov chain(s).\n drift_parts: Python `list` of `Tensor`s representing value of the drift\n `_get_drift(*state_parts, ..)`. Must broadcast with the shape of\n `state_parts`.\n step_size_parts: Python `list` of `Tensor`s representing the step size for\n the Euler-Maruyama method. Must broadcast with the shape of\n `state_parts`. Larger step sizes lead to faster progress, but\n too-large step sizes make rejection exponentially more likely. When\n possible, it's often helpful to match per-variable step sizes to the\n standard deviations of the target distribution in each variable.\n volatility_parts: Python `list` of `Tensor`s representing the value of\n `volatility_fn(*state_parts)`. Must broadcast with the shape of\n `state_parts`.\n name: Python `str` name prefixed to Ops created by this function.\n Default value: `None` (i.e., 'mala_euler_method').\n\n Returns:\n proposed_state_parts: Tensor or Python list of `Tensor`s representing the\n state(s) of the Markov chain(s) at each result step. Has same shape as\n input `current_state_parts`.\n \"\"\"\n with tf.compat.v1.name_scope(name, 'mala_euler_method', [\n random_draw_parts, state_parts, drift_parts, step_size_parts,\n volatility_parts\n ]):\n proposed_state_parts = []\n for random_draw, state, drift, step_size, volatility in zip(\n random_draw_parts,\n state_parts,\n drift_parts,\n step_size_parts,\n volatility_parts):\n proposal = state + drift + volatility * tf.sqrt(step_size) * random_draw\n proposed_state_parts.append(proposal)\n\n return proposed_state_parts", "response": "Applies one step of Euler - Maruyama method."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _get_drift(step_size_parts, volatility_parts, grads_volatility,\n grads_target_log_prob,\n name=None):\n \"\"\"Compute diffusion drift at the current location `current_state`.\n\n The drift of the diffusion at is computed as\n\n ```none\n 0.5 * `step_size` * volatility_parts * `target_log_prob_fn(current_state)`\n + `step_size` * `grads_volatility`\n ```\n\n where `volatility_parts` = `volatility_fn(current_state)**2` and\n `grads_volatility` is a gradient of `volatility_parts` at the `current_state`.\n\n Args:\n step_size_parts: Python `list` of `Tensor`s representing the step size for\n Euler-Maruyama method. Must broadcast with the shape of\n `volatility_parts`. Larger step sizes lead to faster progress, but\n too-large step sizes make rejection exponentially more likely. When\n possible, it's often helpful to match per-variable step sizes to the\n standard deviations of the target distribution in each variable.\n volatility_parts: Python `list` of `Tensor`s representing the value of\n `volatility_fn(*state_parts)`.\n grads_volatility: Python list of `Tensor`s representing the value of the\n gradient of `volatility_parts**2` wrt the state of the chain.\n grads_target_log_prob: Python list of `Tensor`s representing\n gradient of `target_log_prob_fn(*state_parts`) wrt `state_parts`. Must\n have same shape as `volatility_parts`.\n name: Python `str` name prefixed to Ops created by this function.\n Default value: `None` (i.e., 'mala_get_drift').\n\n Returns:\n drift_parts: Tensor or Python list of `Tensor`s representing the\n state(s) of the Markov chain(s) at each result step. Has same shape as\n input `current_state_parts`.\n \"\"\"\n\n with tf.compat.v1.name_scope(name, 'mala_get_drift', [\n step_size_parts, volatility_parts, grads_volatility, grads_target_log_prob\n ]):\n\n drift_parts = []\n\n for step_size, volatility, grad_volatility, grad_target_log_prob in (\n zip(step_size_parts,\n volatility_parts,\n grads_volatility,\n grads_target_log_prob)):\n volatility_squared = tf.square(volatility)\n drift = 0.5 * step_size * (volatility_squared * grad_target_log_prob\n + grad_volatility)\n drift_parts.append(drift)\n\n return drift_parts", "response": "Compute the diffusion drift at the current location."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _maybe_broadcast_volatility(volatility_parts,\n state_parts):\n \"\"\"Helper to broadcast `volatility_parts` to the shape of `state_parts`.\"\"\"\n return [v + tf.zeros_like(sp, dtype=sp.dtype.base_dtype)\n for v, sp in zip(volatility_parts, state_parts)]", "response": "Helper to broadcast volatility_parts to the shape of state_parts."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nbuilding an AR transition matrix for a autoregressive state space model.", "response": "def make_ar_transition_matrix(coefficients):\n \"\"\"Build transition matrix for an autoregressive StateSpaceModel.\n\n When applied to a vector of previous values, this matrix computes\n the expected new value (summing the previous states according to the\n autoregressive coefficients) in the top dimension of the state space,\n and moves all previous values down by one dimension, 'forgetting' the\n final (least recent) value. That is, it looks like this:\n\n ```\n ar_matrix = [ coefs[0], coefs[1], ..., coefs[order]\n 1., 0 , ..., 0.\n 0., 1., ..., 0.\n ...\n 0., 0., ..., 1., 0. ]\n ```\n\n Args:\n coefficients: float `Tensor` of shape `concat([batch_shape, [order]])`.\n\n Returns:\n ar_matrix: float `Tensor` with shape `concat([batch_shape,\n [order, order]])`.\n \"\"\"\n\n top_row = tf.expand_dims(coefficients, -2)\n coef_shape = dist_util.prefer_static_shape(coefficients)\n batch_shape, order = coef_shape[:-1], coef_shape[-1]\n remaining_rows = tf.concat([\n tf.eye(order - 1, dtype=coefficients.dtype, batch_shape=batch_shape),\n tf.zeros(tf.concat([batch_shape, (order - 1, 1)], axis=0),\n dtype=coefficients.dtype)\n ], axis=-1)\n ar_matrix = tf.concat([top_row, remaining_rows], axis=-2)\n return ar_matrix"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef diag_jacobian(xs,\n ys=None,\n sample_shape=None,\n fn=None,\n parallel_iterations=10,\n name=None):\n \"\"\"Computes diagonal of the Jacobian matrix of `ys=fn(xs)` wrt `xs`.\n\n If `ys` is a tensor or a list of tensors of the form `(ys_1, .., ys_n)` and\n `xs` is of the form `(xs_1, .., xs_n)`, the function `jacobians_diag`\n computes the diagonal of the Jacobian matrix, i.e., the partial derivatives\n `(dys_1/dxs_1,.., dys_n/dxs_n`). For definition details, see\n https://en.wikipedia.org/wiki/Jacobian_matrix_and_determinant\n #### Example\n\n ##### Diagonal Hessian of the log-density of a 3D Gaussian distribution\n\n In this example we sample from a standard univariate normal\n distribution using MALA with `step_size` equal to 0.75.\n\n ```python\n import tensorflow as tf\n import tensorflow_probability as tfp\n import numpy as np\n\n tfd = tfp.distributions\n\n dtype = np.float32\n with tf.Session(graph=tf.Graph()) as sess:\n true_mean = dtype([0, 0, 0])\n true_cov = dtype([[1, 0.25, 0.25], [0.25, 2, 0.25], [0.25, 0.25, 3]])\n chol = tf.linalg.cholesky(true_cov)\n target = tfd.MultivariateNormalTriL(loc=true_mean, scale_tril=chol)\n\n # Assume that the state is passed as a list of tensors `x` and `y`.\n # Then the target function is defined as follows:\n def target_fn(x, y):\n # Stack the input tensors together\n z = tf.concat([x, y], axis=-1) - true_mean\n return target.log_prob(z)\n\n sample_shape = [3, 5]\n state = [tf.ones(sample_shape + [2], dtype=dtype),\n tf.ones(sample_shape + [1], dtype=dtype)]\n fn_val, grads = tfp.math.value_and_gradient(target_fn, state)\n\n # We can either pass the `sample_shape` of the `state` or not, which impacts\n # computational speed of `diag_jacobian`\n _, diag_jacobian_shape_passed = diag_jacobian(\n xs=state, ys=grads, sample_shape=tf.shape(fn_val))\n _, diag_jacobian_shape_none = diag_jacobian(\n xs=state, ys=grads)\n\n diag_jacobian_shape_passed_ = sess.run(diag_jacobian_shape_passed)\n diag_jacobian_shape_none_ = sess.run(diag_jacobian_shape_none)\n\n print('hessian computed through `diag_jacobian`, sample_shape passed: ',\n np.concatenate(diag_jacobian_shape_passed_, -1))\n print('hessian computed through `diag_jacobian`, sample_shape skipped',\n np.concatenate(diag_jacobian_shape_none_, -1))\n\n ```\n\n Args:\n xs: `Tensor` or a python `list` of `Tensors` of real-like dtypes and shapes\n `sample_shape` + `event_shape_i`, where `event_shape_i` can be different\n for different tensors.\n ys: `Tensor` or a python `list` of `Tensors` of the same dtype as `xs`. Must\n broadcast with the shape of `xs`. Can be omitted if `fn` is provided.\n sample_shape: A common `sample_shape` of the input tensors of `xs`. If not,\n provided, assumed to be `[1]`, which may result in a slow performance of\n `jacobians_diag`.\n fn: Python callable that takes `xs` as an argument (or `*xs`, if it is a\n list) and returns `ys`. Might be skipped if `ys` is provided and\n `tf.enable_eager_execution()` is disabled.\n parallel_iterations: `int` that specifies the allowed number of coordinates\n of the input tensor `xs`, for which the partial derivatives `dys_i/dxs_i`\n can be computed in parallel.\n name: Python `str` name prefixed to `Ops` created by this function.\n Default value: `None` (i.e., \"diag_jacobian\").\n\n Returns:\n ys: a list, which coincides with the input `ys`, when provided.\n If the input `ys` is None, `fn(*xs)` gets computed and returned as a list.\n jacobians_diag_res: a `Tensor` or a Python list of `Tensor`s of the same\n dtypes and shapes as the input `xs`. This is the diagonal of the Jacobian\n of ys wrt xs.\n\n Raises:\n ValueError: if lists `xs` and `ys` have different length or both `ys` and\n `fn` are `None`, or `fn` is None in the eager execution mode.\n \"\"\"\n with tf.compat.v1.name_scope(name, 'jacobians_diag', [xs, ys]):\n if sample_shape is None:\n sample_shape = [1]\n # Output Jacobian diagonal\n jacobians_diag_res = []\n # Convert input `xs` to a list\n xs = list(xs) if _is_list_like(xs) else [xs]\n xs = [tf.convert_to_tensor(value=x) for x in xs]\n if not tf.executing_eagerly():\n if ys is None:\n if fn is None:\n raise ValueError('Both `ys` and `fn` can not be `None`')\n else:\n ys = fn(*xs)\n # Convert ys to a list\n ys = list(ys) if _is_list_like(ys) else [ys]\n if len(xs) != len(ys):\n raise ValueError('`xs` and `ys` should have the same length')\n for y, x in zip(ys, xs):\n # Broadcast `y` to the shape of `x`.\n y_ = y + tf.zeros_like(x)\n # Change `event_shape` to one-dimension\n y_ = tf.reshape(y, tf.concat([sample_shape, [-1]], -1))\n\n # Declare an iterator and tensor array loop variables for the gradients.\n n = tf.size(input=x) / tf.cast(\n tf.reduce_prod(input_tensor=sample_shape), dtype=tf.int32)\n n = tf.cast(n, dtype=tf.int32)\n loop_vars = [\n 0,\n tf.TensorArray(x.dtype, n)\n ]\n\n def loop_body(j):\n \"\"\"Loop function to compute gradients of the each direction.\"\"\"\n # Gradient along direction `j`.\n res = tf.gradients(ys=y_[..., j], xs=x)[0] # pylint: disable=cell-var-from-loop\n if res is None:\n # Return zero, if the gradient is `None`.\n res = tf.zeros(tf.concat([sample_shape, [1]], -1),\n dtype=x.dtype) # pylint: disable=cell-var-from-loop\n else:\n # Reshape `event_shape` to 1D\n res = tf.reshape(res, tf.concat([sample_shape, [-1]], -1))\n # Add artificial dimension for the case of zero shape input tensor\n res = tf.expand_dims(res, 0)\n res = res[..., j]\n return res # pylint: disable=cell-var-from-loop\n\n # Iterate over all elements of the gradient and compute second order\n # derivatives.\n _, jacobian_diag_res = tf.while_loop(\n cond=lambda j, _: j < n, # pylint: disable=cell-var-from-loop\n body=lambda j, result: (j + 1, result.write(j, loop_body(j))),\n loop_vars=loop_vars,\n parallel_iterations=parallel_iterations)\n\n shape_x = tf.shape(input=x)\n # Stack gradients together and move flattened `event_shape` to the\n # zero position\n reshaped_jacobian_diag = tf.transpose(a=jacobian_diag_res.stack())\n # Reshape to the original tensor\n reshaped_jacobian_diag = tf.reshape(reshaped_jacobian_diag, shape_x)\n jacobians_diag_res.append(reshaped_jacobian_diag)\n\n else:\n if fn is None:\n raise ValueError('`fn` can not be `None` when eager execution is '\n 'enabled')\n if ys is None:\n ys = fn(*xs)\n\n def fn_slice(i, j):\n \"\"\"Broadcast y[i], flatten event shape of y[i], return y[i][..., j].\"\"\"\n def fn_broadcast(*state):\n res = fn(*state)\n res = list(res) if _is_list_like(res) else [res]\n if len(res) != len(state):\n res *= len(state)\n res = [tf.reshape(r + tf.zeros_like(s),\n tf.concat([sample_shape, [-1]], -1))\n for r, s in zip(res, state)]\n return res\n # Expand dimensions before returning in order to support 0D input `xs`\n return lambda *state: tf.expand_dims(fn_broadcast(*state)[i], 0)[..., j]\n\n def make_loop_body(i, x):\n \"\"\"Loop function to compute gradients of the each direction.\"\"\"\n def _fn(j, result):\n res = value_and_gradient(fn_slice(i, j), xs)[1][i]\n if res is None:\n res = tf.zeros(tf.concat([sample_shape, [1]], -1), dtype=x.dtype)\n else:\n res = tf.reshape(res, tf.concat([sample_shape, [-1]], -1))\n res = res[..., j]\n return j + 1, result.write(j, res)\n return _fn\n\n for i, x in enumerate(xs):\n # Declare an iterator and tensor array loop variables for the gradients.\n n = tf.size(input=x) / tf.cast(\n tf.reduce_prod(input_tensor=sample_shape), dtype=tf.int32)\n n = tf.cast(n, dtype=tf.int32)\n loop_vars = [\n 0,\n tf.TensorArray(x.dtype, n)\n ]\n\n # Iterate over all elements of the gradient and compute second order\n # derivatives.\n _, jacobian_diag_res = tf.while_loop(\n cond=lambda j, _: j < n,\n body=make_loop_body(i, x),\n loop_vars=loop_vars,\n parallel_iterations=parallel_iterations)\n\n shape_x = tf.shape(input=x)\n # Stack gradients together and move flattened `event_shape` to the\n # zero position\n reshaped_jacobian_diag = tf.transpose(a=jacobian_diag_res.stack())\n # Reshape to the original tensor\n reshaped_jacobian_diag = tf.reshape(reshaped_jacobian_diag, shape_x)\n jacobians_diag_res.append(reshaped_jacobian_diag)\n\n return ys, jacobians_diag_res", "response": "Diagonal Hessian of the Jacobian matrix of ys = fn ( xs ys_n ) wrt xs."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef calculate_reshape(original_shape, new_shape, validate=False, name=None):\n batch_shape_static = tensorshape_util.constant_value_as_shape(new_shape)\n if tensorshape_util.is_fully_defined(batch_shape_static):\n return np.int32(batch_shape_static), batch_shape_static, []\n with tf.name_scope(name or \"calculate_reshape\"):\n original_size = tf.reduce_prod(input_tensor=original_shape)\n implicit_dim = tf.equal(new_shape, -1)\n size_implicit_dim = (\n original_size // tf.maximum(1, -tf.reduce_prod(input_tensor=new_shape)))\n new_ndims = tf.shape(input=new_shape)\n expanded_new_shape = tf.where( # Assumes exactly one `-1`.\n implicit_dim, tf.fill(new_ndims, size_implicit_dim), new_shape)\n validations = [] if not validate else [ # pylint: disable=g-long-ternary\n assert_util.assert_rank(\n original_shape, 1, message=\"Original shape must be a vector.\"),\n assert_util.assert_rank(\n new_shape, 1, message=\"New shape must be a vector.\"),\n assert_util.assert_less_equal(\n tf.math.count_nonzero(implicit_dim, dtype=tf.int32),\n 1,\n message=\"At most one dimension can be unknown.\"),\n assert_util.assert_positive(\n expanded_new_shape, message=\"Shape elements must be >=-1.\"),\n assert_util.assert_equal(\n tf.reduce_prod(input_tensor=expanded_new_shape),\n original_size,\n message=\"Shape sizes do not match.\"),\n ]\n return expanded_new_shape, batch_shape_static, validations", "response": "Calculates the reshaped dimensions."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef validate_init_args_statically(distribution, batch_shape):\n if tensorshape_util.rank(batch_shape.shape) is not None:\n if tensorshape_util.rank(batch_shape.shape) != 1:\n raise ValueError(\"`batch_shape` must be a vector \"\n \"(saw rank: {}).\".format(\n tensorshape_util.rank(batch_shape.shape)))\n\n batch_shape_static = tensorshape_util.constant_value_as_shape(batch_shape)\n batch_size_static = tensorshape_util.num_elements(batch_shape_static)\n dist_batch_size_static = tensorshape_util.num_elements(\n distribution.batch_shape)\n\n if batch_size_static is not None and dist_batch_size_static is not None:\n if batch_size_static != dist_batch_size_static:\n raise ValueError(\"`batch_shape` size ({}) must match \"\n \"`distribution.batch_shape` size ({}).\".format(\n batch_size_static, dist_batch_size_static))\n\n if tensorshape_util.dims(batch_shape_static) is not None:\n if any(\n tf.compat.dimension_value(dim) is not None and\n tf.compat.dimension_value(dim) < 1 for dim in batch_shape_static):\n raise ValueError(\"`batch_shape` elements must be >=-1.\")", "response": "Helper to __init__ which makes or raises assertions."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef _sample_shape(self, x):\n x_ndims = (\n tf.rank(x) if tensorshape_util.rank(x.shape) is None else\n tensorshape_util.rank(x.shape))\n event_ndims = (\n tf.size(input=self.event_shape_tensor())\n if tensorshape_util.rank(self.event_shape) is None else\n tensorshape_util.rank(self.event_shape))\n batch_ndims = (\n tf.size(input=self._batch_shape_unexpanded)\n if tensorshape_util.rank(self.batch_shape) is None else\n tensorshape_util.rank(self.batch_shape))\n sample_ndims = x_ndims - batch_ndims - event_ndims\n if isinstance(sample_ndims, int):\n static_sample_shape = x.shape[:sample_ndims]\n else:\n static_sample_shape = tf.TensorShape(None)\n if tensorshape_util.is_fully_defined(static_sample_shape):\n sample_shape = np.int32(static_sample_shape)\n else:\n sample_shape = tf.shape(input=x)[:sample_ndims]\n return sample_shape, static_sample_shape", "response": "Computes graph and static sample_shape."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\ncalls fn appropriately reshaping its input x and output.", "response": "def _call_reshape_input_output(self, fn, x, extra_kwargs=None):\n \"\"\"Calls `fn`, appropriately reshaping its input `x` and output.\"\"\"\n # Note: we take `extra_kwargs` as a dict rather than `**extra_kwargs`\n # because it is possible the user provided extra kwargs would itself\n # have `fn` and/or `x` as a key.\n with tf.control_dependencies(self._runtime_assertions +\n self._validate_sample_arg(x)):\n sample_shape, static_sample_shape = self._sample_shape(x)\n old_shape = tf.concat(\n [\n sample_shape,\n self.distribution.batch_shape_tensor(),\n self.event_shape_tensor(),\n ],\n axis=0)\n x_reshape = tf.reshape(x, old_shape)\n result = fn(x_reshape, **extra_kwargs) if extra_kwargs else fn(x_reshape)\n new_shape = tf.concat(\n [\n sample_shape,\n self._batch_shape_unexpanded,\n ], axis=0)\n result = tf.reshape(result, new_shape)\n if (tensorshape_util.rank(static_sample_shape) is not None and\n tensorshape_util.rank(self.batch_shape) is not None):\n new_shape = tensorshape_util.concatenate(static_sample_shape,\n self.batch_shape)\n tensorshape_util.set_shape(result, new_shape)\n return result"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _call_and_reshape_output(\n self,\n fn,\n event_shape_list=None,\n static_event_shape_list=None,\n extra_kwargs=None):\n \"\"\"Calls `fn` and appropriately reshapes its output.\"\"\"\n # Note: we take `extra_kwargs` as a dict rather than `**extra_kwargs`\n # because it is possible the user provided extra kwargs would itself\n # have `fn`, `event_shape_list`, `static_event_shape_list` and/or\n # `extra_kwargs` as keys.\n with tf.control_dependencies(self._runtime_assertions):\n if event_shape_list is None:\n event_shape_list = [self._event_shape_tensor()]\n if static_event_shape_list is None:\n static_event_shape_list = [self.event_shape]\n new_shape = tf.concat(\n [self._batch_shape_unexpanded] + event_shape_list, axis=0)\n result = tf.reshape(fn(**extra_kwargs) if extra_kwargs else fn(),\n new_shape)\n if (tensorshape_util.rank(self.batch_shape) is not None and\n tensorshape_util.rank(self.event_shape) is not None):\n event_shape = tf.TensorShape([])\n for rss in static_event_shape_list:\n event_shape = tensorshape_util.concatenate(event_shape, rss)\n static_shape = tensorshape_util.concatenate(\n self.batch_shape, event_shape)\n tensorshape_util.set_shape(result, static_shape)\n return result", "response": "Calls fn and appropriately reshapes its output."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _validate_sample_arg(self, x):\n with tf.name_scope(\"validate_sample_arg\"):\n x_ndims = (\n tf.rank(x) if tensorshape_util.rank(x.shape) is None else\n tensorshape_util.rank(x.shape))\n event_ndims = (\n tf.size(input=self.event_shape_tensor())\n if tensorshape_util.rank(self.event_shape) is None else\n tensorshape_util.rank(self.event_shape))\n batch_ndims = (\n tf.size(input=self._batch_shape_unexpanded)\n if tensorshape_util.rank(self.batch_shape) is None else\n tensorshape_util.rank(self.batch_shape))\n expected_batch_event_ndims = batch_ndims + event_ndims\n\n if (isinstance(x_ndims, int) and\n isinstance(expected_batch_event_ndims, int)):\n if x_ndims < expected_batch_event_ndims:\n raise NotImplementedError(\n \"Broadcasting is not supported; too few batch and event dims \"\n \"(expected at least {}, saw {}).\".format(\n expected_batch_event_ndims, x_ndims))\n ndims_assertion = []\n elif self.validate_args:\n ndims_assertion = [\n assert_util.assert_greater_equal(\n x_ndims,\n expected_batch_event_ndims,\n message=(\"Broadcasting is not supported; too few \"\n \"batch and event dims.\"),\n name=\"assert_batch_and_event_ndims_large_enough\"),\n ]\n\n if (tensorshape_util.is_fully_defined(self.batch_shape) and\n tensorshape_util.is_fully_defined(self.event_shape)):\n expected_batch_event_shape = np.int32(\n tensorshape_util.concatenate(self.batch_shape, self.event_shape))\n else:\n expected_batch_event_shape = tf.concat(\n [\n self.batch_shape_tensor(),\n self.event_shape_tensor(),\n ], axis=0)\n\n sample_ndims = x_ndims - expected_batch_event_ndims\n if isinstance(sample_ndims, int):\n sample_ndims = max(sample_ndims, 0)\n if (isinstance(sample_ndims, int) and\n tensorshape_util.is_fully_defined(x.shape[sample_ndims:])):\n actual_batch_event_shape = np.int32(x.shape[sample_ndims:])\n else:\n sample_ndims = tf.maximum(sample_ndims, 0)\n actual_batch_event_shape = tf.shape(input=x)[sample_ndims:]\n\n if (isinstance(expected_batch_event_shape, np.ndarray) and\n isinstance(actual_batch_event_shape, np.ndarray)):\n if any(expected_batch_event_shape != actual_batch_event_shape):\n raise NotImplementedError(\"Broadcasting is not supported; \"\n \"unexpected batch and event shape \"\n \"(expected {}, saw {}).\".format(\n expected_batch_event_shape,\n actual_batch_event_shape))\n # We need to set the final runtime-assertions to `ndims_assertion` since\n # its possible this assertion was created. We could add a condition to\n # only do so if `self.validate_args == True`, however this is redundant\n # as `ndims_assertion` already encodes this information.\n runtime_assertions = ndims_assertion\n elif self.validate_args:\n # We need to make the `ndims_assertion` a control dep because otherwise\n # TF itself might raise an exception owing to this assertion being\n # ill-defined, ie, one cannot even compare different rank Tensors.\n with tf.control_dependencies(ndims_assertion):\n shape_assertion = assert_util.assert_equal(\n expected_batch_event_shape,\n actual_batch_event_shape,\n message=(\"Broadcasting is not supported; \"\n \"unexpected batch and event shape.\"),\n name=\"assert_batch_and_event_shape_same\")\n runtime_assertions = [shape_assertion]\n else:\n runtime_assertions = []\n\n return runtime_assertions", "response": "Helper which validates sample arg."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _maybe_assert_valid_sample(self, counts):\n if not self.validate_args:\n return counts\n counts = distribution_util.embed_check_nonnegative_integer_form(counts)\n return distribution_util.with_dependencies([\n assert_util.assert_less_equal(\n counts,\n self.total_count,\n message=\"counts are not less than or equal to n.\"),\n ], counts)", "response": "Check counts for proper shape values then return tensor version."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef _flat_sample_distributions(self, sample_shape=(), seed=None, value=None):\n ds = []\n values_out = []\n seed = seed_stream.SeedStream('JointDistributionCoroutine', seed)\n gen = self._model()\n index = 0\n d = next(gen)\n try:\n while True:\n actual_distribution = d.distribution if isinstance(d, self.Root) else d\n ds.append(actual_distribution)\n if (value is not None and len(value) > index and\n value[index] is not None):\n seed()\n next_value = value[index]\n else:\n next_value = actual_distribution.sample(\n sample_shape=sample_shape if isinstance(d, self.Root) else (),\n seed=seed())\n values_out.append(next_value)\n index += 1\n d = gen.send(next_value)\n except StopIteration:\n pass\n return ds, values_out", "response": "Executes model and creates both samples and distributions."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef _kl_pareto_pareto(a, b, name=None):\n with tf.name_scope(name or \"kl_pareto_pareto\"):\n # Consistent with\n # http://www.mast.queensu.ca/~communications/Papers/gil-msc11.pdf, page 55\n # Terminology is different from source to source for Pareto distributions.\n # The 'concentration' parameter corresponds to 'a' in that source, and the\n # 'scale' parameter corresponds to 'm'.\n final_batch_shape = distribution_util.get_broadcast_shape(\n a.concentration, b.concentration, a.scale, b.scale)\n common_type = dtype_util.common_dtype(\n [a.concentration, b.concentration, a.scale, b.scale], tf.float32)\n return tf.where(\n a.scale >= b.scale,\n b.concentration * (tf.math.log(a.scale) - tf.math.log(b.scale)) +\n tf.math.log(a.concentration) - tf.math.log(b.concentration) +\n b.concentration / a.concentration - 1.0,\n tf.broadcast_to(tf.cast(np.inf, common_type), final_batch_shape))", "response": "Calculate the batched KL divergence of a Pareto distribution."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nextending the support of a single function at x.", "response": "def _extend_support(self, x, f, alt):\n \"\"\"Returns `f(x)` if x is in the support, and `alt` otherwise.\n\n Given `f` which is defined on the support of this distribution\n (e.g. x > scale), extend the function definition to the real line\n by defining `f(x) = alt` for `x < scale`.\n\n Args:\n x: Floating-point Tensor to evaluate `f` at.\n f: Lambda that takes in a tensor and returns a tensor. This represents\n the function who we want to extend the domain of definition.\n alt: Python or numpy literal representing the value to use for extending\n the domain.\n Returns:\n Tensor representing an extension of `f(x)`.\n \"\"\"\n # We need to do a series of broadcasts for the tf.where.\n scale = self.scale + tf.zeros_like(self.concentration)\n is_invalid = x < scale\n scale = scale + tf.zeros_like(x)\n x = x + tf.zeros_like(scale)\n # We need to do this to ensure gradients are sound.\n y = f(tf.where(is_invalid, scale, x))\n if alt == 0.:\n alt = tf.zeros_like(y)\n elif alt == 1.:\n alt = tf.ones_like(y)\n else:\n alt = tf.fill(\n dims=tf.shape(input=y),\n value=dtype_util.as_numpy_dtype(self.dtype)(alt))\n return tf.where(is_invalid, alt, y)"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\ncreate the variational distribution for LDA.", "response": "def make_lda_variational(activation, num_topics, layer_sizes):\n \"\"\"Creates the variational distribution for LDA.\n\n Args:\n activation: Activation function to use.\n num_topics: The number of topics.\n layer_sizes: The number of hidden units per layer in the encoder.\n\n Returns:\n lda_variational: A function that takes a bag-of-words Tensor as\n input and returns a distribution over topics.\n \"\"\"\n encoder_net = tf.keras.Sequential()\n for num_hidden_units in layer_sizes:\n encoder_net.add(\n tf.keras.layers.Dense(\n num_hidden_units,\n activation=activation,\n kernel_initializer=tf.compat.v1.glorot_normal_initializer()))\n encoder_net.add(\n tf.keras.layers.Dense(\n num_topics,\n activation=tf.nn.softplus,\n kernel_initializer=tf.compat.v1.glorot_normal_initializer()))\n\n def lda_variational(bag_of_words):\n concentration = _clip_dirichlet_parameters(encoder_net(bag_of_words))\n return ed.Dirichlet(concentration=concentration, name=\"topics_posterior\")\n\n return lda_variational"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef model_fn(features, labels, mode, params, config):\n del labels, config\n\n # Set up the model's learnable parameters.\n logit_concentration = tf.compat.v1.get_variable(\n \"logit_concentration\",\n shape=[1, params[\"num_topics\"]],\n initializer=tf.compat.v1.initializers.constant(\n _softplus_inverse(params[\"prior_initial_value\"])))\n concentration = _clip_dirichlet_parameters(\n tf.nn.softplus(logit_concentration))\n\n num_words = features.shape[1]\n topics_words_logits = tf.compat.v1.get_variable(\n \"topics_words_logits\",\n shape=[params[\"num_topics\"], num_words],\n initializer=tf.compat.v1.glorot_normal_initializer())\n topics_words = tf.nn.softmax(topics_words_logits, axis=-1)\n\n # Compute expected log-likelihood. First, sample from the variational\n # distribution; second, compute the log-likelihood given the sample.\n lda_variational = make_lda_variational(\n params[\"activation\"],\n params[\"num_topics\"],\n params[\"layer_sizes\"])\n with ed.tape() as variational_tape:\n _ = lda_variational(features)\n\n with ed.tape() as model_tape:\n with ed.interception(\n make_value_setter(topics=variational_tape[\"topics_posterior\"])):\n posterior_predictive = latent_dirichlet_allocation(concentration,\n topics_words)\n\n log_likelihood = posterior_predictive.distribution.log_prob(features)\n tf.compat.v1.summary.scalar(\"log_likelihood\",\n tf.reduce_mean(input_tensor=log_likelihood))\n\n # Compute the KL-divergence between two Dirichlets analytically.\n # The sampled KL does not work well for \"sparse\" distributions\n # (see Appendix D of [2]).\n kl = variational_tape[\"topics_posterior\"].distribution.kl_divergence(\n model_tape[\"topics\"].distribution)\n tf.compat.v1.summary.scalar(\"kl\", tf.reduce_mean(input_tensor=kl))\n\n # Ensure that the KL is non-negative (up to a very small slack).\n # Negative KL can happen due to numerical instability.\n with tf.control_dependencies(\n [tf.compat.v1.assert_greater(kl, -1e-3, message=\"kl\")]):\n kl = tf.identity(kl)\n\n elbo = log_likelihood - kl\n avg_elbo = tf.reduce_mean(input_tensor=elbo)\n tf.compat.v1.summary.scalar(\"elbo\", avg_elbo)\n loss = -avg_elbo\n\n # Perform variational inference by minimizing the -ELBO.\n global_step = tf.compat.v1.train.get_or_create_global_step()\n optimizer = tf.compat.v1.train.AdamOptimizer(params[\"learning_rate\"])\n\n # This implements the \"burn-in\" for prior parameters (see Appendix D of [2]).\n # For the first prior_burn_in_steps steps they are fixed, and then trained\n # jointly with the other parameters.\n grads_and_vars = optimizer.compute_gradients(loss)\n grads_and_vars_except_prior = [\n x for x in grads_and_vars if x[1] != logit_concentration]\n\n def train_op_except_prior():\n return optimizer.apply_gradients(\n grads_and_vars_except_prior,\n global_step=global_step)\n\n def train_op_all():\n return optimizer.apply_gradients(\n grads_and_vars,\n global_step=global_step)\n\n train_op = tf.cond(\n pred=global_step < params[\"prior_burn_in_steps\"],\n true_fn=train_op_except_prior,\n false_fn=train_op_all)\n\n # The perplexity is an exponent of the average negative ELBO per word.\n words_per_document = tf.reduce_sum(input_tensor=features, axis=1)\n log_perplexity = -elbo / words_per_document\n tf.compat.v1.summary.scalar(\n \"perplexity\", tf.exp(tf.reduce_mean(input_tensor=log_perplexity)))\n (log_perplexity_tensor,\n log_perplexity_update) = tf.compat.v1.metrics.mean(log_perplexity)\n perplexity_tensor = tf.exp(log_perplexity_tensor)\n\n # Obtain the topics summary. Implemented as a py_func for simplicity.\n topics = tf.compat.v1.py_func(\n functools.partial(get_topics_strings, vocabulary=params[\"vocabulary\"]),\n [topics_words, concentration],\n tf.string,\n stateful=False)\n tf.compat.v1.summary.text(\"topics\", topics)\n\n return tf.estimator.EstimatorSpec(\n mode=mode,\n loss=loss,\n train_op=train_op,\n eval_metric_ops={\n \"elbo\": tf.compat.v1.metrics.mean(elbo),\n \"log_likelihood\": tf.compat.v1.metrics.mean(log_likelihood),\n \"kl\": tf.compat.v1.metrics.mean(kl),\n \"perplexity\": (perplexity_tensor, log_perplexity_update),\n \"topics\": (topics, tf.no_op()),\n },\n )", "response": "Builds the model function for use in an Estimator."} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nreturn the summary of the learned topics.", "response": "def get_topics_strings(topics_words, alpha, vocabulary,\n topics_to_print=10, words_per_topic=10):\n \"\"\"Returns the summary of the learned topics.\n\n Arguments:\n topics_words: KxV tensor with topics as rows and words as columns.\n alpha: 1xK tensor of prior Dirichlet concentrations for the\n topics.\n vocabulary: A mapping of word's integer index to the corresponding string.\n topics_to_print: The number of topics with highest prior weight to\n summarize.\n words_per_topic: Number of wodrs per topic to return.\n\n Returns:\n summary: A np.array with strings.\n \"\"\"\n alpha = np.squeeze(alpha, axis=0)\n # Use a stable sorting algorithm so that when alpha is fixed\n # we always get the same topics.\n highest_weight_topics = np.argsort(-alpha, kind=\"mergesort\")\n top_words = np.argsort(-topics_words, axis=1)\n\n res = []\n for topic_idx in highest_weight_topics[:topics_to_print]:\n l = [\"index={} alpha={:.2f}\".format(topic_idx, alpha[topic_idx])]\n l += [vocabulary[word] for word in top_words[topic_idx, :words_per_topic]]\n res.append(\" \".join(l))\n\n return np.array(res)"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef build_fake_input_fns(batch_size):\n num_words = 1000\n vocabulary = [str(i) for i in range(num_words)]\n\n random_sample = np.random.randint(\n 10, size=(batch_size, num_words)).astype(np.float32)\n\n def train_input_fn():\n dataset = tf.data.Dataset.from_tensor_slices(random_sample)\n dataset = dataset.batch(batch_size).repeat()\n return tf.compat.v1.data.make_one_shot_iterator(dataset).get_next()\n\n def eval_input_fn():\n dataset = tf.data.Dataset.from_tensor_slices(random_sample)\n dataset = dataset.batch(batch_size)\n return tf.compat.v1.data.make_one_shot_iterator(dataset).get_next()\n\n return train_input_fn, eval_input_fn, vocabulary", "response": "Builds fake data for unit testing."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef build_input_fns(data_dir, batch_size):\n\n with open(download(data_dir, \"vocab.pkl\"), \"r\") as f:\n words_to_idx = pickle.load(f)\n num_words = len(words_to_idx)\n\n vocabulary = [None] * num_words\n for word, idx in words_to_idx.items():\n vocabulary[idx] = word\n\n # Build an iterator over training batches.\n def train_input_fn():\n dataset = newsgroups_dataset(\n data_dir, \"train\", num_words, shuffle_and_repeat=True)\n # Prefetching makes training about 1.5x faster.\n dataset = dataset.batch(batch_size).prefetch(32)\n return tf.compat.v1.data.make_one_shot_iterator(dataset).get_next()\n\n # Build an iterator over the heldout set.\n def eval_input_fn():\n dataset = newsgroups_dataset(\n data_dir, \"test\", num_words, shuffle_and_repeat=False)\n dataset = dataset.batch(batch_size)\n return tf.compat.v1.data.make_one_shot_iterator(dataset).get_next()\n\n return train_input_fn, eval_input_fn, vocabulary", "response": "Builds iterators for train and evaluation data."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\ncalculating the batched KL divergence KL with a and b Chi.", "response": "def _kl_chi_chi(a, b, name=None):\n \"\"\"Calculate the batched KL divergence KL(a || b) with a and b Chi.\n\n Args:\n a: instance of a Chi distribution object.\n b: instance of a Chi distribution object.\n name: (optional) Name to use for created operations.\n default is \"kl_chi_chi\".\n\n Returns:\n Batchwise KL(a || b)\n \"\"\"\n with tf.name_scope(name or \"kl_chi_chi\"):\n # Consistent with\n # https://mast.queensu.ca/~communications/Papers/gil-msc11.pdf, page 118\n # The paper introduces an additional scaling parameter; setting that\n # parameter to 1 and simplifying yields the expression we use here.\n return (0.5 * tf.math.digamma(0.5 * a.df) * (a.df - b.df) +\n tf.math.lgamma(0.5 * b.df) - tf.math.lgamma(0.5 * a.df))"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nreturn a dense column of a Tensor or SparseTensor.", "response": "def _sparse_or_dense_matmul_onehot(sparse_or_dense_matrix, col_index):\n \"\"\"Returns a (dense) column of a Tensor or SparseTensor.\n\n Args:\n sparse_or_dense_matrix: matrix-shaped, `float` `Tensor` or `SparseTensor`.\n col_index: scalar, `int` `Tensor` representing the index of the desired\n column.\n\n Returns:\n column: vector-shaped, `float` `Tensor` with the same dtype as\n `sparse_or_dense_matrix`, representing the `col_index`th column of\n `sparse_or_dense_matrix`.\n \"\"\"\n if isinstance(sparse_or_dense_matrix,\n (tf.SparseTensor, tf.compat.v1.SparseTensorValue)):\n # TODO(b/111924846): Implement better (ideally in a way that allows us to\n # eliminate the `num_rows` arg, if possible).\n num_rows = _get_shape(sparse_or_dense_matrix)[-2]\n batch_shape = _get_shape(sparse_or_dense_matrix)[:-2]\n slice_start = tf.concat([tf.zeros_like(batch_shape), [0, col_index]],\n axis=0)\n slice_size = tf.concat([batch_shape, [num_rows, 1]], axis=0)\n # We momentarily lose static shape information in tf.sparse_slice. However\n # we regain it in the following tf.reshape.\n sparse_slice = tf.sparse.slice(sparse_or_dense_matrix,\n tf.cast(slice_start, tf.int64),\n tf.cast(slice_size, tf.int64))\n\n output_shape = tf.concat([batch_shape, [num_rows]], axis=0)\n return tf.reshape(tf.sparse.to_dense(sparse_slice), output_shape)\n else:\n return tf.gather(sparse_or_dense_matrix, col_index, axis=-1)"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef minimize(grad_and_hessian_loss_fn,\n x_start,\n tolerance,\n l1_regularizer,\n l2_regularizer=None,\n maximum_iterations=1,\n maximum_full_sweeps_per_iteration=1,\n learning_rate=None,\n name=None):\n \"\"\"Minimize using Hessian-informed proximal gradient descent.\n\n This function solves the regularized minimization problem\n\n ```none\n argmin{ Loss(x)\n + l1_regularizer * ||x||_1\n + l2_regularizer * ||x||_2**2\n : x in R^n }\n ```\n\n where `Loss` is a convex C^2 function (typically, `Loss` is the negative log\n likelihood of a model and `x` is a vector of model coefficients). The `Loss`\n function does not need to be supplied directly, but this optimizer does need a\n way to compute the gradient and Hessian of the Loss function at a given value\n of `x`. The gradient and Hessian are often computationally expensive, and\n this optimizer calls them relatively few times compared with other algorithms.\n\n Args:\n grad_and_hessian_loss_fn: callable that takes as input a (batch of) `Tensor`\n of the same shape and dtype as `x_start` and returns the triple\n `(gradient_unregularized_loss, hessian_unregularized_loss_outer,\n hessian_unregularized_loss_middle)` as defined in the argument spec of\n `minimize_one_step`.\n x_start: (Batch of) vector-shaped, `float` `Tensor` representing the initial\n value of the argument to the `Loss` function.\n tolerance: scalar, `float` `Tensor` representing the tolerance for each\n optimization step; see the `tolerance` argument of\n `minimize_one_step`.\n l1_regularizer: scalar, `float` `Tensor` representing the weight of the L1\n regularization term (see equation above).\n l2_regularizer: scalar, `float` `Tensor` representing the weight of the L2\n regularization term (see equation above).\n Default value: `None` (i.e., no L2 regularization).\n maximum_iterations: Python integer specifying the maximum number of\n iterations of the outer loop of the optimizer. After this many iterations\n of the outer loop, the algorithm will terminate even if the return value\n `optimal_x` has not converged.\n Default value: `1`.\n maximum_full_sweeps_per_iteration: Python integer specifying the maximum\n number of sweeps allowed in each iteration of the outer loop of the\n optimizer. Passed as the `maximum_full_sweeps` argument to\n `minimize_one_step`.\n Default value: `1`.\n learning_rate: scalar, `float` `Tensor` representing a multiplicative factor\n used to dampen the proximal gradient descent steps.\n Default value: `None` (i.e., factor is conceptually `1`).\n name: Python string representing the name of the TensorFlow operation.\n The default name is `\"minimize\"`.\n\n Returns:\n x: `Tensor` of the same shape and dtype as `x_start`, representing the\n (batches of) computed values of `x` which minimizes `Loss(x)`.\n is_converged: scalar, `bool` `Tensor` indicating whether the minimization\n procedure converged within the specified number of iterations across all\n batches. Here convergence means that an iteration of the inner loop\n (`minimize_one_step`) returns `True` for its `is_converged` output value.\n iter: scalar, `int` `Tensor` indicating the actual number of iterations of\n the outer loop of the optimizer completed (i.e., number of calls to\n `minimize_one_step` before achieving convergence).\n\n #### References\n\n [1]: Jerome Friedman, Trevor Hastie and Rob Tibshirani. Regularization Paths\n for Generalized Linear Models via Coordinate Descent. _Journal of\n Statistical Software_, 33(1), 2010.\n https://www.jstatsoft.org/article/view/v033i01/v33i01.pdf\n\n [2]: Guo-Xun Yuan, Chia-Hua Ho and Chih-Jen Lin. An Improved GLMNET for\n L1-regularized Logistic Regression. _Journal of Machine Learning\n Research_, 13, 2012.\n http://www.jmlr.org/papers/volume13/yuan12a/yuan12a.pdf\n \"\"\"\n graph_deps = [\n x_start,\n l1_regularizer,\n l2_regularizer,\n maximum_iterations,\n maximum_full_sweeps_per_iteration,\n tolerance,\n learning_rate,\n ],\n with tf.compat.v1.name_scope(name, 'minimize', graph_deps):\n\n def _loop_cond(x_start, converged, iter_):\n del x_start\n return tf.logical_and(iter_ < maximum_iterations,\n tf.logical_not(converged))\n\n def _loop_body(x_start, converged, iter_): # pylint: disable=missing-docstring\n g, h_outer, h_middle = grad_and_hessian_loss_fn(x_start)\n x_start, converged, _ = minimize_one_step(\n gradient_unregularized_loss=g,\n hessian_unregularized_loss_outer=h_outer,\n hessian_unregularized_loss_middle=h_middle,\n x_start=x_start,\n l1_regularizer=l1_regularizer,\n l2_regularizer=l2_regularizer,\n maximum_full_sweeps=maximum_full_sweeps_per_iteration,\n tolerance=tolerance,\n learning_rate=learning_rate)\n return x_start, converged, iter_ + 1\n\n return tf.while_loop(\n cond=_loop_cond,\n body=_loop_body,\n loop_vars=[\n x_start,\n tf.zeros([], np.bool, name='converged'),\n tf.zeros([], np.int32, name='iter'),\n ])", "response": "Minimize using regularized minimization problem."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\ncreate the encoder function.", "response": "def make_encoder(base_depth, activation, latent_size, code_size):\n \"\"\"Creates the encoder function.\n\n Args:\n base_depth: Layer base depth in encoder net.\n activation: Activation function in hidden layers.\n latent_size: The number of latent variables in the code.\n code_size: The dimensionality of each latent variable.\n\n Returns:\n encoder: A `callable` mapping a `Tensor` of images to a `Tensor` of shape\n `[..., latent_size, code_size]`.\n \"\"\"\n conv = functools.partial(\n tf.keras.layers.Conv2D, padding=\"SAME\", activation=activation)\n\n encoder_net = tf.keras.Sequential([\n conv(base_depth, 5, 1),\n conv(base_depth, 5, 2),\n conv(2 * base_depth, 5, 1),\n conv(2 * base_depth, 5, 2),\n conv(4 * latent_size, 7, padding=\"VALID\"),\n tf.keras.layers.Flatten(),\n tf.keras.layers.Dense(latent_size * code_size, activation=None),\n tf.keras.layers.Reshape([latent_size, code_size])\n ])\n\n def encoder(images):\n \"\"\"Encodes a batch of images.\n\n Args:\n images: A `Tensor` representing the inputs to be encoded, of shape `[...,\n channels]`.\n\n Returns:\n codes: A `float`-like `Tensor` of shape `[..., latent_size, code_size]`.\n It represents latent vectors to be matched with the codebook.\n \"\"\"\n images = 2 * tf.cast(images, dtype=tf.float32) - 1\n codes = encoder_net(images)\n return codes\n\n return encoder"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef make_decoder(base_depth, activation, input_size, output_shape):\n deconv = functools.partial(\n tf.keras.layers.Conv2DTranspose, padding=\"SAME\", activation=activation)\n conv = functools.partial(\n tf.keras.layers.Conv2D, padding=\"SAME\", activation=activation)\n decoder_net = tf.keras.Sequential([\n tf.keras.layers.Reshape((1, 1, input_size)),\n deconv(2 * base_depth, 7, padding=\"VALID\"),\n deconv(2 * base_depth, 5),\n deconv(2 * base_depth, 5, 2),\n deconv(base_depth, 5),\n deconv(base_depth, 5, 2),\n deconv(base_depth, 5),\n conv(output_shape[-1], 5, activation=None),\n tf.keras.layers.Reshape(output_shape),\n ])\n\n def decoder(codes):\n \"\"\"Builds a distribution over images given codes.\n\n Args:\n codes: A `Tensor` representing the inputs to be decoded, of shape `[...,\n code_size]`.\n\n Returns:\n decoder_distribution: A multivariate `Bernoulli` distribution.\n \"\"\"\n logits = decoder_net(codes)\n return tfd.Independent(tfd.Bernoulli(logits=logits),\n reinterpreted_batch_ndims=len(output_shape),\n name=\"decoder_distribution\")\n\n return decoder", "response": "Creates the decoder function."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nadds control dependencies to the commmitment loss.", "response": "def add_ema_control_dependencies(vector_quantizer,\n one_hot_assignments,\n codes,\n commitment_loss,\n decay):\n \"\"\"Add control dependencies to the commmitment loss to update the codebook.\n\n Args:\n vector_quantizer: An instance of the VectorQuantizer class.\n one_hot_assignments: The one-hot vectors corresponding to the matched\n codebook entry for each code in the batch.\n codes: A `float`-like `Tensor` containing the latent vectors to be compared\n to the codebook.\n commitment_loss: The commitment loss from comparing the encoder outputs to\n their neighboring codebook entries.\n decay: Decay factor for exponential moving average.\n\n Returns:\n commitment_loss: Commitment loss with control dependencies.\n \"\"\"\n # Use an exponential moving average to update the codebook.\n updated_ema_count = moving_averages.assign_moving_average(\n vector_quantizer.ema_count,\n tf.reduce_sum(input_tensor=one_hot_assignments, axis=[0, 1]),\n decay,\n zero_debias=False)\n updated_ema_means = moving_averages.assign_moving_average(\n vector_quantizer.ema_means,\n tf.reduce_sum(\n input_tensor=tf.expand_dims(codes, 2) *\n tf.expand_dims(one_hot_assignments, 3),\n axis=[0, 1]),\n decay,\n zero_debias=False)\n\n # Add small value to avoid dividing by zero.\n perturbed_ema_count = updated_ema_count + 1e-5\n with tf.control_dependencies([commitment_loss]):\n update_means = tf.compat.v1.assign(\n vector_quantizer.codebook,\n updated_ema_means / perturbed_ema_count[..., tf.newaxis])\n with tf.control_dependencies([update_means]):\n return tf.identity(commitment_loss)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nsaving a grid of images to a PNG file.", "response": "def save_imgs(x, fname):\n \"\"\"Helper method to save a grid of images to a PNG file.\n\n Args:\n x: A numpy array of shape [n_images, height, width].\n fname: The filename to write to (including extension).\n \"\"\"\n n = x.shape[0]\n fig = figure.Figure(figsize=(n, 1), frameon=False)\n canvas = backend_agg.FigureCanvasAgg(fig)\n for i in range(n):\n ax = fig.add_subplot(1, n, i+1)\n ax.imshow(x[i].squeeze(),\n interpolation=\"none\",\n cmap=cm.get_cmap(\"binary\"))\n ax.axis(\"off\")\n canvas.print_figure(fname, format=\"png\")\n print(\"saved %s\" % fname)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef load_bernoulli_mnist_dataset(directory, split_name):\n amat_file = download(directory, FILE_TEMPLATE.format(split=split_name))\n dataset = tf.data.TextLineDataset(amat_file)\n str_to_arr = lambda string: np.array([c == b\"1\" for c in string.split()])\n\n def _parser(s):\n booltensor = tf.compat.v1.py_func(str_to_arr, [s], tf.bool)\n reshaped = tf.reshape(booltensor, [28, 28, 1])\n return tf.cast(reshaped, dtype=tf.float32), tf.constant(0, tf.int32)\n\n return dataset.map(_parser)", "response": "Returns Hugo Larochelle s binary static MNIST dataset."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nbuilding an iterator that can switch between train and heldout data.", "response": "def build_input_pipeline(data_dir, batch_size, heldout_size, mnist_type):\n \"\"\"Builds an Iterator switching between train and heldout data.\"\"\"\n # Build an iterator over training batches.\n if mnist_type in [MnistType.FAKE_DATA, MnistType.THRESHOLD]:\n if mnist_type == MnistType.FAKE_DATA:\n mnist_data = build_fake_data()\n else:\n mnist_data = mnist.read_data_sets(data_dir)\n training_dataset = tf.data.Dataset.from_tensor_slices(\n (mnist_data.train.images, np.int32(mnist_data.train.labels)))\n heldout_dataset = tf.data.Dataset.from_tensor_slices(\n (mnist_data.validation.images,\n np.int32(mnist_data.validation.labels)))\n elif mnist_type == MnistType.BERNOULLI:\n training_dataset = load_bernoulli_mnist_dataset(data_dir, \"train\")\n heldout_dataset = load_bernoulli_mnist_dataset(data_dir, \"valid\")\n else:\n raise ValueError(\"Unknown MNIST type.\")\n\n training_batches = training_dataset.repeat().batch(batch_size)\n training_iterator = tf.compat.v1.data.make_one_shot_iterator(training_batches)\n\n # Build a iterator over the heldout set with batch_size=heldout_size,\n # i.e., return the entire heldout set as a constant.\n heldout_frozen = (heldout_dataset.take(heldout_size).\n repeat().batch(heldout_size))\n heldout_iterator = tf.compat.v1.data.make_one_shot_iterator(heldout_frozen)\n\n # Combine these into a feedable iterator that can switch between training\n # and validation inputs.\n handle = tf.compat.v1.placeholder(tf.string, shape=[])\n feedable_iterator = tf.compat.v1.data.Iterator.from_string_handle(\n handle, training_batches.output_types, training_batches.output_shapes)\n images, labels = feedable_iterator.get_next()\n # Reshape as a pixel image and binarize pixels.\n images = tf.reshape(images, shape=[-1] + IMAGE_SHAPE)\n if mnist_type in [MnistType.FAKE_DATA, MnistType.THRESHOLD]:\n images = tf.cast(images > 0.5, dtype=tf.int32)\n\n return images, labels, handle, training_iterator, heldout_iterator"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nreturn a np. dtype based on this dtype.", "response": "def as_numpy_dtype(dtype):\n \"\"\"Returns a `np.dtype` based on this `dtype`.\"\"\"\n dtype = tf.as_dtype(dtype)\n if hasattr(dtype, 'as_numpy_dtype'):\n return dtype.as_numpy_dtype\n return dtype"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nreturns a non - reference dtype based on this dtype.", "response": "def base_dtype(dtype):\n \"\"\"Returns a non-reference `dtype` based on this `dtype`.\"\"\"\n dtype = tf.as_dtype(dtype)\n if hasattr(dtype, 'base_dtype'):\n return dtype.base_dtype\n return dtype"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nreturns whether this is a boolean data type.", "response": "def is_bool(dtype):\n \"\"\"Returns whether this is a boolean data type.\"\"\"\n dtype = tf.as_dtype(dtype)\n if hasattr(dtype, 'is_bool'):\n return dtype.is_bool\n # We use `kind` because:\n # np.issubdtype(np.uint8, np.bool) == True.\n return np.dtype(dtype).kind == 'b'"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nreturns whether this is a complex floating point type.", "response": "def is_complex(dtype):\n \"\"\"Returns whether this is a complex floating point type.\"\"\"\n dtype = tf.as_dtype(dtype)\n if hasattr(dtype, 'is_complex'):\n return dtype.is_complex\n return np.issubdtype(np.dtype(dtype), np.complex)"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nreturn whether this is a non - quantized floating point type.", "response": "def is_floating(dtype):\n \"\"\"Returns whether this is a (non-quantized, real) floating point type.\"\"\"\n dtype = tf.as_dtype(dtype)\n if hasattr(dtype, 'is_floating'):\n return dtype.is_floating\n return np.issubdtype(np.dtype(dtype), np.float)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef is_integer(dtype):\n dtype = tf.as_dtype(dtype)\n if hasattr(dtype, 'is_integer'):\n return dtype.is_integer\n return np.issubdtype(np.dtype(dtype), np.integer)", "response": "Returns whether this is a non - quantized integer type."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef max(dtype): # pylint: disable=redefined-builtin\n dtype = tf.as_dtype(dtype)\n if hasattr(dtype, 'max'):\n return dtype.max\n use_finfo = is_floating(dtype) or is_complex(dtype)\n return np.finfo(dtype).max if use_finfo else np.iinfo(dtype).max", "response": "Returns the maximum representable value in this data type."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef name(dtype):\n dtype = tf.as_dtype(dtype)\n if hasattr(dtype, 'name'):\n return dtype.name\n if hasattr(dtype, '__name__'):\n return dtype.__name__\n return str(dtype)", "response": "Returns the string name for this dtype."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nreturns the number of bytes to represent this dtype.", "response": "def size(dtype):\n \"\"\"Returns the number of bytes to represent this `dtype`.\"\"\"\n dtype = tf.as_dtype(dtype)\n if hasattr(dtype, 'size'):\n return dtype.size\n return np.dtype(dtype).itemsize"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _assert_same_base_type(items, expected_type=None):\n original_expected_type = expected_type\n mismatch = False\n for item in items:\n if item is not None:\n item_type = base_dtype(item.dtype)\n if not expected_type:\n expected_type = item_type\n elif expected_type != item_type:\n mismatch = True\n break\n if mismatch:\n # Loop back through and build up an informative error message (this is very\n # slow, so we don't do it unless we found an error above).\n expected_type = original_expected_type\n original_item_str = None\n get_name = lambda x: x.name if hasattr(x, 'name') else str(x)\n for item in items:\n if item is not None:\n item_type = base_dtype(item.dtype)\n if not expected_type:\n expected_type = item_type\n original_item_str = get_name(item)\n elif expected_type != item_type:\n raise ValueError(\n '{}, type={}, must be of the same type ({}){}.'.format(\n get_name(item),\n item_type,\n expected_type,\n ((' as {}'.format(original_item_str))\n if original_item_str else '')))\n return expected_type # Should be unreachable\n else:\n return expected_type", "response": "r Asserts all items are of the same base type."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef assert_same_float_dtype(tensors=None, dtype=None):\n if tensors:\n dtype = _assert_same_base_type(tensors, dtype)\n if not dtype:\n dtype = tf.float32\n elif not is_floating(dtype):\n raise ValueError('Expected floating point type, got {}.'.format(dtype))\n return dtype", "response": "Validate and return float type based on tensors and dtype."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _kl_categorical_categorical(a, b, name=None):\n with tf.name_scope(name or \"kl_categorical_categorical\"):\n # sum(p ln(p / q))\n return tf.reduce_sum(\n input_tensor=tf.nn.softmax(a.logits) *\n (tf.nn.log_softmax(a.logits) - tf.nn.log_softmax(b.logits)),\n axis=-1)", "response": "Calculate the batched KL divergence KL with a b OneHotCategorical distribution."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nminimizing an objective function using the Nelder Mead simplex algorithm.", "response": "def minimize(objective_function,\n initial_simplex=None,\n initial_vertex=None,\n step_sizes=None,\n objective_at_initial_simplex=None,\n objective_at_initial_vertex=None,\n batch_evaluate_objective=False,\n func_tolerance=1e-8,\n position_tolerance=1e-8,\n parallel_iterations=1,\n max_iterations=None,\n reflection=None,\n expansion=None,\n contraction=None,\n shrinkage=None,\n name=None):\n \"\"\"Minimum of the objective function using the Nelder Mead simplex algorithm.\n\n Performs an unconstrained minimization of a (possibly non-smooth) function\n using the Nelder Mead simplex method. Nelder Mead method does not support\n univariate functions. Hence the dimensions of the domain must be 2 or greater.\n For details of the algorithm, see\n [Press, Teukolsky, Vetterling and Flannery(2007)][1].\n\n Points in the domain of the objective function may be represented as a\n `Tensor` of general shape but with rank at least 1. The algorithm proceeds\n by modifying a full rank simplex in the domain. The initial simplex may\n either be specified by the user or can be constructed using a single vertex\n supplied by the user. In the latter case, if `v0` is the supplied vertex,\n the simplex is the convex hull of the set:\n\n ```None\n S = {v0} + {v0 + step_i * e_i}\n ```\n\n Here `e_i` is a vector which is `1` along the `i`-th axis and zero elsewhere\n and `step_i` is a characteristic length scale along the `i`-th axis. If the\n step size is not supplied by the user, a unit step size is used in every axis.\n Alternately, a single step size may be specified which is used for every\n axis. The most flexible option is to supply a bespoke step size for every\n axis.\n\n ### Usage:\n\n The following example demonstrates the usage of the Nelder Mead minimzation\n on a two dimensional problem with the minimum located at a non-differentiable\n point.\n\n ```python\n # The objective function\n def sqrt_quadratic(x):\n return tf.sqrt(tf.reduce_sum(x ** 2, axis=-1))\n\n start = tf.constant([6.0, -21.0]) # Starting point for the search.\n optim_results = tfp.optimizer.nelder_mead_minimize(\n sqrt_quadratic, initial_vertex=start, func_tolerance=1e-8,\n batch_evaluate_objective=True)\n\n with tf.Session() as session:\n results = session.run(optim_results)\n # Check that the search converged\n assert(results.converged)\n # Check that the argmin is close to the actual value.\n np.testing.assert_allclose(results.position, np.array([0.0, 0.0]),\n atol=1e-7)\n # Print out the total number of function evaluations it took.\n print (\"Function evaluations: %d\" % results.num_objective_evaluations)\n ```\n\n ### References:\n [1]: William Press, Saul Teukolsky, William Vetterling and Brian Flannery.\n Numerical Recipes in C++, third edition. pp. 502-507. (2007).\n http://numerical.recipes/cpppages/chap0sel.pdf\n\n [2]: Jeffrey Lagarias, James Reeds, Margaret Wright and Paul Wright.\n Convergence properties of the Nelder-Mead simplex method in low dimensions,\n Siam J. Optim., Vol 9, No. 1, pp. 112-147. (1998).\n http://www.math.kent.edu/~reichel/courses/Opt/reading.material.2/nelder.mead.pdf\n\n [3]: Fuchang Gao and Lixing Han. Implementing the Nelder-Mead simplex\n algorithm with adaptive parameters. Computational Optimization and\n Applications, Vol 51, Issue 1, pp 259-277. (2012).\n https://pdfs.semanticscholar.org/15b4/c4aa7437df4d032c6ee6ce98d6030dd627be.pdf\n\n Args:\n objective_function: A Python callable that accepts a point as a\n real `Tensor` and returns a `Tensor` of real dtype containing\n the value of the function at that point. The function\n to be minimized. If `batch_evaluate_objective` is `True`, the callable\n may be evaluated on a `Tensor` of shape `[n+1] + s ` where `n` is\n the dimension of the problem and `s` is the shape of a single point\n in the domain (so `n` is the size of a `Tensor` representing a\n single point).\n In this case, the expected return value is a `Tensor` of shape `[n+1]`.\n Note that this method does not support univariate functions so the problem\n dimension `n` must be strictly greater than 1.\n initial_simplex: (Optional) `Tensor` of real dtype. The initial simplex to\n start the search. If supplied, should be a `Tensor` of shape `[n+1] + s`\n where `n` is the dimension of the problem and `s` is the shape of a\n single point in the domain. Each row (i.e. the `Tensor` with a given\n value of the first index) is interpreted as a vertex of a simplex and\n hence the rows must be affinely independent. If not supplied, an axes\n aligned simplex is constructed using the `initial_vertex` and\n `step_sizes`. Only one and at least one of `initial_simplex` and\n `initial_vertex` must be supplied.\n initial_vertex: (Optional) `Tensor` of real dtype and any shape that can\n be consumed by the `objective_function`. A single point in the domain that\n will be used to construct an axes aligned initial simplex.\n step_sizes: (Optional) `Tensor` of real dtype and shape broadcasting\n compatible with `initial_vertex`. Supplies the simplex scale along each\n axes. Only used if `initial_simplex` is not supplied. See description\n above for details on how step sizes and initial vertex are used to\n construct the initial simplex.\n objective_at_initial_simplex: (Optional) Rank `1` `Tensor` of real dtype\n of a rank `1` `Tensor`. The value of the objective function at the\n initial simplex. May be supplied only if `initial_simplex` is\n supplied. If not supplied, it will be computed.\n objective_at_initial_vertex: (Optional) Scalar `Tensor` of real dtype. The\n value of the objective function at the initial vertex. May be supplied\n only if the `initial_vertex` is also supplied.\n batch_evaluate_objective: (Optional) Python `bool`. If True, the objective\n function will be evaluated on all the vertices of the simplex packed\n into a single tensor. If False, the objective will be mapped across each\n vertex separately. Evaluating the objective function in a batch allows\n use of vectorization and should be preferred if the objective function\n allows it.\n func_tolerance: (Optional) Scalar `Tensor` of real dtype. The algorithm\n stops if the absolute difference between the largest and the smallest\n function value on the vertices of the simplex is below this number.\n position_tolerance: (Optional) Scalar `Tensor` of real dtype. The\n algorithm stops if the largest absolute difference between the\n coordinates of the vertices is below this threshold.\n parallel_iterations: (Optional) Positive integer. The number of iterations\n allowed to run in parallel.\n max_iterations: (Optional) Scalar positive `Tensor` of dtype `int32`.\n The maximum number of iterations allowed. If `None` then no limit is\n applied.\n reflection: (Optional) Positive Scalar `Tensor` of same dtype as\n `initial_vertex`. This parameter controls the scaling of the reflected\n vertex. See, [Press et al(2007)][1] for details. If not specified,\n uses the dimension dependent prescription of [Gao and Han(2012)][3].\n expansion: (Optional) Positive Scalar `Tensor` of same dtype as\n `initial_vertex`. Should be greater than `1` and `reflection`. This\n parameter controls the expanded scaling of a reflected vertex.\n See, [Press et al(2007)][1] for details. If not specified, uses the\n dimension dependent prescription of [Gao and Han(2012)][3].\n contraction: (Optional) Positive scalar `Tensor` of same dtype as\n `initial_vertex`. Must be between `0` and `1`. This parameter controls\n the contraction of the reflected vertex when the objective function at\n the reflected point fails to show sufficient decrease.\n See, [Press et al(2007)][1] for more details. If not specified, uses\n the dimension dependent prescription of [Gao and Han(2012][3].\n shrinkage: (Optional) Positive scalar `Tensor` of same dtype as\n `initial_vertex`. Must be between `0` and `1`. This parameter is the scale\n by which the simplex is shrunk around the best point when the other\n steps fail to produce improvements.\n See, [Press et al(2007)][1] for more details. If not specified, uses\n the dimension dependent prescription of [Gao and Han(2012][3].\n name: (Optional) Python str. The name prefixed to the ops created by this\n function. If not supplied, the default name 'minimize' is used.\n\n Returns:\n optimizer_results: A namedtuple containing the following items:\n converged: Scalar boolean tensor indicating whether the minimum was\n found within tolerance.\n num_objective_evaluations: The total number of objective\n evaluations performed.\n position: A `Tensor` containing the last argument value found\n during the search. If the search converged, then\n this value is the argmin of the objective function.\n objective_value: A tensor containing the value of the objective\n function at the `position`. If the search\n converged, then this is the (local) minimum of\n the objective function.\n final_simplex: The last simplex constructed before stopping.\n final_objective_values: The objective function evaluated at the\n vertices of the final simplex.\n initial_simplex: The starting simplex.\n initial_objective_values: The objective function evaluated at the\n vertices of the initial simplex.\n num_iterations: The number of iterations of the main algorithm body.\n\n Raises:\n ValueError: If any of the following conditions hold\n 1. If none or more than one of `initial_simplex` and `initial_vertex` are\n supplied.\n 2. If `initial_simplex` and `step_sizes` are both specified.\n \"\"\"\n with tf.compat.v1.name_scope(name, 'minimize', [\n initial_simplex, initial_vertex, step_sizes, objective_at_initial_simplex,\n objective_at_initial_vertex, func_tolerance, position_tolerance\n ]):\n (\n dim,\n _,\n simplex,\n objective_at_simplex,\n num_evaluations\n ) = _prepare_args(objective_function,\n initial_simplex,\n initial_vertex,\n step_sizes,\n objective_at_initial_simplex,\n objective_at_initial_vertex,\n batch_evaluate_objective)\n domain_dtype = simplex.dtype\n (\n reflection,\n expansion,\n contraction,\n shrinkage\n ) = _resolve_parameters(dim,\n reflection,\n expansion,\n contraction,\n shrinkage,\n domain_dtype)\n\n closure_kwargs = dict(\n objective_function=objective_function,\n dim=dim,\n func_tolerance=func_tolerance,\n position_tolerance=position_tolerance,\n batch_evaluate_objective=batch_evaluate_objective,\n reflection=reflection,\n expansion=expansion,\n contraction=contraction,\n shrinkage=shrinkage)\n\n def _loop_body(_, iterations, simplex, objective_at_simplex,\n num_evaluations):\n (\n converged,\n next_simplex,\n next_objective,\n evaluations\n ) = nelder_mead_one_step(simplex, objective_at_simplex, **closure_kwargs)\n\n return (converged, iterations + 1, next_simplex, next_objective,\n num_evaluations + evaluations)\n\n initial_args = (False, 0, simplex, objective_at_simplex,\n num_evaluations)\n # Loop until either we have converged or if the max iterations are supplied\n # then until we have converged or exhausted the available iteration budget.\n def _is_converged(converged, num_iterations, *ignored_args): # pylint:disable=unused-argument\n # It is important to ensure that not_converged is a tensor. If\n # converged is not a tensor but a Python bool, then the overloaded\n # op '~' acts as bitwise complement so ~True = -2 and ~False = -1.\n # In that case, the loop will never terminate.\n not_converged = tf.logical_not(converged)\n return (not_converged if max_iterations is None\n else (not_converged & (num_iterations < max_iterations)))\n\n (converged, num_iterations, final_simplex, final_objective_values,\n final_evaluations) = tf.while_loop(\n cond=_is_converged,\n body=_loop_body,\n loop_vars=initial_args,\n parallel_iterations=parallel_iterations)\n order = tf.argsort(\n final_objective_values, direction='ASCENDING', stable=True)\n best_index = order[0]\n # The explicit cast to Tensor below is done to avoid returning a mixture\n # of Python types and Tensors which cause problems with session.run.\n # In the eager mode, converged may remain a Python bool. Trying to evaluate\n # the whole tuple in one evaluate call will raise an exception because\n # of the presence of non-tensors. This is very annoying so we explicitly\n # cast those arguments to Tensors.\n return NelderMeadOptimizerResults(\n converged=tf.convert_to_tensor(value=converged),\n num_objective_evaluations=final_evaluations,\n position=final_simplex[best_index],\n objective_value=final_objective_values[best_index],\n final_simplex=final_simplex,\n final_objective_values=final_objective_values,\n num_iterations=tf.convert_to_tensor(value=num_iterations),\n initial_simplex=simplex,\n initial_objective_values=objective_at_simplex)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _accept_reflected_fn(simplex,\n objective_values,\n worst_index,\n reflected,\n objective_at_reflected):\n \"\"\"Creates the condition function pair for a reflection to be accepted.\"\"\"\n def _replace_worst_with_reflected():\n next_simplex = _replace_at_index(simplex, worst_index, reflected)\n next_objective_values = _replace_at_index(objective_values, worst_index,\n objective_at_reflected)\n return False, next_simplex, next_objective_values, 0\n return _replace_worst_with_reflected", "response": "Creates a condition function pair for a reflection to be accepted."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef _expansion_fn(objective_function,\n simplex,\n objective_values,\n worst_index,\n reflected,\n objective_at_reflected,\n face_centroid,\n expansion):\n \"\"\"Creates the condition function pair for an expansion.\"\"\"\n def _expand_and_maybe_replace():\n \"\"\"Performs the expansion step.\"\"\"\n expanded = face_centroid + expansion * (reflected - face_centroid)\n expanded_objective_value = objective_function(expanded)\n expanded_is_better = (expanded_objective_value <\n objective_at_reflected)\n accept_expanded_fn = lambda: (expanded, expanded_objective_value)\n accept_reflected_fn = lambda: (reflected, objective_at_reflected)\n next_pt, next_objective_value = prefer_static.cond(\n expanded_is_better, accept_expanded_fn, accept_reflected_fn)\n next_simplex = _replace_at_index(simplex, worst_index, next_pt)\n next_objective_at_simplex = _replace_at_index(objective_values,\n worst_index,\n next_objective_value)\n return False, next_simplex, next_objective_at_simplex, 1\n return _expand_and_maybe_replace", "response": "Creates a condition function pair for an expansion."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\ncreating a condition function pair for an outside contraction.", "response": "def _outside_contraction_fn(objective_function,\n simplex,\n objective_values,\n face_centroid,\n best_index,\n worst_index,\n reflected,\n objective_at_reflected,\n contraction,\n shrinkage,\n batch_evaluate_objective):\n \"\"\"Creates the condition function pair for an outside contraction.\"\"\"\n def _contraction():\n \"\"\"Performs a contraction.\"\"\"\n contracted = face_centroid + contraction * (reflected - face_centroid)\n objective_at_contracted = objective_function(contracted)\n is_contracted_acceptable = objective_at_contracted <= objective_at_reflected\n def _accept_contraction():\n next_simplex = _replace_at_index(simplex, worst_index, contracted)\n objective_at_next_simplex = _replace_at_index(\n objective_values,\n worst_index,\n objective_at_contracted)\n return (False,\n next_simplex,\n objective_at_next_simplex,\n 1)\n\n def _reject_contraction():\n return _shrink_towards_best(objective_function,\n simplex,\n best_index,\n shrinkage,\n batch_evaluate_objective)\n\n return prefer_static.cond(is_contracted_acceptable,\n _accept_contraction,\n _reject_contraction)\n return _contraction"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _shrink_towards_best(objective_function,\n simplex,\n best_index,\n shrinkage,\n batch_evaluate_objective):\n \"\"\"Shrinks the simplex around the best vertex.\"\"\"\n\n # If the contraction step fails to improve the average objective enough,\n # the simplex is shrunk towards the best vertex.\n best_vertex = simplex[best_index]\n shrunk_simplex = best_vertex + shrinkage * (simplex - best_vertex)\n objective_at_shrunk_simplex, evals = _evaluate_objective_multiple(\n objective_function,\n shrunk_simplex,\n batch_evaluate_objective)\n return (False,\n shrunk_simplex,\n objective_at_shrunk_simplex,\n evals)", "response": "Shrinks the simplex around the best vertex."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _replace_at_index(x, index, replacement):\n x_new = tf.concat([x[:index], tf.expand_dims(replacement, axis=0),\n x[(index + 1):]], axis=0)\n return x_new", "response": "Replaces an element at supplied index."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _check_convergence(simplex,\n best_vertex,\n best_objective,\n worst_objective,\n func_tolerance,\n position_tolerance):\n \"\"\"Returns True if the simplex has converged.\n\n If the simplex size is smaller than the `position_tolerance` or the variation\n of the function value over the vertices of the simplex is smaller than the\n `func_tolerance` return True else False.\n\n Args:\n simplex: `Tensor` of real dtype. The simplex to test for convergence. For\n more details, see the docstring for `initial_simplex` argument\n of `minimize`.\n best_vertex: `Tensor` of real dtype and rank one less than `simplex`. The\n vertex with the best (i.e. smallest) objective value.\n best_objective: Scalar `Tensor` of real dtype. The best (i.e. smallest)\n value of the objective function at a vertex.\n worst_objective: Scalar `Tensor` of same dtype as `best_objective`. The\n worst (i.e. largest) value of the objective function at a vertex.\n func_tolerance: Scalar positive `Tensor`. The tolerance for the variation\n of the objective function value over the simplex. If the variation over\n the simplex vertices is below this threshold, convergence is True.\n position_tolerance: Scalar positive `Tensor`. The algorithm stops if the\n lengths (under the supremum norm) of edges connecting to the best vertex\n are below this threshold.\n\n Returns:\n has_converged: A scalar boolean `Tensor` indicating whether the algorithm\n is deemed to have converged.\n \"\"\"\n objective_convergence = tf.abs(worst_objective -\n best_objective) < func_tolerance\n simplex_degeneracy = tf.reduce_max(\n input_tensor=tf.abs(simplex - best_vertex)) < position_tolerance\n return objective_convergence | simplex_degeneracy", "response": "Checks if the simplex has converged."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _prepare_args(objective_function,\n initial_simplex,\n initial_vertex,\n step_sizes,\n objective_at_initial_simplex,\n objective_at_initial_vertex,\n batch_evaluate_objective):\n \"\"\"Computes the initial simplex and the objective values at the simplex.\n\n Args:\n objective_function: A Python callable that accepts a point as a\n real `Tensor` and returns a `Tensor` of real dtype containing\n the value of the function at that point. The function\n to be evaluated at the simplex. If `batch_evaluate_objective` is `True`,\n the callable may be evaluated on a `Tensor` of shape `[n+1] + s `\n where `n` is the dimension of the problem and `s` is the shape of a\n single point in the domain (so `n` is the size of a `Tensor`\n representing a single point).\n In this case, the expected return value is a `Tensor` of shape `[n+1]`.\n initial_simplex: None or `Tensor` of real dtype. The initial simplex to\n start the search. If supplied, should be a `Tensor` of shape `[n+1] + s`\n where `n` is the dimension of the problem and `s` is the shape of a\n single point in the domain. Each row (i.e. the `Tensor` with a given\n value of the first index) is interpreted as a vertex of a simplex and\n hence the rows must be affinely independent. If not supplied, an axes\n aligned simplex is constructed using the `initial_vertex` and\n `step_sizes`. Only one and at least one of `initial_simplex` and\n `initial_vertex` must be supplied.\n initial_vertex: None or `Tensor` of real dtype and any shape that can\n be consumed by the `objective_function`. A single point in the domain that\n will be used to construct an axes aligned initial simplex.\n step_sizes: None or `Tensor` of real dtype and shape broadcasting\n compatible with `initial_vertex`. Supplies the simplex scale along each\n axes. Only used if `initial_simplex` is not supplied. See the docstring\n of `minimize` for more details.\n objective_at_initial_simplex: None or rank `1` `Tensor` of real dtype.\n The value of the objective function at the initial simplex.\n May be supplied only if `initial_simplex` is\n supplied. If not supplied, it will be computed.\n objective_at_initial_vertex: None or scalar `Tensor` of real dtype. The\n value of the objective function at the initial vertex. May be supplied\n only if the `initial_vertex` is also supplied.\n batch_evaluate_objective: Python `bool`. If True, the objective function\n will be evaluated on all the vertices of the simplex packed into a\n single tensor. If False, the objective will be mapped across each\n vertex separately.\n\n Returns:\n prepared_args: A tuple containing the following elements:\n dimension: Scalar `Tensor` of `int32` dtype. The dimension of the problem\n as inferred from the supplied arguments.\n num_vertices: Scalar `Tensor` of `int32` dtype. The number of vertices\n in the simplex.\n simplex: A `Tensor` of same dtype as `initial_simplex`\n (or `initial_vertex`). The first component of the shape of the\n `Tensor` is `num_vertices` and each element represents a vertex of\n the simplex.\n objective_at_simplex: A `Tensor` of same dtype as the dtype of the\n return value of objective_function. The shape is a vector of size\n `num_vertices`. The objective function evaluated at the simplex.\n num_evaluations: An `int32` scalar `Tensor`. The number of points on\n which the objective function was evaluated.\n\n Raises:\n ValueError: If any of the following conditions hold\n 1. If none or more than one of `initial_simplex` and `initial_vertex` are\n supplied.\n 2. If `initial_simplex` and `step_sizes` are both specified.\n \"\"\"\n if objective_at_initial_simplex is not None and initial_simplex is None:\n raise ValueError('`objective_at_initial_simplex` specified but the'\n '`initial_simplex` was not.')\n\n if objective_at_initial_vertex is not None and initial_vertex is None:\n raise ValueError('`objective_at_initial_vertex` specified but the'\n '`initial_vertex` was not.')\n\n # The full simplex was specified.\n if initial_simplex is not None:\n if initial_vertex is not None:\n raise ValueError('Both `initial_simplex` and `initial_vertex` specified.'\n ' Only one of the two should be specified.')\n\n if step_sizes is not None:\n raise ValueError('`step_sizes` must not be specified when an'\n ' `initial_simplex` has been specified.')\n return _prepare_args_with_initial_simplex(objective_function,\n initial_simplex,\n objective_at_initial_simplex,\n batch_evaluate_objective)\n\n if initial_vertex is None:\n raise ValueError('One of `initial_simplex` or `initial_vertex`'\n ' must be supplied')\n\n if step_sizes is None:\n step_sizes = _default_step_sizes(initial_vertex)\n\n return _prepare_args_with_initial_vertex(objective_function,\n initial_vertex,\n step_sizes,\n objective_at_initial_vertex,\n batch_evaluate_objective)", "response": "Prepare the arguments for the objective function at the simplex."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef _default_step_sizes(reference_vertex):\n # Step size to choose when the coordinate is zero.\n small_sizes = tf.ones_like(reference_vertex) * 0.00025\n # Step size to choose when the coordinate is non-zero.\n large_sizes = reference_vertex * 0.05\n return tf.where(tf.abs(reference_vertex) < _EPSILON,\n small_sizes,\n large_sizes)", "response": "Chooses default step sizes according to [ Gao and Han 2000][ 3 ]."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\npreparing the args for the objective function at the specified initial simplex.", "response": "def _prepare_args_with_initial_simplex(objective_function,\n initial_simplex,\n objective_at_initial_simplex,\n batch_evaluate_objective):\n \"\"\"Evaluates the objective function at the specified initial simplex.\"\"\"\n initial_simplex = tf.convert_to_tensor(value=initial_simplex)\n\n # If d is the dimension of the problem, the number of vertices in the\n # simplex should be d+1. From this, we can infer the number of dimensions\n # as n - 1 where n is the number of vertices specified.\n num_vertices = tf.shape(input=initial_simplex)[0]\n dim = num_vertices - 1\n num_evaluations = 0\n\n if objective_at_initial_simplex is None:\n objective_at_initial_simplex, n_evals = _evaluate_objective_multiple(\n objective_function, initial_simplex, batch_evaluate_objective)\n num_evaluations += n_evals\n objective_at_initial_simplex = tf.convert_to_tensor(\n value=objective_at_initial_simplex)\n return (dim,\n num_vertices,\n initial_simplex,\n objective_at_initial_simplex,\n num_evaluations)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _prepare_args_with_initial_vertex(objective_function,\n initial_vertex,\n step_sizes,\n objective_at_initial_vertex,\n batch_evaluate_objective):\n \"\"\"Constructs a standard axes aligned simplex.\"\"\"\n dim = tf.size(input=initial_vertex)\n num_vertices = dim + 1\n unit_vectors_along_axes = tf.reshape(\n tf.eye(dim, dim, dtype=initial_vertex.dtype.base_dtype),\n tf.concat([[dim], tf.shape(input=initial_vertex)], axis=0))\n\n # If step_sizes does not broadcast to initial_vertex, the multiplication\n # in the second term will fail.\n simplex_face = initial_vertex + step_sizes * unit_vectors_along_axes\n simplex = tf.concat([tf.expand_dims(initial_vertex, axis=0),\n simplex_face], axis=0)\n num_evaluations = 0\n # Evaluate the objective function at the simplex vertices.\n if objective_at_initial_vertex is None:\n objective_at_initial_vertex = objective_function(initial_vertex)\n num_evaluations += 1\n\n objective_at_simplex_face, num_evals = _evaluate_objective_multiple(\n objective_function, simplex_face, batch_evaluate_objective)\n num_evaluations += num_evals\n\n objective_at_simplex = tf.concat(\n [\n tf.expand_dims(objective_at_initial_vertex, axis=0),\n objective_at_simplex_face\n ], axis=0)\n\n return (dim,\n num_vertices,\n simplex,\n objective_at_simplex,\n num_evaluations)", "response": "Constructs a standard axes aligned simplex."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\napply the Gao and Han presciption to the unspecified parameters.", "response": "def _resolve_parameters(dim,\n reflection,\n expansion,\n contraction,\n shrinkage,\n dtype):\n \"\"\"Applies the [Gao and Han][3] presciption to the unspecified parameters.\"\"\"\n dim = tf.cast(dim, dtype=dtype)\n reflection = 1. if reflection is None else reflection\n expansion = (1. + 2. / dim) if expansion is None else expansion\n contraction = (0.75 - 1. / (2 * dim)) if contraction is None else contraction\n shrinkage = (1. - 1. / dim) if shrinkage is None else shrinkage\n return reflection, expansion, contraction, shrinkage"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _evaluate_objective_multiple(objective_function, arg_batch,\n batch_evaluate_objective):\n \"\"\"Evaluates the objective function on a batch of points.\n\n If `batch_evaluate_objective` is True, returns\n `objective function(arg_batch)` else it maps the `objective_function`\n across the `arg_batch`.\n\n Args:\n objective_function: A Python callable that accepts a single `Tensor` of\n rank 'R > 1' and any shape 's' and returns a scalar `Tensor` of real dtype\n containing the value of the function at that point. If\n `batch a `Tensor` of shape `[batch_size] + s ` where `batch_size` is the\n size of the batch of args. In this case, the expected return value is a\n `Tensor` of shape `[batch_size]`.\n arg_batch: A `Tensor` of real dtype. The batch of arguments at which to\n evaluate the `objective_function`. If `batch_evaluate_objective` is False,\n `arg_batch` will be unpacked along the zeroth axis and the\n `objective_function` will be applied to each element.\n batch_evaluate_objective: `bool`. Whether the `objective_function` can\n evaluate a batch of arguments at once.\n\n Returns:\n A tuple containing:\n objective_values: A `Tensor` of real dtype and shape `[batch_size]`.\n The value of the objective function evaluated at the supplied\n `arg_batch`.\n num_evaluations: An `int32` scalar `Tensor`containing the number of\n points on which the objective function was evaluated (i.e `batch_size`).\n \"\"\"\n n_points = tf.shape(input=arg_batch)[0]\n if batch_evaluate_objective:\n return objective_function(arg_batch), n_points\n return tf.map_fn(objective_function, arg_batch), n_points", "response": "Evaluates the objective function on a batch of points."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nsaving a PNG plot with histograms of weight means and stddevs.", "response": "def plot_weight_posteriors(names, qm_vals, qs_vals, fname):\n \"\"\"Save a PNG plot with histograms of weight means and stddevs.\n\n Args:\n names: A Python `iterable` of `str` variable names.\n qm_vals: A Python `iterable`, the same length as `names`,\n whose elements are Numpy `array`s, of any shape, containing\n posterior means of weight varibles.\n qs_vals: A Python `iterable`, the same length as `names`,\n whose elements are Numpy `array`s, of any shape, containing\n posterior standard deviations of weight varibles.\n fname: Python `str` filename to save the plot to.\n \"\"\"\n fig = figure.Figure(figsize=(6, 3))\n canvas = backend_agg.FigureCanvasAgg(fig)\n\n ax = fig.add_subplot(1, 2, 1)\n for n, qm in zip(names, qm_vals):\n sns.distplot(qm.flatten(), ax=ax, label=n)\n ax.set_title(\"weight means\")\n ax.set_xlim([-1.5, 1.5])\n ax.legend()\n\n ax = fig.add_subplot(1, 2, 2)\n for n, qs in zip(names, qs_vals):\n sns.distplot(qs.flatten(), ax=ax)\n ax.set_title(\"weight stddevs\")\n ax.set_xlim([0, 1.])\n\n fig.tight_layout()\n canvas.print_figure(fname, format=\"png\")\n print(\"saved {}\".format(fname))"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nsaving a PNG plot visualizing posterior uncertainty on heldout data.", "response": "def plot_heldout_prediction(input_vals, probs,\n fname, n=10, title=\"\"):\n \"\"\"Save a PNG plot visualizing posterior uncertainty on heldout data.\n\n Args:\n input_vals: A `float`-like Numpy `array` of shape\n `[num_heldout] + IMAGE_SHAPE`, containing heldout input images.\n probs: A `float`-like Numpy array of shape `[num_monte_carlo,\n num_heldout, num_classes]` containing Monte Carlo samples of\n class probabilities for each heldout sample.\n fname: Python `str` filename to save the plot to.\n n: Python `int` number of datapoints to vizualize.\n title: Python `str` title for the plot.\n \"\"\"\n fig = figure.Figure(figsize=(9, 3*n))\n canvas = backend_agg.FigureCanvasAgg(fig)\n for i in range(n):\n ax = fig.add_subplot(n, 3, 3*i + 1)\n ax.imshow(input_vals[i, :].reshape(IMAGE_SHAPE[:-1]), interpolation=\"None\")\n\n ax = fig.add_subplot(n, 3, 3*i + 2)\n for prob_sample in probs:\n sns.barplot(np.arange(10), prob_sample[i, :], alpha=0.1, ax=ax)\n ax.set_ylim([0, 1])\n ax.set_title(\"posterior samples\")\n\n ax = fig.add_subplot(n, 3, 3*i + 3)\n sns.barplot(np.arange(10), np.mean(probs[:, i, :], axis=0), ax=ax)\n ax.set_ylim([0, 1])\n ax.set_title(\"predictive probs\")\n fig.suptitle(title)\n fig.tight_layout()\n\n canvas.print_figure(fname, format=\"png\")\n print(\"saved {}\".format(fname))"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nbuilds an iterator that can switch between train and heldout data.", "response": "def build_input_pipeline(mnist_data, batch_size, heldout_size):\n \"\"\"Build an Iterator switching between train and heldout data.\"\"\"\n\n # Build an iterator over training batches.\n training_dataset = tf.data.Dataset.from_tensor_slices(\n (mnist_data.train.images, np.int32(mnist_data.train.labels)))\n training_batches = training_dataset.shuffle(\n 50000, reshuffle_each_iteration=True).repeat().batch(batch_size)\n training_iterator = tf.compat.v1.data.make_one_shot_iterator(training_batches)\n\n # Build a iterator over the heldout set with batch_size=heldout_size,\n # i.e., return the entire heldout set as a constant.\n heldout_dataset = tf.data.Dataset.from_tensor_slices(\n (mnist_data.validation.images,\n np.int32(mnist_data.validation.labels)))\n heldout_frozen = (heldout_dataset.take(heldout_size).\n repeat().batch(heldout_size))\n heldout_iterator = tf.compat.v1.data.make_one_shot_iterator(heldout_frozen)\n\n # Combine these into a feedable iterator that can switch between training\n # and validation inputs.\n handle = tf.compat.v1.placeholder(tf.string, shape=[])\n feedable_iterator = tf.compat.v1.data.Iterator.from_string_handle(\n handle, training_batches.output_types, training_batches.output_shapes)\n images, labels = feedable_iterator.get_next()\n\n return images, labels, handle, training_iterator, heldout_iterator"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nbuilds fake MNIST - style data for unit testing.", "response": "def build_fake_data(num_examples=10):\n \"\"\"Build fake MNIST-style data for unit testing.\"\"\"\n\n class Dummy(object):\n pass\n\n num_examples = 10\n mnist_data = Dummy()\n mnist_data.train = Dummy()\n mnist_data.train.images = np.float32(np.random.randn(\n num_examples, *IMAGE_SHAPE))\n mnist_data.train.labels = np.int32(np.random.permutation(\n np.arange(num_examples)))\n mnist_data.train.num_examples = num_examples\n mnist_data.validation = Dummy()\n mnist_data.validation.images = np.float32(np.random.randn(\n num_examples, *IMAGE_SHAPE))\n mnist_data.validation.labels = np.int32(np.random.permutation(\n np.arange(num_examples)))\n mnist_data.validation.num_examples = num_examples\n return mnist_data"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _kl_bernoulli_bernoulli(a, b, name=None):\n with tf.name_scope(name or \"kl_bernoulli_bernoulli\"):\n delta_probs0 = tf.nn.softplus(-b.logits) - tf.nn.softplus(-a.logits)\n delta_probs1 = tf.nn.softplus(b.logits) - tf.nn.softplus(a.logits)\n return (tf.sigmoid(a.logits) * delta_probs0\n + tf.sigmoid(-a.logits) * delta_probs1)", "response": "Calculate the batched KL divergence KL with a and b Bernoulli."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nreturns the configuration as a JSON - serializable dict.", "response": "def get_config(self):\n \"\"\"Returns initializer configuration as a JSON-serializable dict.\"\"\"\n return {\n 'initializers': [\n tf.compat.v2.initializers.serialize(\n tf.keras.initializers.get(init))\n for init in self.initializers\n ],\n 'sizes': self.sizes,\n 'validate_args': self.validate_args,\n }"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef from_config(cls, config):\n return cls(**{\n 'initializers': [tf.compat.v2.initializers.deserialize(init)\n for init in config.get('initializers', [])],\n 'sizes': config.get('sizes', []),\n 'validate_args': config.get('validate_args', False),\n })", "response": "Instantiates an initializer from a configuration dictionary."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nwraps for scipy. sparse. matrix. matmul.", "response": "def _matmul(a, b,\n transpose_a=False, transpose_b=False,\n adjoint_a=False, adjoint_b=False,\n a_is_sparse=False, b_is_sparse=False,\n name=None): # pylint: disable=unused-argument\n \"\"\"Numpy matmul wrapper.\"\"\"\n if a_is_sparse or b_is_sparse:\n raise NotImplementedError('Numpy backend does not support sparse matmul.')\n if transpose_a or adjoint_a:\n a = _matrix_transpose(a, conjugate=adjoint_a)\n if transpose_b or adjoint_b:\n b = _matrix_transpose(b, conjugate=adjoint_b)\n return np.matmul(a, b)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef assign_moving_mean_variance(\n mean_var, variance_var, value, decay, name=None):\n \"\"\"Compute exponentially weighted moving {mean,variance} of a streaming value.\n\n The `value` updated exponentially weighted moving `mean_var` and\n `variance_var` are given by the following recurrence relations:\n\n ```python\n variance_var = decay * (variance_var + (1 - decay) * (value - mean_var)**2)\n mean_var = decay * mean_var + (1 - decay) * value\n ```\n\n Note: `mean_var` is updated *after* `variance_var`, i.e., `variance_var` uses\n the lag-1 mean.\n\n For derivation justification, see [Finch (2009; Eq. 143)][1].\n Parameterization: Finch's `alpha` is `1 - decay`.\n\n Args:\n mean_var: `float`-like `Variable` representing the exponentially weighted\n moving mean. Same shape as `variance_var` and `value`.\n variance_var: `float`-like `Variable` representing the\n exponentially weighted moving variance. Same shape as `mean_var` and\n `value`.\n value: `float`-like `Tensor`. Same shape as `mean_var` and `variance_var`.\n decay: A `float`-like `Tensor`. The moving mean decay. Typically close to\n `1.`, e.g., `0.999`.\n name: Optional name of the returned operation.\n\n Returns:\n mean_var: `Variable` representing the `value`-updated exponentially weighted\n moving mean.\n variance_var: `Variable` representing the `value`-updated\n exponentially weighted moving variance.\n\n Raises:\n TypeError: if `mean_var` does not have float type `dtype`.\n TypeError: if `mean_var`, `variance_var`, `value`, `decay` have different\n `base_dtype`.\n\n #### References\n\n [1]: Tony Finch. Incremental calculation of weighted mean and variance.\n _Technical Report_, 2009.\n http://people.ds.cam.ac.uk/fanf2/hermes/doc/antiforgery/stats.pdf\n \"\"\"\n with tf.compat.v1.name_scope(name, \"assign_moving_mean_variance\",\n [variance_var, mean_var, value, decay]):\n with tf.compat.v1.colocate_with(variance_var):\n with tf.compat.v1.colocate_with(mean_var):\n base_dtype = mean_var.dtype.base_dtype\n if not base_dtype.is_floating:\n raise TypeError(\n \"mean_var.base_dtype({}) does not have float type \"\n \"`dtype`.\".format(base_dtype.name))\n if base_dtype != variance_var.dtype.base_dtype:\n raise TypeError(\n \"mean_var.base_dtype({}) != variance_var.base_dtype({})\".format(\n base_dtype.name,\n variance_var.dtype.base_dtype.name))\n value = tf.convert_to_tensor(\n value=value, dtype=base_dtype, name=\"value\")\n decay = tf.convert_to_tensor(\n value=decay, dtype=base_dtype, name=\"decay\")\n delta = value - mean_var\n with tf.control_dependencies([delta]):\n # We want mean_{t+1} = decay * mean_t + (1. - decay) * value\n # We compute mean += decay * mean_t - mean_t + (1. - decay) * value =\n # = (1. - decay) * (value - mean_t)\n mean_var = mean_var.assign_add((1. - decay) * delta)\n # We want variance_{t+1} = decay * (variance_t +\n # + (1 - decay) * (value - mean_var)**2).\n # We compute variance -= variance_t - decay * (variance_t +\n # + (1 - decay) * (value - mean_var)**2) =\n # = (1 - decay) * variance_t\n # - decay * (1 - decay) * (value - mean_var)**2\n # = (1 - decay) * (variance_t - decay * (value - mean_var)**2).\n variance_var = variance_var.assign_sub(\n (1. - decay) * (variance_var - decay * tf.square(delta)))\n return mean_var, variance_var", "response": "Assign a new value to the exponentially weighted moving mean and variance."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef assign_log_moving_mean_exp(\n log_mean_exp_var, log_value, decay, name=None):\n \"\"\"Compute the log of the exponentially weighted moving mean of the exp.\n\n If `log_value` is a draw from a stationary random variable, this function\n approximates `log(E[exp(log_value)])`, i.e., a weighted log-sum-exp. More\n precisely, a `tf.Variable`, `log_mean_exp_var`, is updated by `log_value`\n using the following identity:\n\n ```none\n log_mean_exp_var =\n = log(decay exp(log_mean_exp_var) + (1 - decay) exp(log_value))\n = log(exp(log_mean_exp_var + log(decay)) + exp(log_value + log1p(-decay)))\n = log_mean_exp_var\n + log( exp(log_mean_exp_var - log_mean_exp_var + log(decay))\n + exp(log_value - log_mean_exp_var + log1p(-decay)))\n = log_mean_exp_var\n + log_sum_exp([log(decay), log_value - log_mean_exp_var + log1p(-decay)]).\n ```\n\n In addition to numerical stability, this formulation is advantageous because\n `log_mean_exp_var` can be updated in a lock-free manner, i.e., using\n `assign_add`. (Note: the updates are not thread-safe; it's just that the\n update to the tf.Variable is presumed efficient due to being lock-free.)\n\n Args:\n log_mean_exp_var: `float`-like `Variable` representing the log of the\n exponentially weighted moving mean of the exp. Same shape as `log_value`.\n log_value: `float`-like `Tensor` representing a new (streaming) observation.\n Same shape as `log_mean_exp_var`.\n decay: A `float`-like `Tensor`. The moving mean decay. Typically close to\n `1.`, e.g., `0.999`.\n name: Optional name of the returned operation.\n\n Returns:\n log_mean_exp_var: A reference to the input 'Variable' tensor with the\n `log_value`-updated log of the exponentially weighted moving mean of exp.\n\n Raises:\n TypeError: if `log_mean_exp_var` does not have float type `dtype`.\n TypeError: if `log_mean_exp_var`, `log_value`, `decay` have different\n `base_dtype`.\n \"\"\"\n with tf.compat.v1.name_scope(name, \"assign_log_moving_mean_exp\",\n [log_mean_exp_var, log_value, decay]):\n # We want to update the variable in a numerically stable and lock-free way.\n # To do this, observe that variable `x` updated by `v` is:\n # x = log(w exp(x) + (1-w) exp(v))\n # = log(exp(x + log(w)) + exp(v + log1p(-w)))\n # = x + log(exp(x - x + log(w)) + exp(v - x + log1p(-w)))\n # = x + lse([log(w), v - x + log1p(-w)])\n with tf.compat.v1.colocate_with(log_mean_exp_var):\n base_dtype = log_mean_exp_var.dtype.base_dtype\n if not base_dtype.is_floating:\n raise TypeError(\n \"log_mean_exp_var.base_dtype({}) does not have float type \"\n \"`dtype`.\".format(base_dtype.name))\n log_value = tf.convert_to_tensor(\n value=log_value, dtype=base_dtype, name=\"log_value\")\n decay = tf.convert_to_tensor(value=decay, dtype=base_dtype, name=\"decay\")\n delta = (log_value - log_mean_exp_var)[tf.newaxis, ...]\n x = tf.concat([\n tf.math.log(decay) * tf.ones_like(delta),\n delta + tf.math.log1p(-decay)\n ],\n axis=0)\n x = tf.reduce_logsumexp(input_tensor=x, axis=0)\n return log_mean_exp_var.assign_add(x)", "response": "Assign a log of the exponentially weighted moving mean of the exp."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\ncomputes exponentially weighted moving mean and variance.", "response": "def moving_mean_variance(value, decay, name=None):\n \"\"\"Compute exponentially weighted moving {mean,variance} of a streaming value.\n\n The exponentially-weighting moving `mean_var` and `variance_var` are updated\n by `value` according to the following recurrence:\n\n ```python\n variance_var = decay * (variance_var + (1-decay) * (value - mean_var)**2)\n mean_var = decay * mean_var + (1 - decay) * value\n ```\n\n Note: `mean_var` is updated *after* `variance_var`, i.e., `variance_var` uses\n the lag-`1` mean.\n\n For derivation justification, see [Finch (2009; Eq. 143)][1].\n\n Unlike `assign_moving_mean_variance`, this function handles\n variable creation.\n\n Args:\n value: `float`-like `Tensor`. Same shape as `mean_var` and `variance_var`.\n decay: A `float`-like `Tensor`. The moving mean decay. Typically close to\n `1.`, e.g., `0.999`.\n name: Optional name of the returned operation.\n\n Returns:\n mean_var: `Variable` representing the `value`-updated exponentially weighted\n moving mean.\n variance_var: `Variable` representing the `value`-updated\n exponentially weighted moving variance.\n\n Raises:\n TypeError: if `value_var` does not have float type `dtype`.\n TypeError: if `value`, `decay` have different `base_dtype`.\n\n #### References\n\n [1]: Tony Finch. Incremental calculation of weighted mean and variance.\n _Technical Report_, 2009.\n http://people.ds.cam.ac.uk/fanf2/hermes/doc/antiforgery/stats.pdf\n \"\"\"\n with tf.compat.v1.variable_scope(name, \"moving_mean_variance\",\n [value, decay]):\n value = tf.convert_to_tensor(value=value, name=\"value\")\n base_dtype = value.dtype.base_dtype\n if not base_dtype.is_floating:\n raise TypeError(\n \"value.base_dtype({}) does not have float type `dtype`.\".format(\n base_dtype.name))\n decay = tf.convert_to_tensor(value=decay, dtype=base_dtype, name=\"decay\")\n variance_var = tf.compat.v2.Variable(\n name=\"moving_variance\",\n initial_value=tf.zeros(shape=value.shape, dtype=value.dtype),\n trainable=False)\n mean_var = tf.compat.v2.Variable(\n name=\"moving_mean\",\n initial_value=tf.zeros(shape=value.shape, dtype=value.dtype),\n trainable=False)\n return assign_moving_mean_variance(\n mean_var, variance_var, value, decay)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nensuring non - scalar input has at least one column.", "response": "def _make_columnar(self, x):\n \"\"\"Ensures non-scalar input has at least one column.\n\n Example:\n If `x = [1, 2, 3]` then the output is `[[1], [2], [3]]`.\n\n If `x = [[1, 2, 3], [4, 5, 6]]` then the output is unchanged.\n\n If `x = 1` then the output is unchanged.\n\n Args:\n x: `Tensor`.\n\n Returns:\n columnar_x: `Tensor` with at least two dimensions.\n \"\"\"\n if tensorshape_util.rank(x.shape) is not None:\n if tensorshape_util.rank(x.shape) == 1:\n x = x[tf.newaxis, :]\n return x\n shape = tf.shape(input=x)\n maybe_expanded_shape = tf.concat([\n shape[:-1],\n distribution_util.pick_vector(\n tf.equal(tf.rank(x), 1), [1], np.array([], dtype=np.int32)),\n shape[-1:],\n ], 0)\n return tf.reshape(x, maybe_expanded_shape)"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\ncalculates the batched KL divergence KL with a and b Laplace.", "response": "def _kl_laplace_laplace(a, b, name=None):\n \"\"\"Calculate the batched KL divergence KL(a || b) with a and b Laplace.\n\n Args:\n a: instance of a Laplace distribution object.\n b: instance of a Laplace distribution object.\n name: (optional) Name to use for created operations.\n default is \"kl_laplace_laplace\".\n\n Returns:\n Batchwise KL(a || b)\n \"\"\"\n with tf.name_scope(name or \"kl_laplace_laplace\"):\n # Consistent with\n # http://www.mast.queensu.ca/~communications/Papers/gil-msc11.pdf, page 38\n distance = tf.abs(a.loc - b.loc)\n ratio = a.scale / b.scale\n\n return (-tf.math.log(ratio) - 1 + distance / b.scale +\n ratio * tf.exp(-distance / a.scale))"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef random_rademacher(shape, dtype=tf.float32, seed=None, name=None):\n with tf.compat.v1.name_scope(name, 'random_rademacher', [shape, seed]):\n # Choose the dtype to cause `2 * random_bernoulli - 1` to run in the same\n # memory (host or device) as the downstream cast will want to put it. The\n # convention on GPU is that int32 are in host memory and int64 are in device\n # memory.\n generation_dtype = tf.int64 if tf.as_dtype(dtype) != tf.int32 else tf.int32\n random_bernoulli = tf.random.uniform(\n shape, minval=0, maxval=2, dtype=generation_dtype, seed=seed)\n return tf.cast(2 * random_bernoulli - 1, dtype)", "response": "Generates a random tensor from the specified shape and dtype."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\ngenerating a Tensor of random positive reals drawn from a Rayleigh distribution with specified shape and dtype.", "response": "def random_rayleigh(shape, scale=None, dtype=tf.float32, seed=None, name=None):\n \"\"\"Generates `Tensor` of positive reals drawn from a Rayleigh distributions.\n\n The probability density function of a Rayleigh distribution with `scale`\n parameter is given by:\n\n ```none\n f(x) = x scale**-2 exp(-x**2 0.5 scale**-2)\n ```\n\n For more details, see [Rayleigh distribution](\n https://en.wikipedia.org/wiki/Rayleigh_distribution)\n\n Args:\n shape: Vector-shaped, `int` `Tensor` representing shape of output.\n scale: (Optional) Positive `float` `Tensor` representing `Rayleigh` scale.\n Default value: `None` (i.e., `scale = 1.`).\n dtype: (Optional) TF `dtype` representing `dtype` of output.\n Default value: `tf.float32`.\n seed: (Optional) Python integer to seed the random number generator.\n Default value: `None` (i.e., no seed).\n name: Python `str` name prefixed to Ops created by this function.\n Default value: `None` (i.e., 'random_rayleigh').\n\n Returns:\n rayleigh: `Tensor` with specified `shape` and `dtype` consisting of positive\n real values drawn from a Rayleigh distribution with specified `scale`.\n \"\"\"\n with tf.compat.v1.name_scope(name, 'random_rayleigh', [shape, scale, seed]):\n if scale is not None:\n # Its important to expand the shape to match scale's, otherwise we won't\n # have independent draws.\n scale = tf.convert_to_tensor(value=scale, dtype=dtype, name='scale')\n shape = tf.broadcast_dynamic_shape(shape, tf.shape(input=scale))\n x = tf.sqrt(-2. * tf.math.log(\n tf.random.uniform(shape, minval=0, maxval=1, dtype=dtype, seed=seed)))\n if scale is None:\n return x\n return x * scale"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nfinishing computation of log_prob on one element of the inverse image.", "response": "def _finish_log_prob_for_one_fiber(self, y, x, ildj, event_ndims,\n **distribution_kwargs):\n \"\"\"Finish computation of log_prob on one element of the inverse image.\"\"\"\n x = self._maybe_rotate_dims(x, rotate_right=True)\n log_prob = self.distribution.log_prob(x, **distribution_kwargs)\n if self._is_maybe_event_override:\n log_prob = tf.reduce_sum(\n input_tensor=log_prob, axis=self._reduce_event_indices)\n log_prob += tf.cast(ildj, log_prob.dtype)\n if self._is_maybe_event_override and isinstance(event_ndims, int):\n tensorshape_util.set_shape(\n log_prob,\n tf.broadcast_static_shape(\n tensorshape_util.with_rank_at_least(y.shape, 1)[:-event_ndims],\n self.batch_shape))\n return log_prob"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nfinish computation of prob on one element of the inverse image.", "response": "def _finish_prob_for_one_fiber(self, y, x, ildj, event_ndims,\n **distribution_kwargs):\n \"\"\"Finish computation of prob on one element of the inverse image.\"\"\"\n x = self._maybe_rotate_dims(x, rotate_right=True)\n prob = self.distribution.prob(x, **distribution_kwargs)\n if self._is_maybe_event_override:\n prob = tf.reduce_prod(input_tensor=prob, axis=self._reduce_event_indices)\n prob *= tf.exp(tf.cast(ildj, prob.dtype))\n if self._is_maybe_event_override and isinstance(event_ndims, int):\n tensorshape_util.set_shape(\n prob,\n tf.broadcast_static_shape(\n tensorshape_util.with_rank_at_least(y.shape, 1)[:-event_ndims],\n self.batch_shape))\n return prob"} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef _maybe_validate_shape_override(self, override_shape, base_is_scalar,\n validate_args, name):\n \"\"\"Helper to __init__ which ensures override batch/event_shape are valid.\"\"\"\n if override_shape is None:\n override_shape = []\n\n override_shape = tf.convert_to_tensor(\n value=override_shape, dtype=tf.int32, name=name)\n\n if not dtype_util.is_integer(override_shape.dtype):\n raise TypeError(\"shape override must be an integer\")\n\n override_is_scalar = _is_scalar_from_shape_tensor(override_shape)\n if tf.get_static_value(override_is_scalar):\n return self._empty\n\n dynamic_assertions = []\n\n if tensorshape_util.rank(override_shape.shape) is not None:\n if tensorshape_util.rank(override_shape.shape) != 1:\n raise ValueError(\"shape override must be a vector\")\n elif validate_args:\n dynamic_assertions += [\n assert_util.assert_rank(\n override_shape, 1, message=\"shape override must be a vector\")\n ]\n\n if tf.get_static_value(override_shape) is not None:\n if any(s < 0 for s in tf.get_static_value(override_shape)):\n raise ValueError(\"shape override must have non-negative elements\")\n elif validate_args:\n dynamic_assertions += [\n assert_util.assert_non_negative(\n override_shape,\n message=\"shape override must have non-negative elements\")\n ]\n\n is_both_nonscalar = prefer_static.logical_and(\n prefer_static.logical_not(base_is_scalar),\n prefer_static.logical_not(override_is_scalar))\n if tf.get_static_value(is_both_nonscalar) is not None:\n if tf.get_static_value(is_both_nonscalar):\n raise ValueError(\"base distribution not scalar\")\n elif validate_args:\n dynamic_assertions += [\n assert_util.assert_equal(\n is_both_nonscalar, False, message=\"base distribution not scalar\")\n ]\n\n if not dynamic_assertions:\n return override_shape\n return distribution_util.with_dependencies(\n dynamic_assertions, override_shape)", "response": "Helper to __init__ which ensures override batch or event_shape are valid."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _undo_batch_normalization(x,\n mean,\n variance,\n offset,\n scale,\n variance_epsilon,\n name=None):\n r\"\"\"Inverse of tf.nn.batch_normalization.\n\n Args:\n x: Input `Tensor` of arbitrary dimensionality.\n mean: A mean `Tensor`.\n variance: A variance `Tensor`.\n offset: An offset `Tensor`, often denoted `beta` in equations, or\n None. If present, will be added to the normalized tensor.\n scale: A scale `Tensor`, often denoted `gamma` in equations, or\n `None`. If present, the scale is applied to the normalized tensor.\n variance_epsilon: A small `float` added to the minibatch `variance` to\n prevent dividing by zero.\n name: A name for this operation (optional).\n\n Returns:\n batch_unnormalized: The de-normalized, de-scaled, de-offset `Tensor`.\n \"\"\"\n with tf.compat.v2.name_scope(name or \"undo_batchnorm\"):\n # inv = tf.rsqrt(variance + variance_epsilon)\n # if scale is not None:\n # inv *= scale\n # return x * inv + (\n # offset - mean * inv if offset is not None else -mean * inv)\n rescale = tf.sqrt(variance + variance_epsilon)\n if scale is not None:\n rescale /= scale\n batch_unnormalized = x * rescale + (\n mean - offset * rescale if offset is not None else mean)\n return batch_unnormalized", "response": "r Inverse of tf. nn. batch_normalization."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef _validate_bn_layer(self, layer):\n if (not isinstance(layer, tf.keras.layers.BatchNormalization) and\n not isinstance(layer, tf.compat.v1.layers.BatchNormalization)):\n raise ValueError(\n \"batchnorm_layer must be an instance of BatchNormalization layer.\")\n if layer.renorm:\n raise ValueError(\"BatchNorm Bijector does not support renormalization.\")\n if layer.virtual_batch_size:\n raise ValueError(\n \"BatchNorm Bijector does not support virtual batch sizes.\")", "response": "Check for valid BatchNormalization layer."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nslice a single parameter of a distribution.", "response": "def _slice_single_param(param, param_event_ndims, slices, dist_batch_shape):\n \"\"\"Slices a single parameter of a distribution.\n\n Args:\n param: A `Tensor`, the original parameter to slice.\n param_event_ndims: `int` event parameterization rank for this parameter.\n slices: A `tuple` of normalized slices.\n dist_batch_shape: The distribution's batch shape `Tensor`.\n\n Returns:\n new_param: A `Tensor`, batch-sliced according to slices.\n \"\"\"\n # Extend param shape with ones on the left to match dist_batch_shape.\n param_shape = tf.shape(input=param)\n insert_ones = tf.ones(\n [tf.size(input=dist_batch_shape) + param_event_ndims - tf.rank(param)],\n dtype=param_shape.dtype)\n new_param_shape = tf.concat([insert_ones, param_shape], axis=0)\n full_batch_param = tf.reshape(param, new_param_shape)\n param_slices = []\n # We separately track the batch axis from the parameter axis because we want\n # them to align for positive indexing, and be offset by param_event_ndims for\n # negative indexing.\n param_dim_idx = 0\n batch_dim_idx = 0\n for slc in slices:\n if slc is tf.newaxis:\n param_slices.append(slc)\n continue\n if slc is Ellipsis:\n if batch_dim_idx < 0:\n raise ValueError('Found multiple `...` in slices {}'.format(slices))\n param_slices.append(slc)\n # Switch over to negative indexing for the broadcast check.\n num_remaining_non_newaxis_slices = sum(\n [s is not tf.newaxis for s in slices[slices.index(Ellipsis) + 1:]])\n batch_dim_idx = -num_remaining_non_newaxis_slices\n param_dim_idx = batch_dim_idx - param_event_ndims\n continue\n # Find the batch dimension sizes for both parameter and distribution.\n param_dim_size = new_param_shape[param_dim_idx]\n batch_dim_size = dist_batch_shape[batch_dim_idx]\n is_broadcast = batch_dim_size > param_dim_size\n # Slices are denoted by start:stop:step.\n if isinstance(slc, slice):\n start, stop, step = slc.start, slc.stop, slc.step\n if start is not None:\n start = tf.where(is_broadcast, 0, start)\n if stop is not None:\n stop = tf.where(is_broadcast, 1, stop)\n if step is not None:\n step = tf.where(is_broadcast, 1, step)\n param_slices.append(slice(start, stop, step))\n else: # int, or int Tensor, e.g. d[d.batch_shape_tensor()[0] // 2]\n param_slices.append(tf.where(is_broadcast, 0, slc))\n param_dim_idx += 1\n batch_dim_idx += 1\n param_slices.extend([ALL_SLICE] * param_event_ndims)\n return full_batch_param.__getitem__(param_slices)"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _slice_params_to_dict(dist, params_event_ndims, slices):\n override_dict = {}\n for param_name, param_event_ndims in six.iteritems(params_event_ndims):\n # Verify that either None or a legit value is in the parameters dict.\n if param_name not in dist.parameters:\n raise ValueError('Distribution {} is missing advertised '\n 'parameter {}'.format(dist, param_name))\n param = dist.parameters[param_name]\n if param is None:\n # some distributions have multiple possible parameterizations; this\n # param was not provided\n continue\n dtype = None\n if hasattr(dist, param_name):\n attr = getattr(dist, param_name)\n dtype = getattr(attr, 'dtype', None)\n if dtype is None:\n dtype = dist.dtype\n warnings.warn('Unable to find property getter for parameter Tensor {} '\n 'on {}, falling back to Distribution.dtype {}'.format(\n param_name, dist, dtype))\n param = tf.convert_to_tensor(value=param, dtype=dtype)\n override_dict[param_name] = _slice_single_param(param, param_event_ndims,\n slices,\n dist.batch_shape_tensor())\n return override_dict", "response": "Slice the parameters of a given distribution into a dictionary of batch - sliced parameters."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef _apply_single_step(dist, params_event_ndims, slices, params_overrides):\n if len(slices) == 1 and slices[0] == Ellipsis:\n # The path used by Distribution.copy: batch_slice(...args..., Ellipsis)\n override_dict = {}\n else:\n override_dict = _slice_params_to_dict(dist, params_event_ndims, slices)\n override_dict.update(params_overrides)\n parameters = dict(dist.parameters, **override_dict)\n new_dist = type(dist)(**parameters)\n return new_dist", "response": "Applies a single slicing step to dist returning a new instance."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _apply_slice_sequence(dist, params_event_ndims, slice_overrides_seq):\n for slices, overrides in slice_overrides_seq:\n dist = _apply_single_step(dist, params_event_ndims, slices, overrides)\n return dist", "response": "Applies a sequence of slice or copy - with - overrides operations to dist."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nslicing dist along its batch dimensions.", "response": "def batch_slice(dist, params_event_ndims, params_overrides, slices):\n \"\"\"Slices `dist` along its batch dimensions. Helper for tfd.Distribution.\n\n Args:\n dist: A `tfd.Distribution` instance.\n params_event_ndims: A `dict` of `str->int` indicating the number of\n dimensions of a given parameter required to parameterize a single event.\n params_overrides: A `dict` of parameter overrides. (e.g. from\n `Distribution.copy`).\n slices: A `slice` or `int` or `int` `Tensor` or `tf.newaxis` or `tuple`\n thereof. (e.g. the argument of a `__getitem__` method).\n\n Returns:\n new_dist: A batch-sliced `tfd.Distribution`.\n \"\"\"\n if not isinstance(slices, collections.Sequence):\n slices = (slices,)\n # We track the history of slice and copy(**param_overrides) in order to trace\n # back to the original distribution's source variables.\n orig_dist, slice_overrides_seq = getattr(dist, PROVENANCE_ATTR, (dist, []))\n slice_overrides_seq += [(slices, params_overrides)]\n # Re-doing the full sequence of slice+copy override work here enables\n # gradients all the way back to the original distribution's arguments.\n dist = _apply_slice_sequence(orig_dist, params_event_ndims,\n slice_overrides_seq)\n setattr(dist, PROVENANCE_ATTR, (orig_dist, slice_overrides_seq))\n return dist"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nfits multiple Fisher scoring steps.", "response": "def fit(\n model_matrix,\n response,\n model,\n model_coefficients_start=None,\n predicted_linear_response_start=None,\n l2_regularizer=None,\n dispersion=None,\n offset=None,\n convergence_criteria_fn=None,\n learning_rate=None,\n fast_unsafe_numerics=True,\n maximum_iterations=None,\n name=None):\n \"\"\"Runs multiple Fisher scoring steps.\n\n Args:\n model_matrix: (Batch of) `float`-like, matrix-shaped `Tensor` where each row\n represents a sample's features.\n response: (Batch of) vector-shaped `Tensor` where each element represents a\n sample's observed response (to the corresponding row of features). Must\n have same `dtype` as `model_matrix`.\n model: `tfp.glm.ExponentialFamily`-like instance which implicitly\n characterizes a negative log-likelihood loss by specifying the\n distribuion's `mean`, `gradient_mean`, and `variance`.\n model_coefficients_start: Optional (batch of) vector-shaped `Tensor`\n representing the initial model coefficients, one for each column in\n `model_matrix`. Must have same `dtype` as `model_matrix`.\n Default value: Zeros.\n predicted_linear_response_start: Optional `Tensor` with `shape`, `dtype`\n matching `response`; represents `offset` shifted initial linear\n predictions based on `model_coefficients_start`.\n Default value: `offset` if `model_coefficients is None`, and\n `tf.linalg.matvec(model_matrix, model_coefficients_start) + offset`\n otherwise.\n l2_regularizer: Optional scalar `Tensor` representing L2 regularization\n penalty, i.e.,\n `loss(w) = sum{-log p(y[i]|x[i],w) : i=1..n} + l2_regularizer ||w||_2^2`.\n Default value: `None` (i.e., no L2 regularization).\n dispersion: Optional (batch of) `Tensor` representing `response` dispersion,\n i.e., as in, `p(y|theta) := exp((y theta - A(theta)) / dispersion)`.\n Must broadcast with rows of `model_matrix`.\n Default value: `None` (i.e., \"no dispersion\").\n offset: Optional `Tensor` representing constant shift applied to\n `predicted_linear_response`. Must broadcast to `response`.\n Default value: `None` (i.e., `tf.zeros_like(response)`).\n convergence_criteria_fn: Python `callable` taking:\n `is_converged_previous`, `iter_`, `model_coefficients_previous`,\n `predicted_linear_response_previous`, `model_coefficients_next`,\n `predicted_linear_response_next`, `response`, `model`, `dispersion` and\n returning a `bool` `Tensor` indicating that Fisher scoring has converged.\n See `convergence_criteria_small_relative_norm_weights_change` as an\n example function.\n Default value: `None` (i.e.,\n `convergence_criteria_small_relative_norm_weights_change`).\n learning_rate: Optional (batch of) scalar `Tensor` used to dampen iterative\n progress. Typically only needed if optimization diverges, should be no\n larger than `1` and typically very close to `1`.\n Default value: `None` (i.e., `1`).\n fast_unsafe_numerics: Optional Python `bool` indicating if faster, less\n numerically accurate methods can be employed for computing the weighted\n least-squares solution.\n Default value: `True` (i.e., \"fast but possibly diminished accuracy\").\n maximum_iterations: Optional maximum number of iterations of Fisher scoring\n to run; \"and-ed\" with result of `convergence_criteria_fn`.\n Default value: `None` (i.e., `infinity`).\n name: Python `str` used as name prefix to ops created by this function.\n Default value: `\"fit\"`.\n\n Returns:\n model_coefficients: (Batch of) vector-shaped `Tensor`; represents the\n fitted model coefficients, one for each column in `model_matrix`.\n predicted_linear_response: `response`-shaped `Tensor` representing linear\n predictions based on new `model_coefficients`, i.e.,\n `tf.linalg.matvec(model_matrix, model_coefficients) + offset`.\n is_converged: `bool` `Tensor` indicating that the returned\n `model_coefficients` met the `convergence_criteria_fn` criteria within the\n `maximum_iterations` limit.\n iter_: `int32` `Tensor` indicating the number of iterations taken.\n\n #### Example\n\n ```python\n from __future__ import print_function\n import numpy as np\n import tensorflow as tf\n import tensorflow_probability as tfp\n tfd = tfp.distributions\n\n def make_dataset(n, d, link, scale=1., dtype=np.float32):\n model_coefficients = tfd.Uniform(\n low=np.array(-1, dtype),\n high=np.array(1, dtype)).sample(d, seed=42)\n radius = np.sqrt(2.)\n model_coefficients *= radius / tf.linalg.norm(model_coefficients)\n model_matrix = tfd.Normal(\n loc=np.array(0, dtype),\n scale=np.array(1, dtype)).sample([n, d], seed=43)\n scale = tf.convert_to_tensor(scale, dtype)\n linear_response = tf.tensordot(\n model_matrix, model_coefficients, axes=[[1], [0]])\n if link == 'linear':\n response = tfd.Normal(loc=linear_response, scale=scale).sample(seed=44)\n elif link == 'probit':\n response = tf.cast(\n tfd.Normal(loc=linear_response, scale=scale).sample(seed=44) > 0,\n dtype)\n elif link == 'logit':\n response = tfd.Bernoulli(logits=linear_response).sample(seed=44)\n else:\n raise ValueError('unrecognized true link: {}'.format(link))\n return model_matrix, response, model_coefficients\n\n X, Y, w_true = make_dataset(n=int(1e6), d=100, link='probit')\n\n w, linear_response, is_converged, num_iter = tfp.glm.fit(\n model_matrix=X,\n response=Y,\n model=tfp.glm.BernoulliNormalCDF())\n log_likelihood = tfp.glm.BernoulliNormalCDF().log_prob(Y, linear_response)\n\n with tf.Session() as sess:\n [w_, linear_response_, is_converged_, num_iter_, Y_, w_true_,\n log_likelihood_] = sess.run([\n w, linear_response, is_converged, num_iter, Y, w_true,\n log_likelihood])\n\n print('is_converged: ', is_converged_)\n print(' num_iter: ', num_iter_)\n print(' accuracy: ', np.mean((linear_response_ > 0.) == Y_))\n print(' deviance: ', 2. * np.mean(log_likelihood_))\n print('||w0-w1||_2 / (1+||w0||_2): ', (np.linalg.norm(w_true_ - w_, ord=2) /\n (1. + np.linalg.norm(w_true_, ord=2))))\n\n # ==>\n # is_converged: True\n # num_iter: 6\n # accuracy: 0.804382\n # deviance: -0.820746600628\n # ||w0-w1||_2 / (1+||w0||_2): 0.00619245105309\n ```\n\n \"\"\"\n graph_deps = [model_matrix, response, model_coefficients_start,\n predicted_linear_response_start, dispersion, offset,\n learning_rate, maximum_iterations]\n with tf.compat.v1.name_scope(name, 'fit', graph_deps):\n [\n model_matrix,\n response,\n model_coefficients_start,\n predicted_linear_response_start,\n offset,\n ] = prepare_args(\n model_matrix,\n response,\n model_coefficients_start,\n predicted_linear_response_start,\n offset)\n if convergence_criteria_fn is None:\n convergence_criteria_fn = (\n convergence_criteria_small_relative_norm_weights_change())\n\n def _body(\n is_converged_previous,\n iter_,\n model_coefficients_previous,\n predicted_linear_response_previous):\n \"\"\"`tf.while_loop` body.\"\"\"\n model_coefficients_next, predicted_linear_response_next = fit_one_step(\n model_matrix,\n response,\n model,\n model_coefficients_previous,\n predicted_linear_response_previous,\n l2_regularizer,\n dispersion,\n offset,\n learning_rate,\n fast_unsafe_numerics)\n is_converged_next = convergence_criteria_fn(\n is_converged_previous=is_converged_previous,\n iter_=iter_,\n model_coefficients_previous=model_coefficients_previous,\n predicted_linear_response_previous=predicted_linear_response_previous,\n model_coefficients_next=model_coefficients_next,\n predicted_linear_response_next=predicted_linear_response_next,\n response=response,\n model=model,\n dispersion=dispersion)\n return [\n is_converged_next,\n iter_ + 1,\n model_coefficients_next,\n predicted_linear_response_next,\n ]\n\n # while not converged:\n # fit_one_step\n [\n is_converged,\n iter_,\n model_coefficients,\n predicted_linear_response,\n ] = tf.while_loop(\n cond=lambda is_converged, *args: tf.logical_not(is_converged),\n body=_body,\n loop_vars=[\n tf.zeros([], np.bool), # is_converged\n tf.zeros([], np.int32), # iter_\n model_coefficients_start,\n predicted_linear_response_start,\n ],\n maximum_iterations=maximum_iterations)\n\n return [\n model_coefficients,\n predicted_linear_response,\n is_converged,\n iter_\n ]"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nfit one step of Fisher scoring.", "response": "def fit_one_step(\n model_matrix,\n response,\n model,\n model_coefficients_start=None,\n predicted_linear_response_start=None,\n l2_regularizer=None,\n dispersion=None,\n offset=None,\n learning_rate=None,\n fast_unsafe_numerics=True,\n name=None):\n \"\"\"Runs one step of Fisher scoring.\n\n Args:\n model_matrix: (Batch of) `float`-like, matrix-shaped `Tensor` where each row\n represents a sample's features.\n response: (Batch of) vector-shaped `Tensor` where each element represents a\n sample's observed response (to the corresponding row of features). Must\n have same `dtype` as `model_matrix`.\n model: `tfp.glm.ExponentialFamily`-like instance used to construct the\n negative log-likelihood loss, gradient, and expected Hessian (i.e., the\n Fisher information matrix).\n model_coefficients_start: Optional (batch of) vector-shaped `Tensor`\n representing the initial model coefficients, one for each column in\n `model_matrix`. Must have same `dtype` as `model_matrix`.\n Default value: Zeros.\n predicted_linear_response_start: Optional `Tensor` with `shape`, `dtype`\n matching `response`; represents `offset` shifted initial linear\n predictions based on `model_coefficients_start`.\n Default value: `offset` if `model_coefficients is None`, and\n `tf.linalg.matvec(model_matrix, model_coefficients_start) + offset`\n otherwise.\n l2_regularizer: Optional scalar `Tensor` representing L2 regularization\n penalty, i.e.,\n `loss(w) = sum{-log p(y[i]|x[i],w) : i=1..n} + l2_regularizer ||w||_2^2`.\n Default value: `None` (i.e., no L2 regularization).\n dispersion: Optional (batch of) `Tensor` representing `response` dispersion,\n i.e., as in, `p(y|theta) := exp((y theta - A(theta)) / dispersion)`.\n Must broadcast with rows of `model_matrix`.\n Default value: `None` (i.e., \"no dispersion\").\n offset: Optional `Tensor` representing constant shift applied to\n `predicted_linear_response`. Must broadcast to `response`.\n Default value: `None` (i.e., `tf.zeros_like(response)`).\n learning_rate: Optional (batch of) scalar `Tensor` used to dampen iterative\n progress. Typically only needed if optimization diverges, should be no\n larger than `1` and typically very close to `1`.\n Default value: `None` (i.e., `1`).\n fast_unsafe_numerics: Optional Python `bool` indicating if solve should be\n based on Cholesky or QR decomposition.\n Default value: `True` (i.e., \"prefer speed via Cholesky decomposition\").\n name: Python `str` used as name prefix to ops created by this function.\n Default value: `\"fit_one_step\"`.\n\n Returns:\n model_coefficients: (Batch of) vector-shaped `Tensor`; represents the\n next estimate of the model coefficients, one for each column in\n `model_matrix`.\n predicted_linear_response: `response`-shaped `Tensor` representing linear\n predictions based on new `model_coefficients`, i.e.,\n `tf.linalg.matvec(model_matrix, model_coefficients_next) + offset`.\n \"\"\"\n graph_deps = [model_matrix, response, model_coefficients_start,\n predicted_linear_response_start, dispersion, learning_rate]\n with tf.compat.v1.name_scope(name, 'fit_one_step', graph_deps):\n\n [\n model_matrix,\n response,\n model_coefficients_start,\n predicted_linear_response_start,\n offset,\n ] = prepare_args(\n model_matrix,\n response,\n model_coefficients_start,\n predicted_linear_response_start,\n offset)\n\n # Compute: mean, grad(mean, predicted_linear_response_start), and variance.\n mean, variance, grad_mean = model(predicted_linear_response_start)\n\n # If either `grad_mean` or `variance is non-finite or zero, then we'll\n # replace it with a value such that the row is zeroed out. Although this\n # procedure may seem circuitous, it is necessary to ensure this algorithm is\n # itself differentiable.\n is_valid = (\n tf.math.is_finite(grad_mean) & tf.not_equal(grad_mean, 0.)\n & tf.math.is_finite(variance) & (variance > 0.))\n\n def mask_if_invalid(x, mask):\n mask = tf.fill(\n tf.shape(input=x), value=np.array(mask, x.dtype.as_numpy_dtype))\n return tf.where(is_valid, x, mask)\n\n # Run one step of iteratively reweighted least-squares.\n # Compute \"`z`\", the adjusted predicted linear response.\n # z = predicted_linear_response_start\n # + learning_rate * (response - mean) / grad_mean\n z = (response - mean) / mask_if_invalid(grad_mean, 1.)\n # TODO(jvdillon): Rather than use learning rate, we should consider using\n # backtracking line search.\n if learning_rate is not None:\n z *= learning_rate[..., tf.newaxis]\n z += predicted_linear_response_start\n if offset is not None:\n z -= offset\n\n # Compute \"`w`\", the per-sample weight.\n if dispersion is not None:\n # For convenience, we'll now scale the variance by the dispersion factor.\n variance *= dispersion\n w = (\n mask_if_invalid(grad_mean, 0.) *\n tf.math.rsqrt(mask_if_invalid(variance, np.inf)))\n\n a = model_matrix * w[..., tf.newaxis]\n b = z * w\n # Solve `min{ || A @ model_coefficients - b ||_2**2 : model_coefficients }`\n # where `@` denotes `matmul`.\n\n if l2_regularizer is None:\n l2_regularizer = np.array(0, a.dtype.as_numpy_dtype)\n else:\n l2_regularizer_ = distribution_util.maybe_get_static_value(\n l2_regularizer, a.dtype.as_numpy_dtype)\n if l2_regularizer_ is not None:\n l2_regularizer = l2_regularizer_\n\n def _embed_l2_regularization():\n \"\"\"Adds synthetic observations to implement L2 regularization.\"\"\"\n # `tf.matrix_solve_ls` does not respect the `l2_regularization` argument\n # when `fast_unsafe_numerics` is `False`. This function adds synthetic\n # observations to the data to implement the regularization instead.\n # Adding observations `sqrt(l2_regularizer) * I` is mathematically\n # equivalent to adding the term\n # `-l2_regularizer ||coefficients||_2**2` to the log-likelihood.\n num_model_coefficients = num_cols(model_matrix)\n batch_shape = tf.shape(input=model_matrix)[:-2]\n eye = tf.eye(\n num_model_coefficients, batch_shape=batch_shape, dtype=a.dtype)\n a_ = tf.concat([a, tf.sqrt(l2_regularizer) * eye], axis=-2)\n b_ = distribution_util.pad(\n b, count=num_model_coefficients, axis=-1, back=True)\n # Return l2_regularizer=0 since its now embedded.\n l2_regularizer_ = np.array(0, a.dtype.as_numpy_dtype)\n return a_, b_, l2_regularizer_\n\n a, b, l2_regularizer = prefer_static.cond(\n prefer_static.reduce_all([not(fast_unsafe_numerics),\n l2_regularizer > 0.]),\n _embed_l2_regularization,\n lambda: (a, b, l2_regularizer))\n\n model_coefficients_next = tf.linalg.lstsq(\n a,\n b[..., tf.newaxis],\n fast=fast_unsafe_numerics,\n l2_regularizer=l2_regularizer,\n name='model_coefficients_next')\n model_coefficients_next = model_coefficients_next[..., 0]\n\n # TODO(b/79122261): The approach used in `matrix_solve_ls` could be made\n # faster by avoiding explicitly forming Q and instead keeping the\n # factorization in 'implicit' form with stacked (rescaled) Householder\n # vectors underneath the 'R' and then applying the (accumulated)\n # reflectors in the appropriate order to apply Q'. However, we don't\n # presently do this because we lack core TF functionality. For reference,\n # the vanilla QR approach is:\n # q, r = tf.linalg.qr(a)\n # c = tf.matmul(q, b, adjoint_a=True)\n # model_coefficients_next = tf.matrix_triangular_solve(\n # r, c, lower=False, name='model_coefficients_next')\n\n predicted_linear_response_next = calculate_linear_predictor(\n model_matrix,\n model_coefficients_next,\n offset,\n name='predicted_linear_response_next')\n\n return model_coefficients_next, predicted_linear_response_next"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nreturns a convergence criteria function which returns True if the fitting procedure has converged.", "response": "def convergence_criteria_small_relative_norm_weights_change(\n tolerance=1e-5,\n norm_order=2):\n \"\"\"Returns Python `callable` which indicates fitting procedure has converged.\n\n Writing old, new `model_coefficients` as `w0`, `w1`, this function\n defines convergence as,\n\n ```python\n relative_euclidean_norm = (tf.norm(w0 - w1, ord=2, axis=-1) /\n (1. + tf.norm(w0, ord=2, axis=-1)))\n reduce_all(relative_euclidean_norm < tolerance)\n ```\n\n where `tf.norm(x, ord=2)` denotes the [Euclidean norm](\n https://en.wikipedia.org/wiki/Norm_(mathematics)#Euclidean_norm) of `x`.\n\n Args:\n tolerance: `float`-like `Tensor` indicating convergence, i.e., when\n max relative Euclidean norm weights difference < tolerance`.\n Default value: `1e-5`.\n norm_order: Order of the norm. Default value: `2` (i.e., \"Euclidean norm\".)\n\n Returns:\n convergence_criteria_fn: Python `callable` which returns `bool` `Tensor`\n indicated fitting procedure has converged. (See inner function\n specification for argument signature.)\n Default value: `1e-5`.\n \"\"\"\n def convergence_criteria_fn(\n is_converged_previous, # pylint: disable=unused-argument\n iter_,\n model_coefficients_previous,\n predicted_linear_response_previous, # pylint: disable=unused-argument\n model_coefficients_next,\n predicted_linear_response_next, # pylint: disable=unused-argument\n response, # pylint: disable=unused-argument\n model, # pylint: disable=unused-argument\n dispersion): # pylint: disable=unused-argument\n \"\"\"Returns `bool` `Tensor` indicating if fitting procedure has converged.\n\n Args:\n is_converged_previous: \"old\" convergence results.\n iter_: Iteration number.\n model_coefficients_previous: \"old\" `model_coefficients`.\n predicted_linear_response_previous: \"old\" `predicted_linear_response`.\n model_coefficients_next: \"new\" `model_coefficients`.\n predicted_linear_response_next: \"new: `predicted_linear_response`.\n response: (Batch of) vector-shaped `Tensor` where each element represents\n a sample's observed response (to the corresponding row of features).\n Must have same `dtype` as `model_matrix`.\n model: `tfp.glm.ExponentialFamily`-like instance used to construct the\n negative log-likelihood loss, gradient, and expected Hessian (i.e., the\n Fisher information matrix).\n dispersion: `Tensor` representing `response` dispersion, i.e., as in:\n `p(y|theta) := exp((y theta - A(theta)) / dispersion)`. Must broadcast\n with rows of `model_matrix`.\n Default value: `None` (i.e., \"no dispersion\").\n\n Returns:\n is_converged: `bool` `Tensor`.\n \"\"\"\n relative_euclidean_norm = (\n tf.norm(\n tensor=model_coefficients_previous - model_coefficients_next,\n ord=norm_order,\n axis=-1) /\n (1. +\n tf.norm(tensor=model_coefficients_previous, ord=norm_order, axis=-1)))\n return (iter_ > 0) & tf.reduce_all(\n input_tensor=relative_euclidean_norm < tolerance)\n\n return convergence_criteria_fn"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\ncomputes model_matrix @ model_coefficients + offset.", "response": "def calculate_linear_predictor(model_matrix, model_coefficients, offset=None,\n name=None):\n \"\"\"Computes `model_matrix @ model_coefficients + offset`.\"\"\"\n with tf.compat.v1.name_scope(name, 'calculate_linear_predictor',\n [model_matrix, model_coefficients, offset]):\n predicted_linear_response = tf.linalg.matvec(model_matrix,\n model_coefficients)\n if offset is not None:\n predicted_linear_response += offset\n return predicted_linear_response"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef num_cols(x):\n if tf.compat.dimension_value(x.shape[-1]) is not None:\n return tf.compat.dimension_value(x.shape[-1])\n return tf.shape(input=x)[-1]", "response": "Returns the number of columns in a given Tensor."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nwraps original_fn preferring to call static_fn when inputs are static.", "response": "def _prefer_static(original_fn, static_fn):\n \"\"\"Wraps original_fn, preferring to call static_fn when inputs are static.\"\"\"\n original_spec = tf_inspect.getfullargspec(original_fn)\n static_spec = tf_inspect.getfullargspec(static_fn)\n if original_spec != static_spec:\n raise ValueError(\n 'Arg specs do not match: original={}, static={}, fn={}'.format(\n original_spec, static_spec, original_fn))\n @decorator.decorator\n def wrap(wrapped_fn, *args, **kwargs):\n del wrapped_fn\n [args_, kwargs_], all_static = _maybe_get_static_args([args, kwargs])\n if all_static:\n return static_fn(*args_, **kwargs_)\n return original_fn(*args, **kwargs)\n return wrap(original_fn)"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _copy_docstring(original_fn, new_fn):\n original_spec = tf_inspect.getfullargspec(original_fn)\n new_spec = tf_inspect.getfullargspec(new_fn)\n if original_spec != new_spec:\n raise ValueError(\n 'Arg specs do not match: original={}, new={}, fn={}'.format(\n original_spec, new_spec, original_fn))\n @decorator.decorator\n def wrap(wrapped_fn, *args, **kwargs):\n del wrapped_fn\n return new_fn(*args, **kwargs)\n return wrap(original_fn)", "response": "Wraps original_fn with the doc of new_fn."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef rank_from_shape(shape_tensor_fn, tensorshape=None):\n\n if tensorshape is None:\n shape_tensor = (shape_tensor_fn() if callable(shape_tensor_fn)\n else shape_tensor_fn)\n if (hasattr(shape_tensor, 'shape') and\n hasattr(shape_tensor.shape, 'num_elements')):\n ndims_ = tensorshape_util.num_elements(shape_tensor.shape)\n else:\n ndims_ = len(shape_tensor)\n ndims_fn = lambda: tf.size(input=shape_tensor)\n else:\n ndims_ = tensorshape_util.rank(tensorshape)\n ndims_fn = lambda: tf.size(input=shape_tensor_fn() # pylint: disable=g-long-lambda\n if callable(shape_tensor_fn)\n else shape_tensor_fn)\n return ndims_fn() if ndims_ is None else ndims_", "response": "Computes rank given a Tensor s shape."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nreturn either true_fn or false_fn depending on the predicate.", "response": "def cond(pred, true_fn=None, false_fn=None, name=None):\n \"\"\"Return either `true_fn()` if predicate `pred` is true else `false_fn()`.\n\n If `pred` is a bool or has a constant value, we return either `true_fn()`\n or `false_fn()`, otherwise we use `tf.cond` to dynamically route to both.\n\n Arguments:\n pred: A scalar determining whether to return the result of `true_fn` or\n `false_fn`.\n true_fn: The callable to be performed if pred is true.\n false_fn: The callable to be performed if pred is false.\n name: Optional name prefix when using `tf.cond`.\n\n Returns:\n Tensors returned by the call to either `true_fn` or `false_fn`.\n\n Raises:\n TypeError: If `true_fn` or `false_fn` is not callable.\n \"\"\"\n if not callable(true_fn):\n raise TypeError('`true_fn` must be callable.')\n if not callable(false_fn):\n raise TypeError('`false_fn` must be callable.')\n\n pred_value = _get_static_predicate(pred)\n if pred_value is not None:\n if pred_value:\n return true_fn()\n else:\n return false_fn()\n else:\n return tf.cond(pred=pred, true_fn=true_fn, false_fn=false_fn, name=name)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nlike tf. case except attempts to statically evaluate predicates.", "response": "def case(pred_fn_pairs, default=None, exclusive=False, name='smart_case'):\n \"\"\"Like tf.case, except attempts to statically evaluate predicates.\n\n If any predicate in `pred_fn_pairs` is a bool or has a constant value, the\n associated callable will be called or omitted depending on its value.\n Otherwise this functions like tf.case.\n\n Args:\n pred_fn_pairs: Dict or list of pairs of a boolean scalar tensor and a\n callable which returns a list of tensors.\n default: Optional callable that returns a list of tensors.\n exclusive: True iff at most one predicate is allowed to evaluate to `True`.\n name: A name for this operation (optional).\n\n Returns:\n The tensors returned by the first pair whose predicate evaluated to True, or\n those returned by `default` if none does.\n\n Raises:\n TypeError: If `pred_fn_pairs` is not a list/dictionary.\n TypeError: If `pred_fn_pairs` is a list but does not contain 2-tuples.\n TypeError: If `fns[i]` is not callable for any i, or `default` is not\n callable.\n \"\"\"\n return control_flow_ops._case_helper( # pylint: disable=protected-access\n cond, pred_fn_pairs, default, exclusive, name, allow_python_preds=True)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef log_prob(self, response, predicted_linear_response, name=None):\n\n with self._name_scope(\n name, 'log_prob', [response, predicted_linear_response]):\n dtype = dtype_util.common_dtype([response, predicted_linear_response])\n response = tf.convert_to_tensor(\n value=response, dtype=dtype, name='response')\n predicted_linear_response = tf.convert_to_tensor(\n value=predicted_linear_response, name='predicted_linear_response')\n return self._log_prob(response, predicted_linear_response)", "response": "Computes D ( param = mean ( r ). log_prob ( response ) for linear response r."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef mixture_stddev(mixture_weight_vector, mean_vector, stddev_vector):\n tensorshape_util.assert_has_rank(mixture_weight_vector.shape, 2)\n if not tensorshape_util.is_compatible_with(mean_vector.shape,\n mixture_weight_vector.shape):\n raise ValueError(\"Expecting means to have same shape as mixture weights.\")\n if not tensorshape_util.is_compatible_with(stddev_vector.shape,\n mixture_weight_vector.shape):\n raise ValueError(\"Expecting stddevs to have same shape as mixture weights.\")\n\n # Reshape the distribution parameters for batched vectorized dot products.\n pi_for_dot_prod = tf.expand_dims(mixture_weight_vector, axis=1)\n mu_for_dot_prod = tf.expand_dims(mean_vector, axis=2)\n sigma_for_dot_prod = tf.expand_dims(stddev_vector, axis=2)\n\n # weighted average of component means under mixture distribution.\n mean_wa = tf.matmul(pi_for_dot_prod, mu_for_dot_prod)\n mean_wa = tf.reshape(mean_wa, (-1,))\n # weighted average of component variances under mixture distribution.\n var_wa = tf.matmul(pi_for_dot_prod, tf.square(sigma_for_dot_prod))\n var_wa = tf.reshape(var_wa, (-1,))\n # weighted average of component squared means under mixture distribution.\n sq_mean_wa = tf.matmul(pi_for_dot_prod, tf.square(mu_for_dot_prod))\n sq_mean_wa = tf.reshape(sq_mean_wa, (-1,))\n mixture_variance = var_wa + sq_mean_wa - tf.square(mean_wa)\n return tf.sqrt(mixture_variance)", "response": "Computes the standard deviation of a mixture distribution."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\ncreating a LinearOperator representing a lower triangular matrix.", "response": "def make_tril_scale(loc=None,\n scale_tril=None,\n scale_diag=None,\n scale_identity_multiplier=None,\n shape_hint=None,\n validate_args=False,\n assert_positive=False,\n name=None):\n \"\"\"Creates a LinearOperator representing a lower triangular matrix.\n\n Args:\n loc: Floating-point `Tensor`. This is used for inferring shape in the case\n where only `scale_identity_multiplier` is set.\n scale_tril: Floating-point `Tensor` representing the diagonal matrix.\n `scale_diag` has shape [N1, N2, ... k, k], which represents a k x k lower\n triangular matrix. When `None` no `scale_tril` term is added to the\n LinearOperator. The upper triangular elements above the diagonal are\n ignored.\n scale_diag: Floating-point `Tensor` representing the diagonal matrix.\n `scale_diag` has shape [N1, N2, ... k], which represents a k x k diagonal\n matrix. When `None` no diagonal term is added to the LinearOperator.\n scale_identity_multiplier: floating point rank 0 `Tensor` representing a\n scaling done to the identity matrix. When `scale_identity_multiplier =\n scale_diag = scale_tril = None` then `scale += IdentityMatrix`. Otherwise\n no scaled-identity-matrix is added to `scale`.\n shape_hint: scalar integer `Tensor` representing a hint at the dimension of\n the identity matrix when only `scale_identity_multiplier` is set.\n validate_args: Python `bool` indicating whether arguments should be checked\n for correctness.\n assert_positive: Python `bool` indicating whether LinearOperator should be\n checked for being positive definite.\n name: Python `str` name given to ops managed by this object.\n\n Returns:\n `LinearOperator` representing a lower triangular matrix.\n\n Raises:\n ValueError: If only `scale_identity_multiplier` is set and `loc` and\n `shape_hint` are both None.\n \"\"\"\n\n def _maybe_attach_assertion(x):\n if not validate_args:\n return x\n if assert_positive:\n return with_dependencies([\n assert_util.assert_positive(\n tf.linalg.diag_part(x), message=\"diagonal part must be positive\"),\n ], x)\n return with_dependencies([\n assert_util.assert_none_equal(\n tf.linalg.diag_part(x),\n tf.zeros([], x.dtype),\n message=\"diagonal part must be non-zero\"),\n ], x)\n\n with tf.name_scope(name or \"make_tril_scale\"):\n\n dtype = dtype_util.common_dtype(\n [loc, scale_tril, scale_diag, scale_identity_multiplier],\n preferred_dtype=tf.float32)\n loc = _convert_to_tensor(loc, name=\"loc\", dtype=dtype)\n scale_tril = _convert_to_tensor(scale_tril, name=\"scale_tril\", dtype=dtype)\n scale_diag = _convert_to_tensor(scale_diag, name=\"scale_diag\", dtype=dtype)\n scale_identity_multiplier = _convert_to_tensor(\n scale_identity_multiplier,\n name=\"scale_identity_multiplier\",\n dtype=dtype)\n\n if scale_tril is not None:\n scale_tril = tf.linalg.band_part(scale_tril, -1, 0) # Zero out TriU.\n tril_diag = tf.linalg.diag_part(scale_tril)\n if scale_diag is not None:\n tril_diag += scale_diag\n if scale_identity_multiplier is not None:\n tril_diag += scale_identity_multiplier[..., tf.newaxis]\n\n scale_tril = tf.linalg.set_diag(scale_tril, tril_diag)\n\n return tf.linalg.LinearOperatorLowerTriangular(\n tril=_maybe_attach_assertion(scale_tril),\n is_non_singular=True,\n is_self_adjoint=False,\n is_positive_definite=assert_positive)\n\n return make_diag_scale(\n loc=loc,\n scale_diag=scale_diag,\n scale_identity_multiplier=scale_identity_multiplier,\n shape_hint=shape_hint,\n validate_args=validate_args,\n assert_positive=assert_positive,\n name=name)"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\ncreates a LinearOperator representing a diagonal matrix.", "response": "def make_diag_scale(loc=None,\n scale_diag=None,\n scale_identity_multiplier=None,\n shape_hint=None,\n validate_args=False,\n assert_positive=False,\n name=None,\n dtype=None):\n \"\"\"Creates a LinearOperator representing a diagonal matrix.\n\n Args:\n loc: Floating-point `Tensor`. This is used for inferring shape in the case\n where only `scale_identity_multiplier` is set.\n scale_diag: Floating-point `Tensor` representing the diagonal matrix.\n `scale_diag` has shape [N1, N2, ... k], which represents a k x k diagonal\n matrix. When `None` no diagonal term is added to the LinearOperator.\n scale_identity_multiplier: floating point rank 0 `Tensor` representing a\n scaling done to the identity matrix. When `scale_identity_multiplier =\n scale_diag = scale_tril = None` then `scale += IdentityMatrix`. Otherwise\n no scaled-identity-matrix is added to `scale`.\n shape_hint: scalar integer `Tensor` representing a hint at the dimension of\n the identity matrix when only `scale_identity_multiplier` is set.\n validate_args: Python `bool` indicating whether arguments should be checked\n for correctness.\n assert_positive: Python `bool` indicating whether LinearOperator should be\n checked for being positive definite.\n name: Python `str` name given to ops managed by this object.\n dtype: TF `DType` to prefer when converting args to `Tensor`s. Else, we fall\n back to a compatible dtype across all of `loc`, `scale_diag`, and\n `scale_identity_multiplier`.\n\n Returns:\n `LinearOperator` representing a lower triangular matrix.\n\n Raises:\n ValueError: If only `scale_identity_multiplier` is set and `loc` and\n `shape_hint` are both None.\n \"\"\"\n\n def _maybe_attach_assertion(x):\n if not validate_args:\n return x\n if assert_positive:\n return with_dependencies([\n assert_util.assert_positive(\n x, message=\"diagonal part must be positive\"),\n ], x)\n return with_dependencies([\n assert_util.assert_none_equal(\n x, tf.zeros([], x.dtype), message=\"diagonal part must be non-zero\")\n ], x)\n\n with tf.name_scope(name or \"make_diag_scale\"):\n if dtype is None:\n dtype = dtype_util.common_dtype(\n [loc, scale_diag, scale_identity_multiplier],\n preferred_dtype=tf.float32)\n loc = _convert_to_tensor(loc, name=\"loc\", dtype=dtype)\n scale_diag = _convert_to_tensor(scale_diag, name=\"scale_diag\", dtype=dtype)\n scale_identity_multiplier = _convert_to_tensor(\n scale_identity_multiplier,\n name=\"scale_identity_multiplier\",\n dtype=dtype)\n\n if scale_diag is not None:\n if scale_identity_multiplier is not None:\n scale_diag += scale_identity_multiplier[..., tf.newaxis]\n return tf.linalg.LinearOperatorDiag(\n diag=_maybe_attach_assertion(scale_diag),\n is_non_singular=True,\n is_self_adjoint=True,\n is_positive_definite=assert_positive)\n\n if loc is None and shape_hint is None:\n raise ValueError(\"Cannot infer `event_shape` unless `loc` or \"\n \"`shape_hint` is specified.\")\n\n num_rows = shape_hint\n del shape_hint\n if num_rows is None:\n num_rows = tf.compat.dimension_value(loc.shape[-1])\n if num_rows is None:\n num_rows = tf.shape(input=loc)[-1]\n\n if scale_identity_multiplier is None:\n return tf.linalg.LinearOperatorIdentity(\n num_rows=num_rows,\n dtype=dtype,\n is_self_adjoint=True,\n is_positive_definite=True,\n assert_proper_shapes=validate_args)\n\n return tf.linalg.LinearOperatorScaledIdentity(\n num_rows=num_rows,\n multiplier=_maybe_attach_assertion(scale_identity_multiplier),\n is_non_singular=True,\n is_self_adjoint=True,\n is_positive_definite=assert_positive,\n assert_proper_shapes=validate_args)"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\ninfer batch and event shapes from a location and scale.", "response": "def shapes_from_loc_and_scale(loc, scale, name=\"shapes_from_loc_and_scale\"):\n \"\"\"Infer distribution batch and event shapes from a location and scale.\n\n Location and scale family distributions determine their batch/event shape by\n broadcasting the `loc` and `scale` args. This helper does that broadcast,\n statically if possible.\n\n Batch shape broadcasts as per the normal rules.\n We allow the `loc` event shape to broadcast up to that of `scale`. We do not\n allow `scale`'s event shape to change. Therefore, the last dimension of `loc`\n must either be size `1`, or the same as `scale.range_dimension`.\n\n See `MultivariateNormalLinearOperator` for a usage example.\n\n Args:\n loc: `Tensor` (already converted to tensor) or `None`. If `None`, or\n `rank(loc)==0`, both batch and event shape are determined by `scale`.\n scale: A `LinearOperator` instance.\n name: A string name to prepend to created ops.\n\n Returns:\n batch_shape: `TensorShape` (if broadcast is done statically), or `Tensor`.\n event_shape: `TensorShape` (if broadcast is done statically), or `Tensor`.\n\n Raises:\n ValueError: If the last dimension of `loc` is determined statically to be\n different than the range of `scale`.\n \"\"\"\n if loc is not None and tensorshape_util.rank(loc.shape) == 0:\n loc = None # scalar loc is irrelevant to determining batch/event shape.\n with tf.name_scope(name):\n # Get event shape.\n event_size = scale.range_dimension_tensor()\n event_size_ = tf.get_static_value(event_size)\n loc_event_size_ = (None if loc is None\n else tf.compat.dimension_value(loc.shape[-1]))\n\n if event_size_ is not None and loc_event_size_ is not None:\n # Static check that event shapes match.\n if loc_event_size_ != 1 and loc_event_size_ != event_size_:\n raise ValueError(\n \"Event size of 'scale' ({}) could not be broadcast up to that \"\n \"of 'loc' ({}).\".format(event_size_, loc_event_size_))\n elif loc_event_size_ is not None and loc_event_size_ != 1:\n event_size_ = loc_event_size_\n\n if event_size_ is None:\n event_shape = event_size[tf.newaxis]\n else:\n event_shape = tf.convert_to_tensor(\n value=np.reshape(event_size_, [1]),\n dtype=tf.int32,\n name=\"event_shape\")\n\n # Get batch shape.\n batch_shape = scale.batch_shape_tensor()\n if loc is not None:\n loc_batch_shape = tensorshape_util.with_rank_at_least(loc.shape, 1)[:-1]\n if tensorshape_util.rank(\n loc.shape) is None or not tensorshape_util.is_fully_defined(\n loc_batch_shape):\n loc_batch_shape = tf.shape(input=loc)[:-1]\n else:\n loc_batch_shape = tf.convert_to_tensor(\n value=loc_batch_shape, dtype=tf.int32, name=\"loc_batch_shape\")\n # This is defined in the core util module.\n batch_shape = prefer_static_broadcast_shape(batch_shape, loc_batch_shape) # pylint: disable=undefined-variable\n batch_shape = tf.convert_to_tensor(\n value=batch_shape, dtype=tf.int32, name=\"batch_shape\")\n\n return batch_shape, event_shape"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\ngetting broadcast shape as a Python list of integers ( preferred ) or a Tensor.", "response": "def get_broadcast_shape(*tensors):\n \"\"\"Get broadcast shape as a Python list of integers (preferred) or `Tensor`.\n\n Args:\n *tensors: One or more `Tensor` objects (already converted!).\n\n Returns:\n broadcast shape: Python list (if shapes determined statically), otherwise\n an `int32` `Tensor`.\n \"\"\"\n # Try static.\n s_shape = tensors[0].shape\n for t in tensors[1:]:\n s_shape = tf.broadcast_static_shape(s_shape, t.shape)\n if tensorshape_util.is_fully_defined(s_shape):\n return tensorshape_util.as_list(s_shape)\n\n # Fallback on dynamic.\n d_shape = tf.shape(input=tensors[0])\n for t in tensors[1:]:\n d_shape = tf.broadcast_dynamic_shape(d_shape, tf.shape(input=t))\n return d_shape"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef is_diagonal_scale(scale):\n if not isinstance(scale, tf.linalg.LinearOperator):\n raise TypeError(\"Expected argument 'scale' to be instance of LinearOperator\"\n \". Found: %s\" % scale)\n return (isinstance(scale, tf.linalg.LinearOperatorIdentity) or\n isinstance(scale, tf.linalg.LinearOperatorScaledIdentity) or\n isinstance(scale, tf.linalg.LinearOperatorDiag))", "response": "Returns True if scale is a diagonal scale."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef pad_mixture_dimensions(x, mixture_distribution, categorical_distribution,\n event_ndims):\n \"\"\"Pad dimensions of event tensors for mixture distributions.\n\n See `Mixture._sample_n` and `MixtureSameFamily._sample_n` for usage examples.\n\n Args:\n x: event tensor to pad.\n mixture_distribution: Base distribution of the mixture.\n categorical_distribution: `Categorical` distribution that mixes the base\n distribution.\n event_ndims: Integer specifying the number of event dimensions in the event\n tensor.\n\n Returns:\n A padded version of `x` that can broadcast with `categorical_distribution`.\n \"\"\"\n with tf.name_scope(\"pad_mix_dims\"):\n\n def _get_ndims(d):\n if tensorshape_util.rank(d.batch_shape) is not None:\n return tensorshape_util.rank(d.batch_shape)\n return tf.shape(input=d.batch_shape_tensor())[0]\n\n dist_batch_ndims = _get_ndims(mixture_distribution)\n cat_batch_ndims = _get_ndims(categorical_distribution)\n pad_ndims = tf.where(categorical_distribution.is_scalar_batch(),\n dist_batch_ndims, dist_batch_ndims - cat_batch_ndims)\n s = tf.shape(input=x)\n x = tf.reshape(\n x,\n shape=tf.concat([\n s[:-1],\n tf.ones([pad_ndims], dtype=tf.int32),\n s[-1:],\n tf.ones([event_ndims], dtype=tf.int32),\n ],\n axis=0))\n return x", "response": "Pad dimensions of event tensors for mixture distributions."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef pick_scalar_condition(pred, true_value, false_value, name=None):\n with tf.name_scope(name or \"pick_scalar_condition\"):\n pred = tf.convert_to_tensor(\n value=pred, dtype_hint=tf.bool, name=\"pred\")\n true_value = tf.convert_to_tensor(value=true_value, name=\"true_value\")\n false_value = tf.convert_to_tensor(value=false_value, name=\"false_value\")\n pred_ = tf.get_static_value(pred)\n if pred_ is None:\n return tf.where(pred, true_value, false_value)\n return true_value if pred_ else false_value", "response": "Convenience function that chooses one of two values based on the predicate."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nmaking possibly negatively indexed axis non - negative.", "response": "def make_non_negative_axis(axis, rank):\n \"\"\"Make (possibly negatively indexed) `axis` argument non-negative.\"\"\"\n axis = tf.convert_to_tensor(value=axis, name=\"axis\")\n rank = tf.convert_to_tensor(value=rank, name=\"rank\")\n axis_ = tf.get_static_value(axis)\n rank_ = tf.get_static_value(rank)\n\n # Static case.\n if axis_ is not None and rank_ is not None:\n is_scalar = axis_.ndim == 0\n if is_scalar:\n axis_ = [axis_]\n positive_axis = []\n for a_ in axis_:\n if a_ < 0:\n positive_axis.append(rank_ + a_)\n else:\n positive_axis.append(a_)\n if is_scalar:\n positive_axis = positive_axis[0]\n return tf.convert_to_tensor(value=positive_axis, dtype=axis.dtype)\n\n # Dynamic case.\n # Unfortunately static values are lost by this tf.where.\n return tf.where(axis < 0, rank + axis, axis)"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef move_dimension(x, source_idx, dest_idx):\n ndims = prefer_static_rank(x)\n dtype = dtype_util.common_dtype([source_idx, dest_idx],\n preferred_dtype=tf.int32)\n source_idx = tf.convert_to_tensor(value=source_idx, dtype=dtype)\n dest_idx = tf.convert_to_tensor(value=dest_idx, dtype=dtype)\n\n # Handle negative indexing.\n source_idx = pick_scalar_condition(source_idx < 0, ndims + source_idx,\n source_idx)\n dest_idx = pick_scalar_condition(dest_idx < 0, ndims + dest_idx, dest_idx)\n\n # Construct the appropriate permutation of dimensions, depending\n # whether the source is before or after the destination.\n def move_left_permutation():\n return prefer_static_value(\n tf.concat([\n tf.range(0, dest_idx, dtype=dtype), [source_idx],\n tf.range(dest_idx, source_idx, dtype=dtype),\n tf.range(source_idx + 1, ndims, dtype=dtype)\n ],\n axis=0))\n\n def move_right_permutation():\n return prefer_static_value(\n tf.concat([\n tf.range(0, source_idx, dtype=dtype),\n tf.range(source_idx + 1, dest_idx + 1, dtype=dtype), [source_idx],\n tf.range(dest_idx + 1, ndims, dtype=dtype)\n ],\n axis=0))\n\n def x_permuted():\n return tf.transpose(\n a=x,\n perm=prefer_static.cond(source_idx < dest_idx,\n move_right_permutation,\n move_left_permutation))\n\n # One final conditional to handle the special case where source\n # and destination indices are equal.\n return prefer_static.cond(tf.equal(source_idx, dest_idx),\n lambda: x, x_permuted)", "response": "Move a single tensor dimension within its shape."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef assert_integer_form(x,\n data=None,\n summarize=None,\n message=None,\n int_dtype=None,\n name=\"assert_integer_form\"):\n \"\"\"Assert that x has integer components (or floats equal to integers).\n\n Args:\n x: Floating-point `Tensor`\n data: The tensors to print out if the condition is `False`. Defaults to\n error message and first few entries of `x` and `y`.\n summarize: Print this many entries of each tensor.\n message: A string to prefix to the default message.\n int_dtype: A `tf.dtype` used to cast the float to. The default (`None`)\n implies the smallest possible signed int will be used for casting.\n name: A name for this operation (optional).\n\n Returns:\n Op raising `InvalidArgumentError` if `cast(x, int_dtype) != x`.\n \"\"\"\n with tf.name_scope(name):\n x = tf.convert_to_tensor(value=x, name=\"x\")\n if dtype_util.is_integer(x.dtype):\n return tf.no_op()\n message = message or \"{} has non-integer components\".format(x)\n if int_dtype is None:\n try:\n int_dtype = {\n tf.float16: tf.int16,\n tf.float32: tf.int32,\n tf.float64: tf.int64,\n }[dtype_util.base_dtype(x.dtype)]\n except KeyError:\n raise TypeError(\"Unrecognized type {}\".format(dtype_util.name(x.dtype)))\n return assert_util.assert_equal(\n x,\n tf.cast(tf.cast(x, int_dtype), x.dtype),\n data=data,\n summarize=summarize,\n message=message,\n name=name)", "response": "Assert that x has integer components or floats equal to integers."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef embed_check_nonnegative_integer_form(\n x, name=\"embed_check_nonnegative_integer_form\"):\n \"\"\"Assert x is a non-negative tensor, and optionally of integers.\"\"\"\n with tf.name_scope(name):\n x = tf.convert_to_tensor(value=x, name=\"x\")\n assertions = [\n assert_util.assert_non_negative(\n x, message=\"'{}' must be non-negative.\".format(x)),\n ]\n if not dtype_util.is_integer(x.dtype):\n assertions += [\n assert_integer_form(\n x,\n message=\"'{}' cannot contain fractional components.\".format(x)),\n ]\n return with_dependencies(assertions, x)", "response": "Embeds x to be a non - negative tensor and optionally of integers."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nreturning True if both tensors have the same dynamic shape.", "response": "def same_dynamic_shape(a, b):\n \"\"\"Returns whether a and b have the same dynamic shape.\n\n Args:\n a: `Tensor`\n b: `Tensor`\n\n Returns:\n `bool` `Tensor` representing if both tensors have the same shape.\n \"\"\"\n a = tf.convert_to_tensor(value=a, name=\"a\")\n b = tf.convert_to_tensor(value=b, name=\"b\")\n\n # Here we can't just do tf.equal(a.shape, b.shape), since\n # static shape inference may break the equality comparison between\n # shape(a) and shape(b) in tf.equal.\n def all_shapes_equal():\n return tf.reduce_all(\n input_tensor=tf.equal(\n tf.concat([tf.shape(input=a), tf.shape(input=b)], 0),\n tf.concat([tf.shape(input=b), tf.shape(input=a)], 0)))\n\n # One of the shapes isn't fully defined, so we need to use the dynamic\n # shape.\n return tf.cond(\n pred=tf.equal(tf.rank(a), tf.rank(b)),\n true_fn=all_shapes_equal,\n false_fn=lambda: tf.constant(False))"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef maybe_get_static_value(x, dtype=None):\n if x is None:\n return x\n try:\n # This returns an np.ndarray.\n x_ = tf.get_static_value(x)\n except TypeError:\n x_ = x\n if x_ is None or dtype is None:\n return x_\n return np.array(x_, dtype)", "response": "Helper which tries to return a static value."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nconvert logit to probabilities or vice - versa and returns both.", "response": "def get_logits_and_probs(logits=None,\n probs=None,\n multidimensional=False,\n validate_args=False,\n name=\"get_logits_and_probs\",\n dtype=None):\n \"\"\"Converts logit to probabilities (or vice-versa), and returns both.\n\n Args:\n logits: Floating-point `Tensor` representing log-odds.\n probs: Floating-point `Tensor` representing probabilities.\n multidimensional: Python `bool`, default `False`. If `True`, represents\n whether the last dimension of `logits` or `probs`, a `[N1, N2, ... k]`\n dimensional tensor, representing the logit or probability of `shape[-1]`\n classes.\n validate_args: Python `bool`, default `False`. When `True`, either assert `0\n <= probs <= 1` (if not `multidimensional`) or that the last dimension of\n `probs` sums to one.\n name: A name for this operation (optional).\n dtype: `tf.DType` to prefer when converting args to `Tensor`s.\n\n Returns:\n logits, probs: Tuple of `Tensor`s. If `probs` has an entry that is `0` or\n `1`, then the corresponding entry in the returned logit will be `-Inf` and\n `Inf` respectively.\n\n Raises:\n ValueError: if neither `probs` nor `logits` were passed in, or both were.\n \"\"\"\n if dtype is None:\n dtype = dtype_util.common_dtype([probs, logits], preferred_dtype=tf.float32)\n with tf.name_scope(name):\n if (probs is None) == (logits is None):\n raise ValueError(\"Must pass probs or logits, but not both.\")\n\n if probs is None:\n logits = tf.convert_to_tensor(value=logits, name=\"logits\", dtype=dtype)\n if not dtype_util.is_floating(logits.dtype):\n raise TypeError(\"logits must having floating type.\")\n # We can early return since we constructed probs and therefore know\n # they're valid.\n if multidimensional:\n if validate_args:\n logits = embed_check_categorical_event_shape(logits)\n return logits, tf.nn.softmax(logits, name=\"probs\")\n return logits, tf.sigmoid(logits, name=\"probs\")\n\n probs = tf.convert_to_tensor(value=probs, name=\"probs\", dtype=dtype)\n if not dtype_util.is_floating(probs.dtype):\n raise TypeError(\"probs must having floating type.\")\n\n if validate_args:\n with tf.name_scope(\"validate_probs\"):\n one = tf.constant(1., probs.dtype)\n dependencies = [assert_util.assert_non_negative(probs)]\n if multidimensional:\n probs = embed_check_categorical_event_shape(probs)\n dependencies += [\n assert_util.assert_near(\n tf.reduce_sum(input_tensor=probs, axis=-1),\n one,\n message=\"probs does not sum to 1.\")\n ]\n else:\n dependencies += [\n assert_util.assert_less_equal(\n probs, one, message=\"probs has components greater than 1.\")\n ]\n probs = with_dependencies(dependencies, probs)\n\n with tf.name_scope(\"logits\"):\n if multidimensional:\n # Here we don't compute the multidimensional case, in a manner\n # consistent with respect to the unidimensional case. We do so\n # following the TF convention. Typically, you might expect to see\n # logits = log(probs) - log(probs[pivot]). A side-effect of\n # being consistent with the TF approach is that the unidimensional case\n # implicitly handles the second dimension but the multidimensional case\n # explicitly keeps the pivot dimension.\n return tf.math.log(probs), probs\n return tf.math.log(probs) - tf.math.log1p(-1. * probs), probs"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef _largest_integer_by_dtype(dt):\n if not _is_known_dtype(dt):\n raise TypeError(\"Unrecognized dtype: {}\".format(dt.name))\n if dt.is_floating:\n return int(2**(np.finfo(dt.as_numpy_dtype).nmant + 1))\n if dt.is_integer:\n return np.iinfo(dt.as_numpy_dtype).max\n if dt.base_dtype == tf.bool:\n return int(1)\n # We actually can't land here but keep the case for completeness.\n raise TypeError(\"Unrecognized dtype: {}\".format(dt.name))", "response": "Helper returning the largest integer exactly representable by dtype."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef embed_check_categorical_event_shape(\n categorical_param, name=\"embed_check_categorical_event_shape\"):\n \"\"\"Embeds checks that categorical distributions don't have too many classes.\n\n A categorical-type distribution is one which, e.g., returns the class label\n rather than a one-hot encoding. E.g., `Categorical(probs)`.\n\n Since distributions output samples in the same dtype as the parameters, we\n must ensure that casting doesn't lose precision. That is, the\n `parameter.dtype` implies a maximum number of classes. However, since shape is\n `int32` and categorical variables are presumed to be indexes into a `Tensor`,\n we must also ensure that the number of classes is no larger than the largest\n possible `int32` index, i.e., `2**31-1`.\n\n In other words the number of classes, `K`, must satisfy the following\n condition:\n\n ```python\n K <= min(\n int(2**31 - 1), # Largest float as an index.\n {\n tf.float16: int(2**11), # Largest int as a float16.\n tf.float32: int(2**24),\n tf.float64: int(2**53),\n }.get(dtype_util.base_dtype(categorical_param.dtype), 0))\n ```\n\n Args:\n categorical_param: Floating-point `Tensor` representing parameters of\n distribution over categories. The rightmost shape is presumed to be the\n number of categories.\n name: A name for this operation (optional).\n\n Returns:\n categorical_param: Input `Tensor` with appropriate assertions embedded.\n\n Raises:\n TypeError: if `categorical_param` has an unknown `dtype`.\n ValueError: if we can statically identify `categorical_param` as being too\n large (for being closed under int32/float casting).\n \"\"\"\n with tf.name_scope(name):\n x = tf.convert_to_tensor(value=categorical_param, name=\"categorical_param\")\n # The size must not exceed both of:\n # - The largest possible int32 (since categorical values are presumed to be\n # indexes into a Tensor).\n # - The largest possible integer exactly representable under the given\n # floating-point dtype (since we need to cast to/from).\n #\n # The chosen floating-point thresholds are 2**(1 + mantissa_bits).\n # For more details, see:\n # https://en.wikipedia.org/wiki/Floating-point_arithmetic#Internal_representation\n x_dtype = dtype_util.base_dtype(x.dtype)\n max_event_size = (\n _largest_integer_by_dtype(x_dtype)\n if dtype_util.is_floating(x_dtype) else 0)\n if max_event_size is 0:\n raise TypeError(\"Unable to validate size of unrecognized dtype \"\n \"({}).\".format(dtype_util.name(x_dtype)))\n try:\n x_shape_static = tensorshape_util.with_rank_at_least(x.shape, 1)\n except ValueError:\n raise ValueError(\"A categorical-distribution parameter must have \"\n \"at least 1 dimension.\")\n event_size = tf.compat.dimension_value(x_shape_static[-1])\n if event_size is not None:\n if event_size < 2:\n raise ValueError(\"A categorical-distribution parameter must have at \"\n \"least 2 events.\")\n if event_size > max_event_size:\n raise ValueError(\"Number of classes exceeds `dtype` precision, i.e., \"\n \"{} implies shape ({}) cannot exceed {}.\".format(\n dtype_util.name(x_dtype), event_size,\n max_event_size))\n return x\n else:\n event_size = tf.shape(input=x, out_type=tf.int64, name=\"x_shape\")[-1]\n return with_dependencies([\n assert_util.assert_rank_at_least(\n x,\n 1,\n message=(\"A categorical-distribution parameter must have \"\n \"at least 1 dimension.\")),\n assert_util.assert_greater_equal(\n tf.shape(input=x)[-1],\n 2,\n message=(\"A categorical-distribution parameter must have at \"\n \"least 2 events.\")),\n assert_util.assert_less_equal(\n event_size,\n tf.convert_to_tensor(max_event_size, dtype=tf.int64),\n message=\"Number of classes exceeds `dtype` precision, \"\n \"i.e., {} dtype cannot exceed {} shape.\".format(\n dtype_util.name(x_dtype), max_event_size)),\n ], x)", "response": "Embeds checks that categorical - type distributions don t have too many classes."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef embed_check_integer_casting_closed(x,\n target_dtype,\n assert_nonnegative=True,\n assert_positive=False,\n name=\"embed_check_casting_closed\"):\n \"\"\"Ensures integers remain unaffected despite casting to/from int/float types.\n\n Example integer-types: `uint8`, `int32`, `bool`.\n Example floating-types: `float32`, `float64`.\n\n The largest possible integer representable by an IEEE754 floating-point is\n `2**(1 + mantissa_bits)` yet the largest possible integer as an int-type is\n `2**(bits - 1) - 1`. This function ensures that a `Tensor` purporting to have\n integer-form values can be cast to some other type without loss of precision.\n\n The smallest representable integer is the negative of the largest\n representable integer, except for types: `uint8`, `uint16`, `bool`. For these\n types, the smallest representable integer is `0`.\n\n Args:\n x: `Tensor` representing integer-form values.\n target_dtype: TF `dtype` under which `x` should have identical values.\n assert_nonnegative: `bool` indicating `x` should contain nonnegative values.\n assert_positive: `bool` indicating `x` should contain positive values.\n name: A name for this operation (optional).\n\n Returns:\n x: Input `Tensor` with appropriate assertions embedded.\n\n Raises:\n TypeError: if `x` is neither integer- nor floating-type.\n TypeError: if `target_dtype` is neither integer- nor floating-type.\n TypeError: if neither `x` nor `target_dtype` are integer-type.\n \"\"\"\n\n with tf.name_scope(name):\n x = tf.convert_to_tensor(value=x, name=\"x\")\n if (not _is_integer_like_by_dtype(x.dtype) and\n not dtype_util.is_floating(x.dtype)):\n raise TypeError(\"{}.dtype must be floating- or \"\n \"integer-type.\".format(dtype_util.name(x.dtype)))\n if (not _is_integer_like_by_dtype(target_dtype) and\n not dtype_util.is_floating(target_dtype)):\n raise TypeError(\"target_dtype ({}) must be floating- or \"\n \"integer-type.\".format(dtype_util.name(target_dtype)))\n if (not _is_integer_like_by_dtype(x.dtype) and\n not _is_integer_like_by_dtype(target_dtype)):\n raise TypeError(\"At least one of {}.dtype ({}) and target_dtype ({}) \"\n \"must be integer-type.\".format(\n x, dtype_util.name(x.dtype),\n dtype_util.name(target_dtype)))\n\n assertions = []\n if assert_positive:\n assertions += [\n assert_util.assert_positive(x, message=\"Elements must be positive.\"),\n ]\n elif assert_nonnegative:\n assertions += [\n assert_util.assert_non_negative(\n x, message=\"Elements must be non-negative.\"),\n ]\n\n if dtype_util.is_floating(x.dtype):\n # Being here means _is_integer_like_by_dtype(target_dtype) = True.\n # Since this check implies the magnitude check below, we need only it.\n assertions += [\n assert_integer_form(\n x,\n int_dtype=target_dtype,\n message=\"Elements must be {}-equivalent.\".format(\n dtype_util.name(target_dtype))),\n ]\n else:\n if (_largest_integer_by_dtype(x.dtype) >\n _largest_integer_by_dtype(target_dtype)):\n # Cast may lose integer precision.\n assertions += [\n assert_util.assert_less_equal(\n x,\n _largest_integer_by_dtype(target_dtype),\n message=(\"Elements cannot exceed {}.\".format(\n _largest_integer_by_dtype(target_dtype)))),\n ]\n if (not assert_nonnegative and (_smallest_integer_by_dtype(\n x.dtype) < _smallest_integer_by_dtype(target_dtype))):\n assertions += [\n assert_util.assert_greater_equal(\n x,\n _smallest_integer_by_dtype(target_dtype),\n message=(\"Elements cannot be smaller than {}.\".format(\n _smallest_integer_by_dtype(target_dtype)))),\n ]\n\n if not assertions:\n return x\n return with_dependencies(assertions, x)", "response": "Embeds an integer - casting - closed tensor to a single integer - form."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\ncompute the log combinations of n and counts.", "response": "def log_combinations(n, counts, name=\"log_combinations\"):\n \"\"\"Multinomial coefficient.\n\n Given `n` and `counts`, where `counts` has last dimension `k`, we compute\n the multinomial coefficient as:\n\n ```n! / sum_i n_i!```\n\n where `i` runs over all `k` classes.\n\n Args:\n n: Floating-point `Tensor` broadcastable with `counts`. This represents `n`\n outcomes.\n counts: Floating-point `Tensor` broadcastable with `n`. This represents\n counts in `k` classes, where `k` is the last dimension of the tensor.\n name: A name for this operation (optional).\n\n Returns:\n `Tensor` representing the multinomial coefficient between `n` and `counts`.\n \"\"\"\n # First a bit about the number of ways counts could have come in:\n # E.g. if counts = [1, 2], then this is 3 choose 2.\n # In general, this is (sum counts)! / sum(counts!)\n # The sum should be along the last dimension of counts. This is the\n # \"distribution\" dimension. Here n a priori represents the sum of counts.\n with tf.name_scope(name):\n n = tf.convert_to_tensor(value=n, name=\"n\")\n counts = tf.convert_to_tensor(value=counts, name=\"counts\")\n total_permutations = tf.math.lgamma(n + 1)\n counts_factorial = tf.math.lgamma(counts + 1)\n redundant_permutations = tf.reduce_sum(\n input_tensor=counts_factorial, axis=[-1])\n return total_permutations - redundant_permutations"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef matrix_diag_transform(matrix, transform=None, name=None):\n with tf.name_scope(name or \"matrix_diag_transform\"):\n matrix = tf.convert_to_tensor(value=matrix, name=\"matrix\")\n if transform is None:\n return matrix\n # Replace the diag with transformed diag.\n diag = tf.linalg.diag_part(matrix)\n transformed_diag = transform(diag)\n transformed_mat = tf.linalg.set_diag(matrix, transformed_diag)\n\n return transformed_mat", "response": "Transform diagonal of matrix."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef rotate_transpose(x, shift, name=\"rotate_transpose\"):\n with tf.name_scope(name):\n x = tf.convert_to_tensor(value=x, name=\"x\")\n shift = tf.convert_to_tensor(value=shift, name=\"shift\")\n # We do not assign back to preserve constant-ness.\n assert_util.assert_integer(shift)\n shift_value_static = tf.get_static_value(shift)\n ndims = tensorshape_util.rank(x.shape)\n if ndims is not None and shift_value_static is not None:\n if ndims < 2:\n return x\n shift_value_static = np.sign(shift_value_static) * (\n abs(shift_value_static) % ndims)\n if shift_value_static == 0:\n return x\n perm = np.roll(np.arange(ndims), shift_value_static)\n return tf.transpose(a=x, perm=perm)\n else:\n # Consider if we always had a positive shift, and some specified\n # direction.\n # When shifting left we want the new array:\n # last(x, n-shift) + first(x, shift)\n # and if shifting right then we want:\n # last(x, shift) + first(x, n-shift)\n # Observe that last(a) == slice(a, n) and first(a) == slice(0, a).\n # Also, we can encode direction and shift as one: direction * shift.\n # Combining these facts, we have:\n # a = cond(shift<0, -shift, n-shift)\n # last(x, n-a) + first(x, a) == x[a:n] + x[0:a]\n # Finally, we transform shift by modulo length so it can be specified\n # independently from the array upon which it operates (like python).\n ndims = tf.rank(x)\n shift = tf.where(\n tf.less(shift, 0), -shift % ndims,\n ndims - shift % ndims)\n first = tf.range(0, shift)\n last = tf.range(shift, ndims)\n perm = tf.concat([last, first], 0)\n return tf.transpose(a=x, perm=perm)", "response": "Rotates x by shift."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef pick_vector(cond, true_vector, false_vector, name=\"pick_vector\"):\n with tf.name_scope(name):\n cond = tf.convert_to_tensor(\n value=cond, dtype_hint=tf.bool, name=\"cond\")\n if cond.dtype != tf.bool:\n raise TypeError(\n \"{}.dtype={} which is not {}\".format(cond, cond.dtype, tf.bool))\n\n true_vector = tf.convert_to_tensor(value=true_vector, name=\"true_vector\")\n false_vector = tf.convert_to_tensor(value=false_vector, name=\"false_vector\")\n if true_vector.dtype != false_vector.dtype:\n raise TypeError(\n \"{}.dtype={} does not match {}.dtype={}\".format(\n true_vector, true_vector.dtype, false_vector, false_vector.dtype))\n\n cond_value_static = tf.get_static_value(cond)\n if cond_value_static is not None:\n return true_vector if cond_value_static else false_vector\n n = tf.shape(input=true_vector)[0]\n return tf.slice(\n tf.concat([true_vector, false_vector], 0), [tf.where(cond, 0, n)],\n [tf.where(cond, n, -1)])", "response": "Picks possibly different length row tensors based on condition."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef gen_new_seed(seed, salt):\n if seed is None:\n return None\n string = (str(seed) + salt).encode(\"utf-8\")\n return int(hashlib.md5(string).hexdigest()[:8], 16) & 0x7FFFFFFF", "response": "Generate a new seed from the given seed and salt."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef fill_triangular(x, upper=False, name=None):\n\n with tf.name_scope(name or \"fill_triangular\"):\n x = tf.convert_to_tensor(value=x, name=\"x\")\n m = tf.compat.dimension_value(\n tensorshape_util.with_rank_at_least(x.shape, 1)[-1])\n if m is not None:\n # Formula derived by solving for n: m = n(n+1)/2.\n m = np.int32(m)\n n = np.sqrt(0.25 + 2. * m) - 0.5\n if n != np.floor(n):\n raise ValueError(\"Input right-most shape ({}) does not \"\n \"correspond to a triangular matrix.\".format(m))\n n = np.int32(n)\n static_final_shape = x.shape[:-1].concatenate([n, n])\n else:\n m = tf.shape(input=x)[-1]\n # For derivation, see above. Casting automatically lops off the 0.5, so we\n # omit it. We don't validate n is an integer because this has\n # graph-execution cost; an error will be thrown from the reshape, below.\n n = tf.cast(\n tf.sqrt(0.25 + tf.cast(2 * m, dtype=tf.float32)), dtype=tf.int32)\n static_final_shape = tensorshape_util.with_rank_at_least(\n x.shape, 1)[:-1].concatenate([None, None])\n\n # Try it out in numpy:\n # n = 3\n # x = np.arange(n * (n + 1) / 2)\n # m = x.shape[0]\n # n = np.int32(np.sqrt(.25 + 2 * m) - .5)\n # x_tail = x[(m - (n**2 - m)):]\n # np.concatenate([x_tail, x[::-1]], 0).reshape(n, n) # lower\n # # ==> array([[3, 4, 5],\n # [5, 4, 3],\n # [2, 1, 0]])\n # np.concatenate([x, x_tail[::-1]], 0).reshape(n, n) # upper\n # # ==> array([[0, 1, 2],\n # [3, 4, 5],\n # [5, 4, 3]])\n #\n # Note that we can't simply do `x[..., -(n**2 - m):]` because this doesn't\n # correctly handle `m == n == 1`. Hence, we do nonnegative indexing.\n # Furthermore observe that:\n # m - (n**2 - m)\n # = n**2 / 2 + n / 2 - (n**2 - n**2 / 2 + n / 2)\n # = 2 (n**2 / 2 + n / 2) - n**2\n # = n**2 + n - n**2\n # = n\n ndims = prefer_static_rank(x)\n if upper:\n x_list = [x, tf.reverse(x[..., n:], axis=[ndims - 1])]\n else:\n x_list = [x[..., n:], tf.reverse(x, axis=[ndims - 1])]\n new_shape = (\n tensorshape_util.as_list(static_final_shape)\n if tensorshape_util.is_fully_defined(static_final_shape) else tf.concat(\n [tf.shape(input=x)[:-1], [n, n]], axis=0))\n x = tf.reshape(tf.concat(x_list, axis=-1), new_shape)\n x = tf.linalg.band_part(\n x, num_lower=(0 if upper else -1), num_upper=(-1 if upper else 0))\n tensorshape_util.set_shape(x, static_final_shape)\n return x", "response": "r Creates a triangular matrix from a vector of inputs."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\ncreate a vector from a batch of triangular elements.", "response": "def fill_triangular_inverse(x, upper=False, name=None):\n \"\"\"Creates a vector from a (batch of) triangular matrix.\n\n The vector is created from the lower-triangular or upper-triangular portion\n depending on the value of the parameter `upper`.\n\n If `x.shape` is `[b1, b2, ..., bB, n, n]` then the output shape is\n `[b1, b2, ..., bB, d]` where `d = n (n + 1) / 2`.\n\n Example:\n\n ```python\n fill_triangular_inverse(\n [[4, 0, 0],\n [6, 5, 0],\n [3, 2, 1]])\n\n # ==> [1, 2, 3, 4, 5, 6]\n\n fill_triangular_inverse(\n [[1, 2, 3],\n [0, 5, 6],\n [0, 0, 4]], upper=True)\n\n # ==> [1, 2, 3, 4, 5, 6]\n ```\n\n Args:\n x: `Tensor` representing lower (or upper) triangular elements.\n upper: Python `bool` representing whether output matrix should be upper\n triangular (`True`) or lower triangular (`False`, default).\n name: Python `str`. The name to give this op.\n\n Returns:\n flat_tril: (Batch of) vector-shaped `Tensor` representing vectorized lower\n (or upper) triangular elements from `x`.\n \"\"\"\n\n with tf.name_scope(name or \"fill_triangular_inverse\"):\n x = tf.convert_to_tensor(value=x, name=\"x\")\n n = tf.compat.dimension_value(\n tensorshape_util.with_rank_at_least(x.shape, 2)[-1])\n if n is not None:\n n = np.int32(n)\n m = np.int32((n * (n + 1)) // 2)\n static_final_shape = x.shape[:-2].concatenate([m])\n else:\n n = tf.shape(input=x)[-1]\n m = (n * (n + 1)) // 2\n static_final_shape = tensorshape_util.with_rank_at_least(\n x.shape, 2)[:-2].concatenate([None])\n ndims = prefer_static_rank(x)\n if upper:\n initial_elements = x[..., 0, :]\n triangular_portion = x[..., 1:, :]\n else:\n initial_elements = tf.reverse(x[..., -1, :], axis=[ndims - 2])\n triangular_portion = x[..., :-1, :]\n rotated_triangular_portion = tf.reverse(\n tf.reverse(triangular_portion, axis=[ndims - 1]), axis=[ndims - 2])\n consolidated_matrix = triangular_portion + rotated_triangular_portion\n end_sequence = tf.reshape(\n consolidated_matrix,\n tf.concat([tf.shape(input=x)[:-2], [n * (n - 1)]], axis=0))\n y = tf.concat([initial_elements, end_sequence[..., :m - n]], axis=-1)\n tensorshape_util.set_shape(y, static_final_shape)\n return y"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\ncreate a matrix with values set above, below, and on the diagonal. Example: ```python tridiag(below=[1., 2., 3.], diag=[4., 5., 6., 7.], above=[8., 9., 10.]) # ==> array([[ 4., 8., 0., 0.], # [ 1., 5., 9., 0.], # [ 0., 2., 6., 10.], # [ 0., 0., 3., 7.]], dtype=float32) ``` Warning: This Op is intended for convenience, not efficiency. Args: below: `Tensor` of shape `[B1, ..., Bb, d-1]` corresponding to the below diagonal part. `None` is logically equivalent to `below = 0`. diag: `Tensor` of shape `[B1, ..., Bb, d]` corresponding to the diagonal part. `None` is logically equivalent to `diag = 0`. above: `Tensor` of shape `[B1, ..., Bb, d-1]` corresponding to the above diagonal part. `None` is logically equivalent to `above = 0`. name: Python `str`. The name to give this op. Returns: tridiag: `Tensor` with values set above, below and on the diagonal. Raises: ValueError: if all inputs are `None`.", "response": "def tridiag(below=None, diag=None, above=None, name=None):\n \"\"\"Creates a matrix with values set above, below, and on the diagonal.\n\n Example:\n\n ```python\n tridiag(below=[1., 2., 3.],\n diag=[4., 5., 6., 7.],\n above=[8., 9., 10.])\n # ==> array([[ 4., 8., 0., 0.],\n # [ 1., 5., 9., 0.],\n # [ 0., 2., 6., 10.],\n # [ 0., 0., 3., 7.]], dtype=float32)\n ```\n\n Warning: This Op is intended for convenience, not efficiency.\n\n Args:\n below: `Tensor` of shape `[B1, ..., Bb, d-1]` corresponding to the below\n diagonal part. `None` is logically equivalent to `below = 0`.\n diag: `Tensor` of shape `[B1, ..., Bb, d]` corresponding to the diagonal\n part. `None` is logically equivalent to `diag = 0`.\n above: `Tensor` of shape `[B1, ..., Bb, d-1]` corresponding to the above\n diagonal part. `None` is logically equivalent to `above = 0`.\n name: Python `str`. The name to give this op.\n\n Returns:\n tridiag: `Tensor` with values set above, below and on the diagonal.\n\n Raises:\n ValueError: if all inputs are `None`.\n \"\"\"\n\n def _pad(x):\n \"\"\"Prepends and appends a zero to every vector in a batch of vectors.\"\"\"\n shape = tf.concat([tf.shape(input=x)[:-1], [1]], axis=0)\n z = tf.zeros(shape, dtype=x.dtype)\n return tf.concat([z, x, z], axis=-1)\n\n def _add(*x):\n \"\"\"Adds list of Tensors, ignoring `None`.\"\"\"\n s = None\n for y in x:\n if y is None:\n continue\n elif s is None:\n s = y\n else:\n s += y\n if s is None:\n raise ValueError(\"Must specify at least one of `below`, `diag`, `above`.\")\n return s\n\n with tf.name_scope(name or \"tridiag\"):\n if below is not None:\n below = tf.convert_to_tensor(value=below, name=\"below\")\n below = tf.linalg.diag(_pad(below))[..., :-1, 1:]\n if diag is not None:\n diag = tf.convert_to_tensor(value=diag, name=\"diag\")\n diag = tf.linalg.diag(diag)\n if above is not None:\n above = tf.convert_to_tensor(value=above, name=\"above\")\n above = tf.linalg.diag(_pad(above))[..., 1:, :-1]\n # TODO(jvdillon): Consider using scatter_nd instead of creating three full\n # matrices.\n return _add(below, diag, above)"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nreducing the logx along the given axis and returns the corresponding logx.", "response": "def reduce_weighted_logsumexp(logx,\n w=None,\n axis=None,\n keep_dims=False,\n return_sign=False,\n name=None):\n \"\"\"Computes `log(abs(sum(weight * exp(elements across tensor dimensions))))`.\n\n If all weights `w` are known to be positive, it is more efficient to directly\n use `reduce_logsumexp`, i.e., `tf.reduce_logsumexp(logx + tf.log(w))` is more\n efficient than `du.reduce_weighted_logsumexp(logx, w)`.\n\n Reduces `input_tensor` along the dimensions given in `axis`.\n Unless `keep_dims` is true, the rank of the tensor is reduced by 1 for each\n entry in `axis`. If `keep_dims` is true, the reduced dimensions\n are retained with length 1.\n\n If `axis` has no entries, all dimensions are reduced, and a\n tensor with a single element is returned.\n\n This function is more numerically stable than log(sum(w * exp(input))). It\n avoids overflows caused by taking the exp of large inputs and underflows\n caused by taking the log of small inputs.\n\n For example:\n\n ```python\n x = tf.constant([[0., 0, 0],\n [0, 0, 0]])\n\n w = tf.constant([[-1., 1, 1],\n [1, 1, 1]])\n\n du.reduce_weighted_logsumexp(x, w)\n # ==> log(-1*1 + 1*1 + 1*1 + 1*1 + 1*1 + 1*1) = log(4)\n\n du.reduce_weighted_logsumexp(x, w, axis=0)\n # ==> [log(-1+1), log(1+1), log(1+1)]\n\n du.reduce_weighted_logsumexp(x, w, axis=1)\n # ==> [log(-1+1+1), log(1+1+1)]\n\n du.reduce_weighted_logsumexp(x, w, axis=1, keep_dims=True)\n # ==> [[log(-1+1+1)], [log(1+1+1)]]\n\n du.reduce_weighted_logsumexp(x, w, axis=[0, 1])\n # ==> log(-1+5)\n ```\n\n Args:\n logx: The tensor to reduce. Should have numeric type.\n w: The weight tensor. Should have numeric type identical to `logx`.\n axis: The dimensions to reduce. If `None` (the default), reduces all\n dimensions. Must be in the range `[-rank(input_tensor),\n rank(input_tensor))`.\n keep_dims: If true, retains reduced dimensions with length 1.\n return_sign: If `True`, returns the sign of the result.\n name: A name for the operation (optional).\n\n Returns:\n lswe: The `log(abs(sum(weight * exp(x))))` reduced tensor.\n sign: (Optional) The sign of `sum(weight * exp(x))`.\n \"\"\"\n with tf.name_scope(name or \"reduce_weighted_logsumexp\"):\n logx = tf.convert_to_tensor(value=logx, name=\"logx\")\n if w is None:\n lswe = tf.reduce_logsumexp(\n input_tensor=logx, axis=axis, keepdims=keep_dims)\n if return_sign:\n sgn = tf.ones_like(lswe)\n return lswe, sgn\n return lswe\n w = tf.convert_to_tensor(value=w, dtype=logx.dtype, name=\"w\")\n log_absw_x = logx + tf.math.log(tf.abs(w))\n max_log_absw_x = tf.reduce_max(\n input_tensor=log_absw_x, axis=axis, keepdims=True)\n # If the largest element is `-inf` or `inf` then we don't bother subtracting\n # off the max. We do this because otherwise we'd get `inf - inf = NaN`. That\n # this is ok follows from the fact that we're actually free to subtract any\n # value we like, so long as we add it back after taking the `log(sum(...))`.\n max_log_absw_x = tf.where(\n tf.math.is_inf(max_log_absw_x), tf.zeros_like(max_log_absw_x),\n max_log_absw_x)\n wx_over_max_absw_x = (tf.sign(w) * tf.exp(log_absw_x - max_log_absw_x))\n sum_wx_over_max_absw_x = tf.reduce_sum(\n input_tensor=wx_over_max_absw_x, axis=axis, keepdims=keep_dims)\n if not keep_dims:\n max_log_absw_x = tf.squeeze(max_log_absw_x, axis)\n sgn = tf.sign(sum_wx_over_max_absw_x)\n lswe = max_log_absw_x + tf.math.log(sgn * sum_wx_over_max_absw_x)\n if return_sign:\n return lswe, sgn\n return lswe"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef softplus_inverse(x, name=None):\n with tf.name_scope(name or \"softplus_inverse\"):\n x = tf.convert_to_tensor(value=x, name=\"x\")\n # We begin by deriving a more numerically stable softplus_inverse:\n # x = softplus(y) = Log[1 + exp{y}], (which means x > 0).\n # ==> exp{x} = 1 + exp{y} (1)\n # ==> y = Log[exp{x} - 1] (2)\n # = Log[(exp{x} - 1) / exp{x}] + Log[exp{x}]\n # = Log[(1 - exp{-x}) / 1] + Log[exp{x}]\n # = Log[1 - exp{-x}] + x (3)\n # (2) is the \"obvious\" inverse, but (3) is more stable than (2) for large x.\n # For small x (e.g. x = 1e-10), (3) will become -inf since 1 - exp{-x} will\n # be zero. To fix this, we use 1 - exp{-x} approx x for small x > 0.\n #\n # In addition to the numerically stable derivation above, we clamp\n # small/large values to be congruent with the logic in:\n # tensorflow/core/kernels/softplus_op.h\n #\n # Finally, we set the input to one whenever the input is too large or too\n # small. This ensures that no unchosen codepath is +/- inf. This is\n # necessary to ensure the gradient doesn't get NaNs. Recall that the\n # gradient of `where` behaves like `pred*pred_true + (1-pred)*pred_false`\n # thus an `inf` in an unselected path results in `0*inf=nan`. We are careful\n # to overwrite `x` with ones only when we will never actually use this\n # value. Note that we use ones and not zeros since `log(expm1(0.)) = -inf`.\n threshold = np.log(np.finfo(dtype_util.as_numpy_dtype(x.dtype)).eps) + 2.\n is_too_small = tf.less(x, np.exp(threshold))\n is_too_large = tf.greater(x, -threshold)\n too_small_value = tf.math.log(x)\n too_large_value = x\n # This `where` will ultimately be a NOP because we won't select this\n # codepath whenever we used the surrogate `ones_like`.\n x = tf.where(tf.logical_or(is_too_small, is_too_large), tf.ones_like(x), x)\n y = x + tf.math.log(-tf.math.expm1(-x)) # == log(expm1(x))\n return tf.where(is_too_small, too_small_value,\n tf.where(is_too_large, too_large_value, y))", "response": "Computes the inverse softplus."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nreturns the size of a specific dimension.", "response": "def dimension_size(x, axis):\n \"\"\"Returns the size of a specific dimension.\"\"\"\n # Since tf.gather isn't \"constant-in, constant-out\", we must first check the\n # static shape or fallback to dynamic shape.\n s = tf.compat.dimension_value(\n tensorshape_util.with_rank_at_least(x.shape, np.abs(axis))[axis])\n if s is not None:\n return s\n return tf.shape(input=x)[axis]"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nvalidate quadrature grid and probs or computes them as necessary.", "response": "def process_quadrature_grid_and_probs(quadrature_grid_and_probs,\n dtype,\n validate_args,\n name=None):\n \"\"\"Validates quadrature grid, probs or computes them as necessary.\n\n Args:\n quadrature_grid_and_probs: Python pair of `float`-like `Tensor`s\n representing the sample points and the corresponding (possibly\n normalized) weight. When `None`, defaults to:\n `np.polynomial.hermite.hermgauss(deg=8)`.\n dtype: The expected `dtype` of `grid` and `probs`.\n validate_args: Python `bool`, default `False`. When `True` distribution\n parameters are checked for validity despite possibly degrading runtime\n performance. When `False` invalid inputs may silently render incorrect\n outputs.\n name: Python `str` name prefixed to Ops created by this class.\n\n Returns:\n quadrature_grid_and_probs: Python pair of `float`-like `Tensor`s\n representing the sample points and the corresponding (possibly\n normalized) weight.\n\n Raises:\n ValueError: if `quadrature_grid_and_probs is not None` and\n `len(quadrature_grid_and_probs[0]) != len(quadrature_grid_and_probs[1])`\n \"\"\"\n with tf.name_scope(name or \"process_quadrature_grid_and_probs\"):\n if quadrature_grid_and_probs is None:\n grid, probs = np.polynomial.hermite.hermgauss(deg=8)\n grid = grid.astype(dtype_util.as_numpy_dtype(dtype))\n probs = probs.astype(dtype_util.as_numpy_dtype(dtype))\n probs /= np.linalg.norm(probs, ord=1, keepdims=True)\n grid = tf.convert_to_tensor(value=grid, name=\"grid\", dtype=dtype)\n probs = tf.convert_to_tensor(value=probs, name=\"probs\", dtype=dtype)\n return grid, probs\n\n grid, probs = tuple(quadrature_grid_and_probs)\n grid = tf.convert_to_tensor(value=grid, name=\"grid\", dtype=dtype)\n probs = tf.convert_to_tensor(\n value=probs, name=\"unnormalized_probs\", dtype=dtype)\n probs /= tf.norm(tensor=probs, ord=1, axis=-1, keepdims=True, name=\"probs\")\n\n def _static_event_size(x):\n \"\"\"Returns the static size of a specific dimension or `None`.\"\"\"\n return tf.compat.dimension_value(\n tensorshape_util.with_rank_at_least(x.shape, 1)[-1])\n\n m, n = _static_event_size(probs), _static_event_size(grid)\n if m is not None and n is not None:\n if m != n:\n raise ValueError(\"`quadrature_grid_and_probs` must be a `tuple` of \"\n \"same-length zero-th-dimension `Tensor`s \"\n \"(saw lengths {}, {})\".format(m, n))\n elif validate_args:\n assertions = [\n assert_util.assert_equal(\n dimension_size(probs, axis=-1),\n dimension_size(grid, axis=-1),\n message=(\"`quadrature_grid_and_probs` must be a `tuple` of \"\n \"same-length zero-th-dimension `Tensor`s\")),\n ]\n with tf.control_dependencies(assertions):\n grid = tf.identity(grid)\n probs = tf.identity(probs)\n return grid, probs"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef pad(x, axis, front=False, back=False, value=0, count=1, name=None):\n with tf.name_scope(name or \"pad\"):\n x = tf.convert_to_tensor(value=x, name=\"x\")\n value = tf.convert_to_tensor(value=value, dtype=x.dtype, name=\"value\")\n count = tf.convert_to_tensor(value=count, name=\"count\")\n if not dtype_util.is_integer(count.dtype):\n raise TypeError(\"`count.dtype` (`{}`) must be `int`-like.\".format(\n dtype_util.name(count.dtype)))\n if not front and not back:\n raise ValueError(\"At least one of `front`, `back` must be `True`.\")\n ndims = (\n tensorshape_util.rank(x.shape)\n if tensorshape_util.rank(x.shape) is not None else tf.rank(\n x, name=\"ndims\"))\n axis = tf.convert_to_tensor(value=axis, name=\"axis\")\n axis_ = tf.get_static_value(axis)\n if axis_ is not None:\n axis = axis_\n if axis < 0:\n axis = ndims + axis\n count_ = tf.get_static_value(count)\n if axis_ >= 0 or tensorshape_util.rank(x.shape) is not None:\n head = x.shape[:axis]\n mid_dim_value = tf.compat.dimension_value(x.shape[axis])\n if count_ is None or mid_dim_value is None:\n middle = tf.TensorShape(None)\n else:\n middle = tf.TensorShape(mid_dim_value + count_ * (front + back))\n tail = x.shape[axis + 1:]\n final_shape = head.concatenate(middle.concatenate(tail))\n else:\n final_shape = None\n else:\n axis = tf.where(axis < 0, ndims + axis, axis)\n final_shape = None\n x = tf.pad(\n tensor=x,\n paddings=tf.one_hot(\n indices=tf.stack([axis if front else -1, axis if back else -1]),\n depth=ndims,\n axis=0,\n on_value=count,\n dtype=tf.int32),\n constant_values=value)\n if final_shape is not None:\n tensorshape_util.set_shape(x, final_shape)\n return x", "response": "Pads value to the front and back of a Tensor dim count times."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef parent_frame_arguments():\n # All arguments and the names used for *varargs, and **kwargs\n arg_names, variable_arg_name, keyword_arg_name, local_vars = (\n tf_inspect._inspect.getargvalues( # pylint: disable=protected-access\n # Get the first frame of the caller of this method.\n tf_inspect._inspect.stack()[1][0])) # pylint: disable=protected-access\n\n # Remove the *varargs, and flatten the **kwargs. Both are\n # nested lists.\n local_vars.pop(variable_arg_name, {})\n keyword_args = local_vars.pop(keyword_arg_name, {})\n\n final_args = {}\n # Copy over arguments and their values. In general, local_vars\n # may contain more than just the arguments, since this method\n # can be called anywhere in a function.\n for arg_name in arg_names:\n final_args[arg_name] = local_vars.pop(arg_name)\n final_args.update(keyword_args)\n\n return final_args", "response": "Returns the arguments of the caller s function\n."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\ntransforms a 0 - D tensor to be 1 - D.", "response": "def expand_to_vector(x, tensor_name=None, op_name=None, validate_args=False):\n \"\"\"Transform a 0-D or 1-D `Tensor` to be 1-D.\n\n For user convenience, many parts of the TensorFlow Probability API accept\n inputs of rank 0 or 1 -- i.e., allowing an `event_shape` of `[5]` to be passed\n to the API as either `5` or `[5]`. This function can be used to transform\n such an argument to always be 1-D.\n\n NOTE: Python or NumPy values will be converted to `Tensor`s with standard type\n inference/conversion. In particular, an empty list or tuple will become an\n empty `Tensor` with dtype `float32`. Callers should convert values to\n `Tensor`s before calling this function if different behavior is desired\n (e.g. converting empty lists / other values to `Tensor`s with dtype `int32`).\n\n Args:\n x: A 0-D or 1-D `Tensor`.\n tensor_name: Python `str` name for `Tensor`s created by this function.\n op_name: Python `str` name for `Op`s created by this function.\n validate_args: Python `bool, default `False`. When `True`, arguments may be\n checked for validity at execution time, possibly degrading runtime\n performance. When `False`, invalid inputs may silently render incorrect\n outputs.\n Returns:\n vector: a 1-D `Tensor`.\n \"\"\"\n with tf.name_scope(op_name or \"expand_to_vector\"):\n x = tf.convert_to_tensor(value=x, name=\"x\")\n ndims = tensorshape_util.rank(x.shape)\n\n if ndims is None:\n # Maybe expand ndims from 0 to 1.\n if validate_args:\n x = with_dependencies([\n assert_util.assert_rank_at_most(\n x, 1, message=\"Input is neither scalar nor vector.\")\n ], x)\n ndims = tf.rank(x)\n expanded_shape = pick_vector(\n tf.equal(ndims, 0), np.array([1], dtype=np.int32), tf.shape(input=x))\n return tf.reshape(x, expanded_shape)\n\n elif ndims == 0:\n # Definitely expand ndims from 0 to 1.\n x_const = tf.get_static_value(x)\n if x_const is not None:\n return tf.convert_to_tensor(\n value=dtype_util.as_numpy_dtype(x.dtype)([x_const]),\n name=tensor_name)\n\n else:\n return tf.reshape(x, [1])\n\n elif ndims != 1:\n raise ValueError(\"Input is neither scalar nor vector.\")\n\n # ndims == 1\n return x"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef with_dependencies(dependencies, output_tensor, name=None):\n if tf.executing_eagerly():\n return output_tensor\n with tf.name_scope(name or \"control_dependency\") as name:\n with tf.control_dependencies(d for d in dependencies if d is not None):\n output_tensor = tf.convert_to_tensor(value=output_tensor)\n if isinstance(output_tensor, tf.Tensor):\n return tf.identity(output_tensor, name=name)\n else:\n return tf.IndexedSlices(\n tf.identity(output_tensor.values, name=name),\n output_tensor.indices,\n output_tensor.dense_shape)", "response": "Produces the content of output_tensor only after all operations in dependencies have run."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\ncheck that the rightmost transposed dims are valid.", "response": "def _maybe_validate_rightmost_transposed_ndims(\n rightmost_transposed_ndims, validate_args, name=None):\n \"\"\"Checks that `rightmost_transposed_ndims` is valid.\"\"\"\n with tf.name_scope(name or 'maybe_validate_rightmost_transposed_ndims'):\n assertions = []\n if not dtype_util.is_integer(rightmost_transposed_ndims.dtype):\n raise TypeError('`rightmost_transposed_ndims` must be integer type.')\n\n if tensorshape_util.rank(rightmost_transposed_ndims.shape) is not None:\n if tensorshape_util.rank(rightmost_transposed_ndims.shape) != 0:\n raise ValueError('`rightmost_transposed_ndims` must be a scalar, '\n 'saw rank: {}.'.format(\n tensorshape_util.rank(\n rightmost_transposed_ndims.shape)))\n elif validate_args:\n assertions += [assert_util.assert_rank(rightmost_transposed_ndims, 0)]\n\n rightmost_transposed_ndims_ = tf.get_static_value(\n rightmost_transposed_ndims)\n msg = '`rightmost_transposed_ndims` must be non-negative.'\n if rightmost_transposed_ndims_ is not None:\n if rightmost_transposed_ndims_ < 0:\n raise ValueError(msg[:-1] + ', saw: {}.'.format(\n rightmost_transposed_ndims_))\n elif validate_args:\n assertions += [\n assert_util.assert_non_negative(\n rightmost_transposed_ndims, message=msg)\n ]\n\n return assertions"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef _maybe_validate_perm(perm, validate_args, name=None):\n with tf.name_scope(name or 'maybe_validate_perm'):\n assertions = []\n if not dtype_util.is_integer(perm.dtype):\n raise TypeError('`perm` must be integer type')\n\n msg = '`perm` must be a vector.'\n if tensorshape_util.rank(perm.shape) is not None:\n if tensorshape_util.rank(perm.shape) != 1:\n raise ValueError(\n msg[:-1] +\n ', saw rank: {}.'.format(tensorshape_util.rank(perm.shape)))\n elif validate_args:\n assertions += [assert_util.assert_rank(perm, 1, message=msg)]\n\n perm_ = tf.get_static_value(perm)\n msg = '`perm` must be a valid permutation vector.'\n if perm_ is not None:\n if not np.all(np.arange(np.size(perm_)) == np.sort(perm_)):\n raise ValueError(msg[:-1] + ', saw: {}.'.format(perm_))\n elif validate_args:\n assertions += [\n assert_util.assert_equal(\n tf.sort(perm), tf.range(tf.size(input=perm)), message=msg)\n ]\n\n return assertions", "response": "Checks that perm is valid."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef concatenate(x, other):\n return type(x)(tf.TensorShape(x).concatenate(other))", "response": "Returns the concatenation of the dimension in x and other."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef constant_value_as_shape(tensor): # pylint: disable=invalid-name\n shape = tf.get_static_value(tensor)\n if shape is not None:\n return [None if dim == -1 else dim for dim in shape]\n return tensor_util.constant_value_as_shape(tensor)", "response": "A version of constant_value that returns a TensorShape."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef dims(x):\n if isinstance(x, tf.TensorShape):\n return x.dims\n r = tf.TensorShape(x).dims\n return None if r is None else list(map(tf.compat.dimension_value, r))", "response": "Returns a list of dimensions for each a\n ."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nreturns a shape combining the information in x and other.", "response": "def merge_with(x, other):\n \"\"\"Returns a shape combining the information in `x` and `other`.\n\n The dimensions in `x` and `other` are merged elementwise, according to the\n rules defined for `tf.Dimension.merge_with()`.\n\n For more details, see `help(tf.TensorShape.merge_with)`.\n\n Args:\n x: object representing a shape; convertible to `tf.TensorShape`.\n other: object representing a shape; convertible to `tf.TensorShape`.\n\n Returns:\n merged_shape: shape having `type(x)` containing the combined information of\n `x` and `other`.\n\n Raises:\n ValueError: If `x` and `other` are not compatible.\n \"\"\"\n return type(x)(tf.TensorShape(x).merge_with(other))"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef with_rank_at_least(x, rank): # pylint: disable=redefined-outer-name\n return type(x)(tf.TensorShape(x).with_rank_at_least(rank))", "response": "Returns a shape based on x with at least the given rank."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef _check_equal_shape(name,\n static_shape,\n dynamic_shape,\n static_target_shape,\n dynamic_target_shape=None):\n \"\"\"Check that source and target shape match, statically if possible.\"\"\"\n\n static_target_shape = tf.TensorShape(static_target_shape)\n if tensorshape_util.is_fully_defined(\n static_shape) and tensorshape_util.is_fully_defined(static_target_shape):\n if static_shape != static_target_shape:\n raise ValueError(\"{}: required shape {} but found {}\".\n format(name, static_target_shape, static_shape))\n return None\n else:\n if dynamic_target_shape is None:\n if tensorshape_util.is_fully_defined(static_target_shape):\n dynamic_target_shape = tensorshape_util.as_list(static_target_shape)\n else:\n raise ValueError(\"{}: cannot infer target shape: no dynamic shape \"\n \"specified and static shape {} is not fully defined\".\n format(name, static_target_shape))\n return assert_util.assert_equal(\n dynamic_shape,\n dynamic_target_shape,\n message=(\"{}: required shape {}\".format(name, static_target_shape)))", "response": "Check that source and target shape match statically if possible."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\naugment a sample shape to broadcast batch dimensions.", "response": "def _augment_sample_shape(partial_batch_dist,\n full_sample_and_batch_shape,\n validate_args=False):\n \"\"\"Augment a sample shape to broadcast batch dimensions.\n\n Computes an augmented sample shape, so that any batch dimensions not\n part of the distribution `partial_batch_dist` are treated as identical\n distributions.\n\n # partial_batch_dist.batch_shape = [ 7]\n # full_sample_and_batch_shape = [3, 4, 7]\n # => return an augmented sample shape of [3, 4] so that\n # partial_batch_dist.sample(augmented_sample_shape) has combined\n # sample and batch shape of [3, 4, 7].\n\n Args:\n partial_batch_dist: `tfd.Distribution` instance with batch shape a\n prefix of `full_sample_and_batch_shape`.\n full_sample_and_batch_shape: a Tensor or Tensor-like shape.\n validate_args: if True, check for shape errors at runtime.\n Returns:\n augmented_sample_shape: sample shape such that\n `partial_batch_dist.sample(augmented_sample_shape)` has combined\n sample and batch shape of `full_sample_and_batch_shape`.\n\n Raises:\n ValueError: if `partial_batch_dist.batch_shape` has more dimensions than\n `full_sample_and_batch_shape`.\n NotImplementedError: if broadcasting would be required to make\n `partial_batch_dist.batch_shape` into a prefix of\n `full_sample_and_batch_shape` .\n \"\"\"\n full_ndims = distribution_util.prefer_static_shape(\n full_sample_and_batch_shape)[0]\n partial_batch_ndims = (\n tensorshape_util.rank(partial_batch_dist.batch_shape) # pylint: disable=g-long-ternary\n if tensorshape_util.rank(partial_batch_dist.batch_shape) is not None\n else distribution_util.prefer_static_shape(\n partial_batch_dist.batch_shape_tensor())[0])\n\n num_broadcast_dims = full_ndims - partial_batch_ndims\n\n expected_partial_batch_shape = (\n full_sample_and_batch_shape[num_broadcast_dims:])\n expected_partial_batch_shape_static = tf.get_static_value(\n full_sample_and_batch_shape[num_broadcast_dims:])\n\n # Raise errors statically if possible.\n num_broadcast_dims_static = tf.get_static_value(num_broadcast_dims)\n if num_broadcast_dims_static is not None:\n if num_broadcast_dims_static < 0:\n raise ValueError(\"Cannot broadcast distribution {} batch shape to \"\n \"target batch shape with fewer dimensions\"\n .format(partial_batch_dist))\n if (expected_partial_batch_shape_static is not None and\n tensorshape_util.is_fully_defined(partial_batch_dist.batch_shape)):\n if (partial_batch_dist.batch_shape and\n any(expected_partial_batch_shape_static != tensorshape_util.as_list(\n partial_batch_dist.batch_shape))):\n raise NotImplementedError(\"Broadcasting is not supported; \"\n \"unexpected batch shape \"\n \"(expected {}, saw {}).\".format(\n expected_partial_batch_shape_static,\n partial_batch_dist.batch_shape\n ))\n runtime_assertions = []\n if validate_args:\n runtime_assertions.append(\n assert_util.assert_greater_equal(\n tf.convert_to_tensor(value=num_broadcast_dims, dtype=tf.int32),\n tf.zeros((), dtype=tf.int32),\n message=(\"Cannot broadcast distribution {} batch shape to \"\n \"target batch shape with fewer dimensions.\".format(\n partial_batch_dist))))\n runtime_assertions.append(\n assert_util.assert_equal(\n expected_partial_batch_shape,\n partial_batch_dist.batch_shape_tensor(),\n message=(\"Broadcasting is not supported; \"\n \"unexpected batch shape.\"),\n name=\"assert_batch_shape_same\"))\n\n with tf.control_dependencies(runtime_assertions):\n return full_sample_and_batch_shape[:num_broadcast_dims]"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef build_backward_pass_step(get_transition_matrix_for_timestep):\n\n def backward_pass_step(state,\n filtered_parameters):\n \"\"\"Run a single step of backward smoothing.\"\"\"\n\n (filtered_mean, filtered_cov,\n predicted_mean, predicted_cov) = filtered_parameters\n transition_matrix = get_transition_matrix_for_timestep(state.timestep)\n\n next_posterior_mean = state.backward_mean\n next_posterior_cov = state.backward_cov\n\n posterior_mean, posterior_cov = backward_smoothing_update(\n filtered_mean,\n filtered_cov,\n predicted_mean,\n predicted_cov,\n next_posterior_mean,\n next_posterior_cov,\n transition_matrix)\n\n return BackwardPassState(backward_mean=posterior_mean,\n backward_cov=posterior_cov,\n timestep=state.timestep-1)\n\n return backward_pass_step", "response": "Builds a function that performs one step for backward smoothing."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef backward_smoothing_update(filtered_mean,\n filtered_cov,\n predicted_mean,\n predicted_cov,\n next_posterior_mean,\n next_posterior_cov,\n transition_matrix):\n \"\"\"Backward update for a Kalman smoother.\n\n Give the `filtered_mean` mu(t | t), `filtered_cov` sigma(t | t),\n `predicted_mean` mu(t+1 | t) and `predicted_cov` sigma(t+1 | t),\n as returns from the `forward_filter` function, as well as\n `next_posterior_mean` mu(t+1 | 1:T) and `next_posterior_cov` sigma(t+1 | 1:T),\n if the `transition_matrix` of states from time t to time t+1\n is given as A(t+1), the 1 step backward smoothed distribution parameter\n could be calculated as:\n p(z(t) | Obs(1:T)) = N( mu(t | 1:T), sigma(t | 1:T)),\n mu(t | 1:T) = mu(t | t) + J(t) * (mu(t+1 | 1:T) - mu(t+1 | t)),\n sigma(t | 1:T) = sigma(t | t)\n + J(t) * (sigma(t+1 | 1:T) - sigma(t+1 | t) * J(t)',\n J(t) = sigma(t | t) * A(t+1)' / sigma(t+1 | t),\n where all the multiplications are matrix multiplication, and `/` is\n the matrix inverse. J(t) is the backward Kalman gain matrix.\n\n The algorithm can be intialized from mu(T | 1:T) and sigma(T | 1:T),\n which are the last step parameters returned by forward_filter.\n\n\n Args:\n filtered_mean: `Tensor` with event shape `[latent_size, 1]` and\n batch shape `B`, containing mu(t | t).\n filtered_cov: `Tensor` with event shape `[latent_size, latent_size]` and\n batch shape `B`, containing sigma(t | t).\n predicted_mean: `Tensor` with event shape `[latent_size, 1]` and\n batch shape `B`, containing mu(t+1 | t).\n predicted_cov: `Tensor` with event shape `[latent_size, latent_size]` and\n batch shape `B`, containing sigma(t+1 | t).\n next_posterior_mean: `Tensor` with event shape `[latent_size, 1]` and\n batch shape `B`, containing mu(t+1 | 1:T).\n next_posterior_cov: `Tensor` with event shape `[latent_size, latent_size]`\n and batch shape `B`, containing sigma(t+1 | 1:T).\n transition_matrix: `LinearOperator` with shape\n `[latent_size, latent_size]` and batch shape broadcastable\n to `B`.\n\n Returns:\n posterior_mean: `Tensor` with event shape `[latent_size, 1]` and\n batch shape `B`, containing mu(t | 1:T).\n posterior_cov: `Tensor` with event shape `[latent_size, latent_size]` and\n batch shape `B`, containing sigma(t | 1:T).\n \"\"\"\n # Compute backward Kalman gain:\n # J = F * T' * P^{-1}\n # Since both F(iltered) and P(redictive) are cov matrices,\n # thus self-adjoint, we can take the transpose.\n # computation:\n # = (P^{-1} * T * F)'\n # = (P^{-1} * tmp_gain_cov) '\n # = (P \\ tmp_gain_cov)'\n tmp_gain_cov = transition_matrix.matmul(filtered_cov)\n predicted_cov_chol = tf.linalg.cholesky(predicted_cov)\n gain_transpose = tf.linalg.cholesky_solve(predicted_cov_chol, tmp_gain_cov)\n\n posterior_mean = (filtered_mean +\n tf.linalg.matmul(gain_transpose,\n next_posterior_mean - predicted_mean,\n adjoint_a=True))\n posterior_cov = (\n filtered_cov +\n tf.linalg.matmul(gain_transpose,\n tf.linalg.matmul(\n next_posterior_cov - predicted_cov, gain_transpose),\n adjoint_a=True))\n\n return (posterior_mean, posterior_cov)", "response": "This function is used to update the backward smoothing of the Kalman distribution."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef build_kalman_filter_step(get_transition_matrix_for_timestep,\n get_transition_noise_for_timestep,\n get_observation_matrix_for_timestep,\n get_observation_noise_for_timestep):\n \"\"\"Build a callable that performs one step of Kalman filtering.\n\n Args:\n get_transition_matrix_for_timestep: callable taking a timestep\n as an integer `Tensor` argument, and returning a `LinearOperator`\n of shape `[latent_size, latent_size]`.\n get_transition_noise_for_timestep: callable taking a timestep as\n an integer `Tensor` argument, and returning a\n `MultivariateNormalLinearOperator` of event shape\n `[latent_size]`.\n get_observation_matrix_for_timestep: callable taking a timestep\n as an integer `Tensor` argument, and returning a `LinearOperator`\n of shape `[observation_size, observation_size]`.\n get_observation_noise_for_timestep: callable taking a timestep as\n an integer `Tensor` argument, and returning a\n `MultivariateNormalLinearOperator` of event shape\n `[observation_size]`.\n\n Returns:\n kalman_filter_step: a callable that updates a KalmanFilterState\n from timestep `t-1` to `t`.\n \"\"\"\n\n def kalman_filter_step(state, elems_t):\n \"\"\"Run a single step of Kalman filtering.\n\n Args:\n state: A `KalmanFilterState` object representing the previous\n filter state at time `t-1`.\n elems_t: A tuple of Tensors `(x_t, mask_t)`, or a `Tensor` `x_t`.\n `x_t` is a `Tensor` with rightmost shape dimensions\n `[observation_size, 1]` representing the vector observed at time `t`,\n and `mask_t` is a `Tensor` with rightmost dimensions`[1, 1]`\n representing the observation mask at time `t`. Both `x_t` and `mask_t`\n may have batch dimensions, which must be compatible with the batch\n dimensions of `state.predicted_mean` and `state.predictived_cov`\n respectively. If `mask_t` is not provided, it is assumed to be `None`.\n\n Returns:\n new_state: A `KalmanFilterState` object representing the new\n filter state at time `t`.\n \"\"\"\n\n if isinstance(elems_t, tuple):\n x_t, mask_t = elems_t\n else:\n x_t = elems_t\n mask_t = None\n\n observation_matrix = get_observation_matrix_for_timestep(state.timestep)\n observation_noise = get_observation_noise_for_timestep(state.timestep)\n if mask_t is not None:\n # Before running the update, fill in masked observations using the prior\n # expectation. The precise filled value shouldn't matter since updates\n # from masked elements will not be selected below, but we need to ensure\n # that any results we incidently compute on masked values are at least\n # finite (not inf or NaN) so that they don't screw up gradient propagation\n # through `tf.where`, as described in\n # https://github.com/tensorflow/tensorflow/issues/2540.\n # We fill with the prior expectation because any fixed value such as zero\n # might be arbitrarily unlikely under the prior, leading to overflow in\n # the updates, but the prior expectation should always be a\n # 'reasonable' observation.\n x_expected = _propagate_mean(state.predicted_mean,\n observation_matrix,\n observation_noise) * tf.ones_like(x_t)\n x_t = tf.where(\n tf.broadcast_to(mask_t, tf.shape(input=x_expected)), x_expected,\n tf.broadcast_to(x_t, tf.shape(input=x_expected)))\n\n # Given predicted mean u_{t|t-1} and covariance P_{t|t-1} from the\n # previous step, incorporate the observation x_t, producing the\n # filtered mean u_t and covariance P_t.\n (filtered_mean,\n filtered_cov,\n observation_dist) = linear_gaussian_update(\n state.predicted_mean, state.predicted_cov,\n observation_matrix, observation_noise,\n x_t)\n\n # Compute the marginal likelihood p(x_{t} | x_{:t-1}) for this\n # observation.\n log_marginal_likelihood = observation_dist.log_prob(x_t[..., 0])\n\n if mask_t is not None:\n filtered_mean = tf.where(\n tf.broadcast_to(mask_t, tf.shape(input=filtered_mean)),\n state.predicted_mean, filtered_mean)\n filtered_cov = tf.where(\n tf.broadcast_to(mask_t, tf.shape(input=filtered_cov)),\n state.predicted_cov, filtered_cov)\n log_marginal_likelihood = tf.where(\n tf.broadcast_to(mask_t[..., 0, 0],\n tf.shape(input=log_marginal_likelihood)),\n tf.zeros_like(log_marginal_likelihood),\n log_marginal_likelihood)\n\n # Run the filtered posterior through the transition\n # model to predict the next time step:\n # u_{t|t-1} = F_t u_{t-1} + b_t\n # P_{t|t-1} = F_t P_{t-1} F_t' + Q_t\n predicted_mean, predicted_cov = kalman_transition(\n filtered_mean,\n filtered_cov,\n get_transition_matrix_for_timestep(state.timestep),\n get_transition_noise_for_timestep(state.timestep))\n\n return KalmanFilterState(\n filtered_mean, filtered_cov,\n predicted_mean, predicted_cov,\n observation_dist.mean()[..., tf.newaxis],\n observation_dist.covariance(),\n log_marginal_likelihood,\n state.timestep+1)\n\n return kalman_filter_step", "response": "Build a Kalman filter step."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nconjugating update for a linear Gaussian model.", "response": "def linear_gaussian_update(\n prior_mean, prior_cov, observation_matrix, observation_noise, x_observed):\n \"\"\"Conjugate update for a linear Gaussian model.\n\n Given a normal prior on a latent variable `z`,\n `p(z) = N(prior_mean, prior_cov) = N(u, P)`,\n for which we observe a linear Gaussian transformation `x`,\n `p(x|z) = N(H * z + c, R)`,\n the posterior is also normal:\n `p(z|x) = N(u*, P*)`.\n\n We can write this update as\n x_expected = H * u + c # pushforward prior mean\n S = R + H * P * H' # pushforward prior cov\n K = P * H' * S^{-1} # optimal Kalman gain\n u* = u + K * (x_observed - x_expected) # posterior mean\n P* = (I - K * H) * P (I - K * H)' + K * R * K' # posterior cov\n (see, e.g., https://en.wikipedia.org/wiki/Kalman_filter#Update)\n\n Args:\n prior_mean: `Tensor` with event shape `[latent_size, 1]` and\n potential batch shape `B = [b1, ..., b_n]`.\n prior_cov: `Tensor` with event shape `[latent_size, latent_size]`\n and batch shape `B` (matching `prior_mean`).\n observation_matrix: `LinearOperator` with shape\n `[observation_size, latent_size]` and batch shape broadcastable\n to `B`.\n observation_noise: potentially-batched\n `MultivariateNormalLinearOperator` instance with event shape\n `[observation_size]` and batch shape broadcastable to `B`.\n x_observed: potentially batched `Tensor` with event shape\n `[observation_size, 1]` and batch shape `B`.\n\n Returns:\n posterior_mean: `Tensor` with event shape `[latent_size, 1]` and\n batch shape `B`.\n posterior_cov: `Tensor` with event shape `[latent_size,\n latent_size]` and batch shape `B`.\n predictive_dist: the prior predictive distribution `p(x|z)`,\n as a `Distribution` instance with event\n shape `[observation_size]` and batch shape `B`. This will\n typically be `tfd.MultivariateNormalTriL`, but when\n `observation_size=1` we return a `tfd.Independent(tfd.Normal)`\n instance as an optimization.\n \"\"\"\n\n # If observations are scalar, we can avoid some matrix ops.\n observation_size_is_static_and_scalar = (\n tf.compat.dimension_value(observation_matrix.shape[-2]) == 1)\n\n # Push the predicted mean for the latent state through the\n # observation model\n x_expected = _propagate_mean(prior_mean,\n observation_matrix,\n observation_noise)\n\n # Push the predictive covariance of the latent state through the\n # observation model:\n # S = R + H * P * H'.\n # We use a temporary variable for H * P,\n # reused below to compute Kalman gain.\n tmp_obs_cov = observation_matrix.matmul(prior_cov)\n predicted_obs_cov = (\n observation_matrix.matmul(tmp_obs_cov, adjoint_arg=True)\n + observation_noise.covariance())\n\n # Compute optimal Kalman gain:\n # K = P * H' * S^{-1}\n # Since both S and P are cov matrices, thus symmetric,\n # we can take the transpose and reuse our previous\n # computation:\n # = (S^{-1} * H * P)'\n # = (S^{-1} * tmp_obs_cov) '\n # = (S \\ tmp_obs_cov)'\n if observation_size_is_static_and_scalar:\n gain_transpose = tmp_obs_cov/predicted_obs_cov\n else:\n predicted_obs_cov_chol = tf.linalg.cholesky(predicted_obs_cov)\n gain_transpose = tf.linalg.cholesky_solve(predicted_obs_cov_chol,\n tmp_obs_cov)\n\n # Compute the posterior mean, incorporating the observation.\n # u* = u + K (x_observed - x_expected)\n posterior_mean = (prior_mean +\n tf.linalg.matmul(gain_transpose, x_observed - x_expected,\n adjoint_a=True))\n\n # For the posterior covariance, we could use the simple update\n # P* = P - K * H * P\n # but this is prone to numerical issues because it subtracts a\n # value from a PSD matrix. We choose instead to use the more\n # expensive Jordan form update\n # P* = (I - K H) * P * (I - K H)' + K R K'\n # which always produces a PSD result. This uses\n # tmp_term = (I - K * H)'\n # as an intermediate quantity.\n tmp_term = -observation_matrix.matmul(gain_transpose, adjoint=True) # -K * H\n tmp_term = tf.linalg.set_diag(tmp_term, tf.linalg.diag_part(tmp_term) + 1)\n posterior_cov = (\n tf.linalg.matmul(\n tmp_term, tf.linalg.matmul(prior_cov, tmp_term), adjoint_a=True)\n + tf.linalg.matmul(gain_transpose,\n tf.linalg.matmul(\n observation_noise.covariance(), gain_transpose),\n adjoint_a=True))\n\n if observation_size_is_static_and_scalar:\n # A plain Normal would have event shape `[]`; wrapping with Independent\n # ensures `event_shape=[1]` as required.\n predictive_dist = independent.Independent(\n normal.Normal(loc=x_expected[..., 0],\n scale=tf.sqrt(predicted_obs_cov[..., 0])),\n reinterpreted_batch_ndims=1)\n\n # Minor hack to define the covariance, so that `predictive_dist` can pass as\n # an MVNTriL-like object.\n predictive_dist.covariance = lambda: predicted_obs_cov\n else:\n predictive_dist = mvn_tril.MultivariateNormalTriL(\n loc=x_expected[..., 0],\n scale_tril=predicted_obs_cov_chol)\n\n return posterior_mean, posterior_cov, predictive_dist"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef kalman_transition(filtered_mean, filtered_cov,\n transition_matrix, transition_noise):\n \"\"\"Propagate a filtered distribution through a transition model.\"\"\"\n\n predicted_mean = _propagate_mean(filtered_mean,\n transition_matrix,\n transition_noise)\n predicted_cov = _propagate_cov(filtered_cov,\n transition_matrix,\n transition_noise)\n return predicted_mean, predicted_cov", "response": "Propagate a filtered distribution through a transition model."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef build_kalman_mean_step(get_transition_matrix_for_timestep,\n get_transition_noise_for_timestep,\n get_observation_matrix_for_timestep,\n get_observation_noise_for_timestep):\n \"\"\"Build a callable that performs one step of Kalman mean recursion.\n\n Args:\n get_transition_matrix_for_timestep: callable taking a timestep\n as an integer `Tensor` argument, and returning a `LinearOperator`\n of shape `[latent_size, latent_size]`.\n get_transition_noise_for_timestep: callable taking a timestep as\n an integer `Tensor` argument, and returning a\n `MultivariateNormalLinearOperator` of event shape\n `[latent_size]`.\n get_observation_matrix_for_timestep: callable taking a timestep\n as an integer `Tensor` argument, and returning a `LinearOperator`\n of shape `[observation_size, observation_size]`.\n get_observation_noise_for_timestep: callable taking a timestep as\n an integer `Tensor` argument, and returning a\n `MultivariateNormalLinearOperator` of event shape\n `[observation_size]`.\n\n Returns:\n kalman_mean_step: a callable that computes latent state and\n observation means at time `t`, given latent mean at time `t-1`.\n \"\"\"\n\n def mean_step(previous_means, t):\n \"\"\"Single step of prior mean recursion.\"\"\"\n previous_latent_mean, _ = previous_means\n\n latent_mean = _propagate_mean(previous_latent_mean,\n get_transition_matrix_for_timestep(t - 1),\n get_transition_noise_for_timestep(t - 1))\n observation_mean = _propagate_mean(latent_mean,\n get_observation_matrix_for_timestep(t),\n get_observation_noise_for_timestep(t))\n return (latent_mean, observation_mean)\n\n return mean_step", "response": "Builds a kalman mean step."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nbuilding a callable that computes the latent state and observation covariance at a given timestep.", "response": "def build_kalman_cov_step(get_transition_matrix_for_timestep,\n get_transition_noise_for_timestep,\n get_observation_matrix_for_timestep,\n get_observation_noise_for_timestep):\n \"\"\"Build a callable for one step of Kalman covariance recursion.\n\n Args:\n get_transition_matrix_for_timestep: callable taking a timestep\n as an integer `Tensor` argument, and returning a `LinearOperator`\n of shape `[latent_size, latent_size]`.\n get_transition_noise_for_timestep: callable taking a timestep as\n an integer `Tensor` argument, and returning a\n `MultivariateNormalLinearOperator` of event shape\n `[latent_size]`.\n get_observation_matrix_for_timestep: callable taking a timestep\n as an integer `Tensor` argument, and returning a `LinearOperator`\n of shape `[observation_size, observation_size]`.\n get_observation_noise_for_timestep: callable taking a timestep as\n an integer `Tensor` argument, and returning a\n `MultivariateNormalLinearOperator` of event shape\n `[observation_size]`.\n\n Returns:\n cov_step: a callable that computes latent state and observation\n covariance at time `t`, given latent covariance at time `t-1`.\n \"\"\"\n\n def cov_step(previous_covs, t):\n \"\"\"Single step of prior covariance recursion.\"\"\"\n previous_latent_cov, _ = previous_covs\n\n latent_cov = _propagate_cov(\n previous_latent_cov,\n get_transition_matrix_for_timestep(t - 1),\n get_transition_noise_for_timestep(t - 1))\n observation_cov = _propagate_cov(\n latent_cov,\n get_observation_matrix_for_timestep(t),\n get_observation_noise_for_timestep(t))\n\n return (latent_cov, observation_cov)\n\n return cov_step"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef build_kalman_sample_step(get_transition_matrix_for_timestep,\n get_transition_noise_for_timestep,\n get_observation_matrix_for_timestep,\n get_observation_noise_for_timestep,\n full_sample_and_batch_shape,\n stream,\n validate_args=False):\n \"\"\"Build a callable for one step of Kalman sampling recursion.\n\n Args:\n get_transition_matrix_for_timestep: callable taking a timestep\n as an integer `Tensor` argument, and returning a `LinearOperator`\n of shape `[latent_size, latent_size]`.\n get_transition_noise_for_timestep: callable taking a timestep as\n an integer `Tensor` argument, and returning a\n `MultivariateNormalLinearOperator` of event shape\n `[latent_size]`.\n get_observation_matrix_for_timestep: callable taking a timestep\n as an integer `Tensor` argument, and returning a `LinearOperator`\n of shape `[observation_size, observation_size]`.\n get_observation_noise_for_timestep: callable taking a timestep as\n an integer `Tensor` argument, and returning a\n `MultivariateNormalLinearOperator` of event shape\n `[observation_size]`.\n full_sample_and_batch_shape: Desired sample and batch shape of the\n returned samples, concatenated in a single `Tensor`.\n stream: `tfd.SeedStream` instance used to generate a\n sequence of random seeds.\n validate_args: if True, perform error checking at runtime.\n\n Returns:\n sample_step: a callable that samples the latent state and\n observation at time `t`, given latent state at time `t-1`.\n \"\"\"\n\n def sample_step(sampled_prev, t):\n \"\"\"Sample values for a single timestep.\"\"\"\n latent_prev, _ = sampled_prev\n\n transition_matrix = get_transition_matrix_for_timestep(t - 1)\n transition_noise = get_transition_noise_for_timestep(t - 1)\n\n latent_pred = transition_matrix.matmul(latent_prev)\n latent_sampled = latent_pred + transition_noise.sample(\n sample_shape=_augment_sample_shape(\n transition_noise,\n full_sample_and_batch_shape,\n validate_args),\n seed=stream())[..., tf.newaxis]\n\n observation_matrix = get_observation_matrix_for_timestep(t)\n observation_noise = get_observation_noise_for_timestep(t)\n\n observation_pred = observation_matrix.matmul(latent_sampled)\n observation_sampled = observation_pred + observation_noise.sample(\n sample_shape=_augment_sample_shape(\n observation_noise,\n full_sample_and_batch_shape,\n validate_args),\n seed=stream())[..., tf.newaxis]\n\n return (latent_sampled, observation_sampled)\n\n return sample_step", "response": "Builds a callable that samples the latent state and observation at a single timestep."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef build_pushforward_latents_step(get_observation_matrix_for_timestep,\n get_observation_noise_for_timestep):\n \"\"\"Build a callable to push latent means/covs to observed means/covs.\n\n Args:\n get_observation_matrix_for_timestep: callable taking a timestep\n as an integer `Tensor` argument, and returning a `LinearOperator`\n of shape `[observation_size, observation_size]`.\n get_observation_noise_for_timestep: callable taking a timestep as\n an integer `Tensor` argument, and returning a\n `MultivariateNormalLinearOperator` of event shape\n `[observation_size]`.\n\n Returns:\n pushforward_latents_step: a callable that computes the observation mean and\n covariance at time `t`, given latent mean and covariance at time `t`.\n \"\"\"\n\n def pushforward_latents_step(_, latent_t_mean_cov):\n \"\"\"Loop body fn to pushforward latents to observations at a time step.\"\"\"\n t, latent_mean, latent_cov = latent_t_mean_cov\n\n observation_matrix = get_observation_matrix_for_timestep(t)\n observation_noise = get_observation_noise_for_timestep(t)\n observation_mean = _propagate_mean(latent_mean,\n observation_matrix,\n observation_noise)\n observation_cov = _propagate_cov(latent_cov,\n observation_matrix,\n observation_noise)\n\n return (observation_mean, observation_cov)\n\n return pushforward_latents_step", "response": "Builds a function that computes the observation mean and covariance at a time step."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef _propagate_mean(mean, linop, dist):\n return linop.matmul(mean) + dist.mean()[..., tf.newaxis]", "response": "Propagate a mean through linear Gaussian transformation."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef _propagate_cov(cov, linop, dist):\n # For linop A and input cov P, returns `A P A' + dist.cov()`\n return linop.matmul(linop.matmul(cov), adjoint_arg=True) + dist.covariance()", "response": "Propagate covariance through linear Gaussian transformation."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nrunning the backward pass in Kalman smoother. The backward smoothing is using Rauch, Tung and Striebel smoother as as discussed in section 18.3.2 of Kevin P. Murphy, 2012, Machine Learning: A Probabilistic Perspective, The MIT Press. The inputs are returned by `forward_filter` function. Args: filtered_means: Means of the per-timestep filtered marginal distributions p(z_t | x_{:t}), as a Tensor of shape `sample_shape(x) + batch_shape + [num_timesteps, latent_size]`. filtered_covs: Covariances of the per-timestep filtered marginal distributions p(z_t | x_{:t}), as a Tensor of shape `batch_shape + [num_timesteps, latent_size, latent_size]`. predicted_means: Means of the per-timestep predictive distributions over latent states, p(z_{t+1} | x_{:t}), as a Tensor of shape `sample_shape(x) + batch_shape + [num_timesteps, latent_size]`. predicted_covs: Covariances of the per-timestep predictive distributions over latent states, p(z_{t+1} | x_{:t}), as a Tensor of shape `batch_shape + [num_timesteps, latent_size, latent_size]`. Returns: posterior_means: Means of the smoothed marginal distributions p(z_t | x_{1:T}), as a Tensor of shape `sample_shape(x) + batch_shape + [num_timesteps, latent_size]`, which is of the same shape as filtered_means. posterior_covs: Covariances of the smoothed marginal distributions p(z_t | x_{1:T}), as a Tensor of shape `batch_shape + [num_timesteps, latent_size, latent_size]`. which is of the same shape as filtered_covs.", "response": "def backward_smoothing_pass(self,\n filtered_means,\n filtered_covs,\n predicted_means,\n predicted_covs):\n \"\"\"Run the backward pass in Kalman smoother.\n\n The backward smoothing is using Rauch, Tung and Striebel smoother as\n as discussed in section 18.3.2 of Kevin P. Murphy, 2012, Machine Learning:\n A Probabilistic Perspective, The MIT Press. The inputs are returned by\n `forward_filter` function.\n\n Args:\n filtered_means: Means of the per-timestep filtered marginal\n distributions p(z_t | x_{:t}), as a Tensor of shape\n `sample_shape(x) + batch_shape + [num_timesteps, latent_size]`.\n filtered_covs: Covariances of the per-timestep filtered marginal\n distributions p(z_t | x_{:t}), as a Tensor of shape\n `batch_shape + [num_timesteps, latent_size, latent_size]`.\n predicted_means: Means of the per-timestep predictive\n distributions over latent states, p(z_{t+1} | x_{:t}), as a\n Tensor of shape `sample_shape(x) + batch_shape +\n [num_timesteps, latent_size]`.\n predicted_covs: Covariances of the per-timestep predictive\n distributions over latent states, p(z_{t+1} | x_{:t}), as a\n Tensor of shape `batch_shape + [num_timesteps, latent_size,\n latent_size]`.\n\n Returns:\n posterior_means: Means of the smoothed marginal distributions\n p(z_t | x_{1:T}), as a Tensor of shape\n `sample_shape(x) + batch_shape + [num_timesteps, latent_size]`,\n which is of the same shape as filtered_means.\n posterior_covs: Covariances of the smoothed marginal distributions\n p(z_t | x_{1:T}), as a Tensor of shape\n `batch_shape + [num_timesteps, latent_size, latent_size]`.\n which is of the same shape as filtered_covs.\n \"\"\"\n with tf.name_scope(\"backward_pass\"):\n filtered_means = tf.convert_to_tensor(\n value=filtered_means, name=\"filtered_means\")\n filtered_covs = tf.convert_to_tensor(\n value=filtered_covs, name=\"filtered_covs\")\n predicted_means = tf.convert_to_tensor(\n value=predicted_means, name=\"predicted_means\")\n predicted_covs = tf.convert_to_tensor(\n value=predicted_covs, name=\"predicted_covs\")\n\n # To scan over time dimension, we need to move 'num_timesteps' from the\n # event shape to the initial dimension of the tensor.\n filtered_means = distribution_util.move_dimension(filtered_means, -2, 0)\n filtered_covs = distribution_util.move_dimension(filtered_covs, -3, 0)\n predicted_means = distribution_util.move_dimension(predicted_means, -2, 0)\n predicted_covs = distribution_util.move_dimension(predicted_covs, -3, 0)\n\n # The means are assumed to be vectors. Adding a dummy index to\n # ensure the `matmul` op working smoothly.\n filtered_means = filtered_means[..., tf.newaxis]\n predicted_means = predicted_means[..., tf.newaxis]\n\n initial_backward_mean = predicted_means[-1, ...]\n initial_backward_cov = predicted_covs[-1, ...]\n\n num_timesteps = tf.shape(input=filtered_means)[0]\n initial_state = BackwardPassState(\n backward_mean=initial_backward_mean,\n backward_cov=initial_backward_cov,\n timestep=self.initial_step + num_timesteps - 1)\n\n update_step_fn = build_backward_pass_step(\n self.get_transition_matrix_for_timestep)\n\n # For backward pass, it scans the `elems` from last to first.\n posterior_states = tf.scan(update_step_fn,\n elems=(filtered_means,\n filtered_covs,\n predicted_means,\n predicted_covs),\n initializer=initial_state,\n reverse=True)\n\n # Move the time dimension back into the event shape.\n posterior_means = distribution_util.move_dimension(\n posterior_states.backward_mean[..., 0], 0, -2)\n posterior_covs = distribution_util.move_dimension(\n posterior_states.backward_cov, 0, -3)\n\n return (posterior_means, posterior_covs)"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _joint_sample_n(self, n, seed=None):\n\n with tf.name_scope(\"sample_n_joint\"):\n stream = seed_stream.SeedStream(\n seed, salt=\"LinearGaussianStateSpaceModel_sample_n_joint\")\n\n sample_and_batch_shape = distribution_util.prefer_static_value(\n tf.concat([[n], self.batch_shape_tensor()],\n axis=0))\n\n # Sample the initial timestep from the prior. Since we want\n # this sample to have full batch shape (not just the batch shape\n # of the self.initial_state_prior object which might in general be\n # smaller), we augment the sample shape to include whatever\n # extra batch dimensions are required.\n with tf.control_dependencies(self.runtime_assertions):\n initial_latent = self.initial_state_prior.sample(\n sample_shape=_augment_sample_shape(\n self.initial_state_prior,\n sample_and_batch_shape,\n self.validate_args),\n seed=stream())\n\n # Add a dummy dimension so that matmul() does matrix-vector\n # multiplication.\n initial_latent = initial_latent[..., tf.newaxis]\n\n initial_observation_matrix = (\n self.get_observation_matrix_for_timestep(self.initial_step))\n initial_observation_noise = (\n self.get_observation_noise_for_timestep(self.initial_step))\n\n initial_observation_pred = initial_observation_matrix.matmul(\n initial_latent)\n initial_observation = (initial_observation_pred +\n initial_observation_noise.sample(\n sample_shape=_augment_sample_shape(\n initial_observation_noise,\n sample_and_batch_shape,\n self.validate_args),\n seed=stream())[..., tf.newaxis])\n\n sample_step = build_kalman_sample_step(\n self.get_transition_matrix_for_timestep,\n self.get_transition_noise_for_timestep,\n self.get_observation_matrix_for_timestep,\n self.get_observation_noise_for_timestep,\n full_sample_and_batch_shape=sample_and_batch_shape,\n stream=stream,\n validate_args=self.validate_args)\n\n # Scan over all timesteps to sample latents and observations.\n (latents, observations) = tf.scan(\n sample_step,\n elems=tf.range(self.initial_step+1, self.final_step),\n initializer=(initial_latent, initial_observation))\n\n # Combine the initial sampled timestep with the remaining timesteps.\n latents = tf.concat([initial_latent[tf.newaxis, ...],\n latents], axis=0)\n observations = tf.concat([initial_observation[tf.newaxis, ...],\n observations], axis=0)\n\n # Put dimensions back in order. The samples we've computed are\n # ordered by timestep, with shape `[num_timesteps, num_samples,\n # batch_shape, size, 1]` where `size` represents `latent_size`\n # or `observation_size` respectively. But timesteps are really\n # part of each probabilistic event, so we need to return a Tensor\n # of shape `[num_samples, batch_shape, num_timesteps, size]`.\n latents = tf.squeeze(latents, -1)\n latents = distribution_util.move_dimension(latents, 0, -2)\n observations = tf.squeeze(observations, -1)\n observations = distribution_util.move_dimension(observations, 0, -2)\n\n return latents, observations", "response": "Draw a joint sample from the prior over latents and observations."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nrunning a Kalman filter over a provided sequence of outputs.", "response": "def forward_filter(self, x, mask=None):\n \"\"\"Run a Kalman filter over a provided sequence of outputs.\n\n Note that the returned values `filtered_means`, `predicted_means`, and\n `observation_means` depend on the observed time series `x`, while the\n corresponding covariances are independent of the observed series; i.e., they\n depend only on the model itself. This means that the mean values have shape\n `concat([sample_shape(x), batch_shape, [num_timesteps,\n {latent/observation}_size]])`, while the covariances have shape\n `concat[(batch_shape, [num_timesteps, {latent/observation}_size,\n {latent/observation}_size]])`, which does not depend on the sample shape.\n\n Args:\n x: a float-type `Tensor` with rightmost dimensions\n `[num_timesteps, observation_size]` matching\n `self.event_shape`. Additional dimensions must match or be\n broadcastable to `self.batch_shape`; any further dimensions\n are interpreted as a sample shape.\n mask: optional bool-type `Tensor` with rightmost dimension\n `[num_timesteps]`; `True` values specify that the value of `x`\n at that timestep is masked, i.e., not conditioned on. Additional\n dimensions must match or be broadcastable to `self.batch_shape`; any\n further dimensions must match or be broadcastable to the sample\n shape of `x`.\n Default value: `None`.\n\n Returns:\n log_likelihoods: Per-timestep log marginal likelihoods `log\n p(x_t | x_{:t-1})` evaluated at the input `x`, as a `Tensor`\n of shape `sample_shape(x) + batch_shape + [num_timesteps].`\n filtered_means: Means of the per-timestep filtered marginal\n distributions p(z_t | x_{:t}), as a Tensor of shape\n `sample_shape(x) + batch_shape + [num_timesteps, latent_size]`.\n filtered_covs: Covariances of the per-timestep filtered marginal\n distributions p(z_t | x_{:t}), as a Tensor of shape\n `sample_shape(mask) + batch_shape + [num_timesteps, latent_size,\n latent_size]`. Note that the covariances depend only on the model and\n the mask, not on the data, so this may have fewer dimensions than\n `filtered_means`.\n predicted_means: Means of the per-timestep predictive\n distributions over latent states, p(z_{t+1} | x_{:t}), as a\n Tensor of shape `sample_shape(x) + batch_shape +\n [num_timesteps, latent_size]`.\n predicted_covs: Covariances of the per-timestep predictive\n distributions over latent states, p(z_{t+1} | x_{:t}), as a\n Tensor of shape `sample_shape(mask) + batch_shape +\n [num_timesteps, latent_size, latent_size]`. Note that the covariances\n depend only on the model and the mask, not on the data, so this may\n have fewer dimensions than `predicted_means`.\n observation_means: Means of the per-timestep predictive\n distributions over observations, p(x_{t} | x_{:t-1}), as a\n Tensor of shape `sample_shape(x) + batch_shape +\n [num_timesteps, observation_size]`.\n observation_covs: Covariances of the per-timestep predictive\n distributions over observations, p(x_{t} | x_{:t-1}), as a\n Tensor of shape `sample_shape(mask) + batch_shape + [num_timesteps,\n observation_size, observation_size]`. Note that the covariances depend\n only on the model and the mask, not on the data, so this may have fewer\n dimensions than `observation_means`.\n \"\"\"\n\n with tf.name_scope(\"forward_filter\"):\n x = tf.convert_to_tensor(value=x, name=\"x\")\n if mask is not None:\n mask = tf.convert_to_tensor(value=mask, name=\"mask\", dtype_hint=tf.bool)\n\n # Check event shape statically if possible\n check_x_shape_op = _check_equal_shape(\n \"x\", x.shape[-2:], tf.shape(input=x)[-2:],\n self.event_shape, self.event_shape_tensor())\n check_mask_dims_op = None\n check_mask_shape_op = None\n if mask is not None:\n if (tensorshape_util.rank(mask.shape) is None or\n tensorshape_util.rank(x.shape) is None):\n check_mask_dims_op = assert_util.assert_greater_equal(\n tf.rank(x),\n tf.rank(mask),\n message=(\"mask cannot have higher rank than x!\"))\n elif tensorshape_util.rank(mask.shape) > tensorshape_util.rank(x.shape):\n raise ValueError(\n \"mask cannot have higher rank than x! ({} vs {})\".format(\n tensorshape_util.rank(mask.shape),\n tensorshape_util.rank(x.shape)))\n check_mask_shape_op = _check_equal_shape(\n \"mask\", mask.shape[-1:], tf.shape(input=mask)[-1:],\n self.event_shape[-2:-1], self.event_shape_tensor()[-2:-1])\n if self.validate_args:\n runtime_assertions = self.runtime_assertions\n if check_x_shape_op is not None:\n runtime_assertions += [check_x_shape_op]\n if check_mask_shape_op is not None:\n runtime_assertions += [check_mask_shape_op]\n if check_mask_dims_op is not None:\n runtime_assertions += [check_mask_dims_op]\n with tf.control_dependencies(runtime_assertions):\n x = tf.identity(x)\n\n # Get the full output sample_shape + batch shape. Usually\n # this will just be x[:-2], i.e. the input shape excluding\n # event shape. But users can specify inputs that broadcast\n # batch dimensions, so we need to broadcast this against\n # self.batch_shape.\n if tensorshape_util.is_fully_defined(\n self.batch_shape) and tensorshape_util.is_fully_defined(x.shape):\n sample_and_batch_shape = tf.broadcast_static_shape(\n x.shape[:-2], self.batch_shape)\n else:\n sample_and_batch_shape = tf.broadcast_dynamic_shape(\n tf.shape(input=x)[:-2], self.batch_shape_tensor())\n\n # Get the full output shape for covariances. The posterior variances\n # in a LGSSM depend only on the model params (batch shape) and on the\n # missingness pattern (mask shape), so in general this may be smaller\n # than the full `sample_and_batch_shape`.\n if mask is None:\n mask_sample_and_batch_shape = self.batch_shape_tensor()\n else:\n if (tensorshape_util.is_fully_defined(self.batch_shape) and\n tensorshape_util.is_fully_defined(mask.shape)):\n mask_sample_and_batch_shape = tf.broadcast_static_shape(\n mask.shape[:-1], self.batch_shape)\n else:\n mask_sample_and_batch_shape = tf.broadcast_dynamic_shape(\n tf.shape(input=mask)[:-1], self.batch_shape_tensor())\n\n # To scan over timesteps we need to move `num_timsteps` from the\n # event shape to the initial dimension of the tensor.\n x = distribution_util.move_dimension(x, -2, 0)\n if mask is not None:\n mask = distribution_util.move_dimension(mask, -1, 0)\n\n # Observations are assumed to be vectors, but we add a dummy\n # extra dimension to allow us to use `matmul` throughout.\n x = x[..., tf.newaxis]\n if mask is not None:\n # Align mask.shape with x.shape, including a unit dimension to broadcast\n # against `observation_size`.\n mask = mask[..., tf.newaxis, tf.newaxis]\n\n # Initialize filtering distribution from the prior. The mean in\n # a Kalman filter depends on data, so should match the full\n # sample and batch shape. The covariance is data-independent, so\n # only has batch shape.\n prior_mean = _broadcast_to_shape(\n self.initial_state_prior.mean()[..., tf.newaxis],\n tf.concat([sample_and_batch_shape,\n [self.latent_size, 1]], axis=0))\n prior_cov = _broadcast_to_shape(\n self.initial_state_prior.covariance(),\n tf.concat([mask_sample_and_batch_shape,\n [self.latent_size, self.latent_size]], axis=0))\n\n initial_observation_matrix = (\n self.get_observation_matrix_for_timestep(self.initial_step))\n initial_observation_noise = (\n self.get_observation_noise_for_timestep(self.initial_step))\n\n initial_observation_mean = _propagate_mean(prior_mean,\n initial_observation_matrix,\n initial_observation_noise)\n initial_observation_cov = _propagate_cov(prior_cov,\n initial_observation_matrix,\n initial_observation_noise)\n\n initial_state = KalmanFilterState(\n predicted_mean=prior_mean,\n predicted_cov=prior_cov,\n filtered_mean=prior_mean, # establishes shape, value ignored\n filtered_cov=prior_cov, # establishes shape, value ignored\n observation_mean=initial_observation_mean,\n observation_cov=initial_observation_cov,\n log_marginal_likelihood=tf.zeros(\n shape=sample_and_batch_shape, dtype=self.dtype),\n timestep=tf.convert_to_tensor(\n value=self.initial_step, dtype=tf.int32, name=\"initial_step\"))\n\n update_step_fn = build_kalman_filter_step(\n self.get_transition_matrix_for_timestep,\n self.get_transition_noise_for_timestep,\n self.get_observation_matrix_for_timestep,\n self.get_observation_noise_for_timestep)\n\n filter_states = tf.scan(update_step_fn,\n elems=x if mask is None else (x, mask),\n initializer=initial_state)\n\n log_likelihoods = distribution_util.move_dimension(\n filter_states.log_marginal_likelihood, 0, -1)\n\n # Move the time dimension back into the event shape.\n filtered_means = distribution_util.move_dimension(\n filter_states.filtered_mean[..., 0], 0, -2)\n filtered_covs = distribution_util.move_dimension(\n filter_states.filtered_cov, 0, -3)\n predicted_means = distribution_util.move_dimension(\n filter_states.predicted_mean[..., 0], 0, -2)\n predicted_covs = distribution_util.move_dimension(\n filter_states.predicted_cov, 0, -3)\n observation_means = distribution_util.move_dimension(\n filter_states.observation_mean[..., 0], 0, -2)\n observation_covs = distribution_util.move_dimension(\n filter_states.observation_cov, 0, -3)\n # We could directly construct the batch Distributions\n # filtered_marginals = tfd.MultivariateNormalFullCovariance(\n # filtered_means, filtered_covs)\n # predicted_marginals = tfd.MultivariateNormalFullCovariance(\n # predicted_means, predicted_covs)\n # but we choose not to: returning the raw means and covariances\n # saves computation in Eager mode (avoiding an immediate\n # Cholesky factorization that the user may not want) and aids\n # debugging of numerical issues.\n\n return (log_likelihoods,\n filtered_means, filtered_covs,\n predicted_means, predicted_covs,\n observation_means, observation_covs)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef posterior_marginals(self, x, mask=None):\n\n with tf.name_scope(\"smooth\"):\n x = tf.convert_to_tensor(value=x, name=\"x\")\n (_, filtered_means, filtered_covs,\n predicted_means, predicted_covs, _, _) = self.forward_filter(\n x, mask=mask)\n\n (smoothed_means, smoothed_covs) = self.backward_smoothing_pass(\n filtered_means, filtered_covs,\n predicted_means, predicted_covs)\n\n return (smoothed_means, smoothed_covs)", "response": "Run a Kalman smoother to return the posterior mean and covariance of the time series x."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _joint_mean(self):\n\n with tf.name_scope(\"mean_joint\"):\n\n # The initial timestep is a special case, since we sample the\n # latent state from the prior rather than the transition model.\n\n with tf.control_dependencies(self.runtime_assertions):\n # Broadcast to ensure we represent the full batch shape.\n initial_latent_mean = _broadcast_to_shape(\n self.initial_state_prior.mean()[..., tf.newaxis],\n tf.concat([self.batch_shape_tensor(),\n [self.latent_size, 1]], axis=0))\n\n initial_observation_mean = _propagate_mean(\n initial_latent_mean,\n self.get_observation_matrix_for_timestep(self.initial_step),\n self.get_observation_noise_for_timestep(self.initial_step))\n\n mean_step = build_kalman_mean_step(\n self.get_transition_matrix_for_timestep,\n self.get_transition_noise_for_timestep,\n self.get_observation_matrix_for_timestep,\n self.get_observation_noise_for_timestep)\n\n # Scan over all timesteps following the initial step.\n (latent_means, observation_means) = tf.scan(\n mean_step,\n elems=tf.range(self.initial_step+1, self.final_step),\n initializer=(initial_latent_mean, initial_observation_mean))\n\n # Squish the initial step back on top of the other (scanned) timesteps\n latent_means = tf.concat([initial_latent_mean[tf.newaxis, ...],\n latent_means], axis=0)\n observation_means = tf.concat([initial_observation_mean[tf.newaxis, ...],\n observation_means], axis=0)\n\n # Put dimensions back in order. The samples we've computed have\n # shape `[num_timesteps, batch_shape, size, 1]`, where `size`\n # is the dimension of the latent or observation spaces\n # respectively, but we want to return values with shape\n # `[batch_shape, num_timesteps, size]`.\n latent_means = tf.squeeze(latent_means, -1)\n latent_means = distribution_util.move_dimension(latent_means, 0, -2)\n observation_means = tf.squeeze(observation_means, -1)\n observation_means = distribution_util.move_dimension(\n observation_means, 0, -2)\n\n return latent_means, observation_means", "response": "Compute prior means for all variables via dynamic programming."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\ncompute prior covariance matrices for all variables via dynamic programming.", "response": "def _joint_covariances(self):\n \"\"\"Compute prior covariances for all variables via dynamic programming.\n\n Returns:\n latent_covs: Prior covariance matrices of latent states `z_t`, as\n a `Tensor` of shape `batch_shape + [num_timesteps,\n latent_size, latent_size]`\n observation_covs: Prior covariance matrices of observations\n `x_t`, as a `Tensor` of shape `batch_shape + [num_timesteps,\n observation_size, observation_size]`\n \"\"\"\n\n with tf.name_scope(\"covariance_joint\"):\n\n with tf.control_dependencies(self.runtime_assertions):\n initial_latent_cov = _broadcast_to_shape(\n self.initial_state_prior.covariance(),\n tf.concat([self.batch_shape_tensor(),\n [self.latent_size, self.latent_size]], axis=0))\n\n initial_observation_cov = _propagate_cov(\n initial_latent_cov,\n self.get_observation_matrix_for_timestep(self.initial_step),\n self.get_observation_noise_for_timestep(self.initial_step))\n\n cov_step = build_kalman_cov_step(\n self.get_transition_matrix_for_timestep,\n self.get_transition_noise_for_timestep,\n self.get_observation_matrix_for_timestep,\n self.get_observation_noise_for_timestep)\n\n # Scan over all timesteps following the initial step.\n (latent_covs, observation_covs) = tf.scan(\n cov_step,\n elems=tf.range(self.initial_step+1, self.final_step),\n initializer=(initial_latent_cov, initial_observation_cov))\n\n # Squish the initial step back on top of the other (scanned) timesteps\n latent_covs = tf.concat([initial_latent_cov[tf.newaxis, ...],\n latent_covs], axis=0)\n observation_covs = tf.concat([initial_observation_cov[tf.newaxis, ...],\n observation_covs], axis=0)\n\n # Put dimensions back in order. The samples we've computed have\n # shape `[num_timesteps, batch_shape, size, size]`, where `size`\n # is the dimension of the state or observation spaces\n # respectively, but we want to return values with shape\n # `[batch_shape, num_timesteps, size, size]`.\n latent_covs = distribution_util.move_dimension(latent_covs, 0, -3)\n observation_covs = distribution_util.move_dimension(\n observation_covs, 0, -3)\n return latent_covs, observation_covs"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef latents_to_observations(self, latent_means, latent_covs):\n\n with tf.name_scope(\"latents_to_observations\"):\n\n pushforward_latents_step = build_pushforward_latents_step(\n self.get_observation_matrix_for_timestep,\n self.get_observation_noise_for_timestep)\n\n latent_means = distribution_util.move_dimension(\n latent_means, source_idx=-2, dest_idx=0)\n latent_means = latent_means[..., tf.newaxis] # Make matmul happy.\n latent_covs = distribution_util.move_dimension(\n latent_covs, source_idx=-3, dest_idx=0)\n\n (initial_observation_mean,\n initial_observation_cov) = pushforward_latents_step(\n _=None, # Loop body ignores previous observations.\n latent_t_mean_cov=(self.initial_step,\n latent_means[self.initial_step],\n latent_covs[self.initial_step]))\n\n # TODO(davmre) this loop is embarassingly parallel; replace with `pfor`.\n timesteps = tf.range(self.initial_step,\n self.initial_step + self.num_timesteps)\n observation_means, observation_covs = tf.scan(\n pushforward_latents_step,\n elems=(timesteps, latent_means, latent_covs),\n initializer=(initial_observation_mean, initial_observation_cov),\n parallel_iterations=10000)\n\n observation_means = distribution_util.move_dimension(\n observation_means[..., 0], source_idx=0, dest_idx=-2)\n observation_covs = distribution_util.move_dimension(\n observation_covs, source_idx=0, dest_idx=-3)\n\n return observation_means, observation_covs", "response": "Push latent means and covariance forward through the observation model."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\ncomputing I_v z using a recurrence relation where z > 0.", "response": "def _bessel_ive(v, z, cache=None):\n \"\"\"Computes I_v(z)*exp(-abs(z)) using a recurrence relation, where z > 0.\"\"\"\n # TODO(b/67497980): Switch to a more numerically faithful implementation.\n z = tf.convert_to_tensor(value=z)\n\n wrap = lambda result: tf.debugging.check_numerics(result, 'besseli{}'.format(v\n ))\n\n if float(v) >= 2:\n raise ValueError(\n 'Evaluating bessel_i by recurrence becomes imprecise for large v')\n\n cache = cache or {}\n safe_z = tf.where(z > 0, z, tf.ones_like(z))\n if v in cache:\n return wrap(cache[v])\n if v == 0:\n cache[v] = tf.math.bessel_i0e(z)\n elif v == 1:\n cache[v] = tf.math.bessel_i1e(z)\n elif v == 0.5:\n # sinh(x)*exp(-abs(x)), sinh(x) = (e^x - e^{-x}) / 2\n sinhe = lambda x: (tf.exp(x - tf.abs(x)) - tf.exp(-x - tf.abs(x))) / 2\n cache[v] = (\n np.sqrt(2 / np.pi) * sinhe(z) *\n tf.where(z > 0, tf.math.rsqrt(safe_z), tf.ones_like(safe_z)))\n elif v == -0.5:\n # cosh(x)*exp(-abs(x)), cosh(x) = (e^x + e^{-x}) / 2\n coshe = lambda x: (tf.exp(x - tf.abs(x)) + tf.exp(-x - tf.abs(x))) / 2\n cache[v] = (\n np.sqrt(2 / np.pi) * coshe(z) *\n tf.where(z > 0, tf.math.rsqrt(safe_z), tf.ones_like(safe_z)))\n if v <= 1:\n return wrap(cache[v])\n # Recurrence relation:\n cache[v] = (_bessel_ive(v - 2, z, cache) -\n (2 * (v - 1)) * _bessel_ive(v - 1, z, cache) / z)\n return wrap(cache[v])"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\ncompute the log - normalizer of the distribution.", "response": "def _log_normalization(self):\n \"\"\"Computes the log-normalizer of the distribution.\"\"\"\n event_dim = tf.compat.dimension_value(self.event_shape[0])\n if event_dim is None:\n raise ValueError('vMF _log_normalizer currently only supports '\n 'statically known event shape')\n safe_conc = tf.where(self.concentration > 0,\n self.concentration,\n tf.ones_like(self.concentration))\n safe_lognorm = ((event_dim / 2 - 1) * tf.math.log(safe_conc) -\n (event_dim / 2) * np.log(2 * np.pi) -\n tf.math.log(_bessel_ive(event_dim / 2 - 1, safe_conc)) -\n tf.abs(safe_conc))\n log_nsphere_surface_area = (\n np.log(2.) + (event_dim / 2) * np.log(np.pi) -\n tf.math.lgamma(tf.cast(event_dim / 2, self.dtype)))\n return tf.where(self.concentration > 0,\n -safe_lognorm,\n log_nsphere_surface_area * tf.ones_like(safe_lognorm))"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef _maybe_assert_valid_sample(self, samples):\n if not self.validate_args:\n return samples\n with tf.control_dependencies([\n assert_util.assert_near(\n 1.,\n tf.linalg.norm(tensor=samples, axis=-1),\n message='samples must be unit length'),\n assert_util.assert_equal(\n tf.shape(input=samples)[-1:],\n self.event_shape_tensor(),\n message=('samples must have innermost dimension matching that of '\n '`self.mean_direction`')),\n ]):\n return tf.identity(samples)", "response": "Check counts for proper shape values then return tensor version."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef _mode(self):\n return (self.mean_direction +\n tf.zeros_like(self.concentration)[..., tf.newaxis])", "response": "The mode of the von Mises - Fisher distribution."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _rotate(self, samples):\n event_dim = (\n tf.compat.dimension_value(self.event_shape[0]) or\n self._event_shape_tensor()[0])\n basis = tf.concat([[1.], tf.zeros([event_dim - 1], dtype=self.dtype)],\n axis=0),\n u = tf.nn.l2_normalize(basis - self.mean_direction, axis=-1)\n return samples - 2 * tf.reduce_sum(\n input_tensor=samples * u, axis=-1, keepdims=True) * u", "response": "Applies a Householder rotation to samples."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nspecializes inversion sampler for 3D.", "response": "def _sample_3d(self, n, seed=None):\n \"\"\"Specialized inversion sampler for 3D.\"\"\"\n seed = seed_stream.SeedStream(seed, salt='von_mises_fisher_3d')\n u_shape = tf.concat([[n], self._batch_shape_tensor()], axis=0)\n z = tf.random.uniform(u_shape, seed=seed(), dtype=self.dtype)\n # TODO(bjp): Higher-order odd dim analytic CDFs are available in [1], could\n # be bisected for bounded sampling runtime (i.e. not rejection sampling).\n # [1]: Inversion sampler via: https://ieeexplore.ieee.org/document/7347705/\n # The inversion is: u = 1 + log(z + (1-z)*exp(-2*kappa)) / kappa\n # We must protect against both kappa and z being zero.\n safe_conc = tf.where(self.concentration > 0,\n self.concentration,\n tf.ones_like(self.concentration))\n safe_z = tf.where(z > 0, z, tf.ones_like(z))\n safe_u = 1 + tf.reduce_logsumexp(\n input_tensor=[\n tf.math.log(safe_z),\n tf.math.log1p(-safe_z) - 2 * safe_conc\n ],\n axis=0) / safe_conc\n # Limit of the above expression as kappa->0 is 2*z-1\n u = tf.where(self.concentration > tf.zeros_like(safe_u), safe_u,\n 2 * z - 1)\n # Limit of the expression as z->0 is -1.\n u = tf.where(tf.equal(z, 0), -tf.ones_like(u), u)\n if not self._allow_nan_stats:\n u = tf.debugging.check_numerics(u, 'u in _sample_3d')\n return u[..., tf.newaxis]"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\ncreate a deep copy of a function.", "response": "def _copy_fn(fn):\n \"\"\"Create a deep copy of fn.\n\n Args:\n fn: a callable\n\n Returns:\n A `FunctionType`: a deep copy of fn.\n\n Raises:\n TypeError: if `fn` is not a callable.\n \"\"\"\n if not callable(fn):\n raise TypeError(\"fn is not callable: {}\".format(fn))\n # The blessed way to copy a function. copy.deepcopy fails to create a\n # non-reference copy. Since:\n # types.FunctionType == type(lambda: None),\n # and the docstring for the function type states:\n #\n # function(code, globals[, name[, argdefs[, closure]]])\n #\n # Create a function object from a code object and a dictionary.\n # ...\n #\n # Here we can use this to create a new function with the old function's\n # code, globals, closure, etc.\n return types.FunctionType(\n code=fn.__code__, globals=fn.__globals__,\n name=fn.__name__, argdefs=fn.__defaults__,\n closure=fn.__closure__)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef _update_docstring(old_str, append_str):\n old_str = old_str or \"\"\n old_str_lines = old_str.split(\"\\n\")\n\n # Step 0: Prepend spaces to all lines of append_str. This is\n # necessary for correct markdown generation.\n append_str = \"\\n\".join(\" %s\" % line for line in append_str.split(\"\\n\"))\n\n # Step 1: Find mention of \"Args\":\n has_args_ix = [\n ix for ix, line in enumerate(old_str_lines)\n if line.strip().lower() == \"args:\"]\n if has_args_ix:\n final_args_ix = has_args_ix[-1]\n return (\"\\n\".join(old_str_lines[:final_args_ix])\n + \"\\n\\n\" + append_str + \"\\n\\n\"\n + \"\\n\".join(old_str_lines[final_args_ix:]))\n else:\n return old_str + \"\\n\\n\" + append_str", "response": "Update old_str by inserting append_str just before the args section."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _convert_to_tensor(value, dtype=None, dtype_hint=None, name=None):\n if (tf.nest.is_nested(dtype) or\n tf.nest.is_nested(dtype_hint)):\n if dtype is None:\n fn = lambda v, pd: tf.convert_to_tensor(v, dtype_hint=pd, name=name)\n return tf.nest.map_structure(fn, value, dtype_hint)\n elif dtype_hint is None:\n fn = lambda v, d: tf.convert_to_tensor(v, dtype=d, name=name)\n return tf.nest.map_structure(fn, value, dtype_hint)\n else:\n fn = lambda v, d, pd: tf.convert_to_tensor( # pylint: disable=g-long-lambda\n v, dtype=d, dtype_hint=pd, name=name)\n return tf.nest.map_structure(fn, value, dtype, dtype_hint)\n return tf.convert_to_tensor(\n value=value, dtype=dtype, dtype_hint=dtype_hint, name=name)", "response": "Converts the given value to a tensor."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nremoves dict keys which have self as value.", "response": "def _remove_dict_keys_with_value(dict_, val):\n \"\"\"Removes `dict` keys which have have `self` as value.\"\"\"\n return {k: v for k, v in dict_.items() if v is not val}"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _recursively_replace_dict_for_pretty_dict(x):\n # We use \"PrettyDict\" because collections.OrderedDict repr/str has the word\n # \"OrderedDict\" in it. We only want to print \"OrderedDict\" if in fact the\n # input really is an OrderedDict.\n if isinstance(x, dict):\n return _PrettyDict({\n k: _recursively_replace_dict_for_pretty_dict(v)\n for k, v in x.items()})\n if (isinstance(x, collections.Sequence) and\n not isinstance(x, six.string_types)):\n args = (_recursively_replace_dict_for_pretty_dict(x_) for x_ in x)\n is_named_tuple = (isinstance(x, tuple) and\n hasattr(x, \"_asdict\") and\n hasattr(x, \"_fields\"))\n return type(x)(*args) if is_named_tuple else type(x)(args)\n if isinstance(x, collections.Mapping):\n return type(x)(**{k: _recursively_replace_dict_for_pretty_dict(v)\n for k, v in x.items()})\n return x", "response": "Recursively replace dict s with _PrettyDict."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\ncomputes the Monte-Carlo approximation of `E_p[f(X)]`. This function computes the Monte-Carlo approximation of an expectation, i.e., ```none E_p[f(X)] approx= m**-1 sum_i^m f(x_j), x_j ~iid p(X) ``` where: - `x_j = samples[j, ...]`, - `log(p(samples)) = log_prob(samples)` and - `m = prod(shape(samples)[axis])`. Tricks: Reparameterization and Score-Gradient When p is \"reparameterized\", i.e., a diffeomorphic transformation of a parameterless distribution (e.g., `Normal(Y; m, s) <=> Y = sX + m, X ~ Normal(0,1)`), we can swap gradient and expectation, i.e., `grad[ Avg{ s_i : i=1...n } ] = Avg{ grad[s_i] : i=1...n }` where `S_n = Avg{s_i}` and `s_i = f(x_i), x_i ~ p`. However, if p is not reparameterized, TensorFlow's gradient will be incorrect since the chain-rule stops at samples of non-reparameterized distributions. (The non-differentiated result, `approx_expectation`, is the same regardless of `use_reparametrization`.) In this circumstance using the Score-Gradient trick results in an unbiased gradient, i.e., ```none grad[ E_p[f(X)] ] = grad[ int dx p(x) f(x) ] = int dx grad[ p(x) f(x) ] = int dx [ p'(x) f(x) + p(x) f'(x) ] = int dx p(x) [p'(x) / p(x) f(x) + f'(x) ] = int dx p(x) grad[ f(x) p(x) / stop_grad[p(x)] ] = E_p[ grad[ f(x) p(x) / stop_grad[p(x)] ] ] ``` Unless p is not reparametrized, it is usually preferable to `use_reparametrization = True`. Warning: users are responsible for verifying `p` is a \"reparameterized\" distribution. Example Use: ```python # Monte-Carlo approximation of a reparameterized distribution, e.g., Normal. num_draws = int(1e5) p = tfp.distributions.Normal(loc=0., scale=1.) q = tfp.distributions.Normal(loc=1., scale=2.) exact_kl_normal_normal = tfp.distributions.kl_divergence(p, q) # ==> 0.44314718 approx_kl_normal_normal = tfp.monte_carlo.expectation( f=lambda x: p.log_prob(x) - q.log_prob(x), samples=p.sample(num_draws, seed=42), log_prob=p.log_prob, use_reparametrization=(p.reparameterization_type == tfp.distributions.FULLY_REPARAMETERIZED)) # ==> 0.44632751 # Relative Error: <1% # Monte-Carlo approximation of non-reparameterized distribution, # e.g., Bernoulli. num_draws = int(1e5) p = tfp.distributions.Bernoulli(probs=0.4) q = tfp.distributions.Bernoulli(probs=0.8) exact_kl_bernoulli_bernoulli = tfp.distributions.kl_divergence(p, q) # ==> 0.38190854 approx_kl_bernoulli_bernoulli = tfp.monte_carlo.expectation( f=lambda x: p.log_prob(x) - q.log_prob(x), samples=p.sample(num_draws, seed=42), log_prob=p.log_prob, use_reparametrization=(p.reparameterization_type == tfp.distributions.FULLY_REPARAMETERIZED)) # ==> 0.38336259 # Relative Error: <1% # For comparing the gradients, see `expectation_test.py`. ``` Note: The above example is for illustration only. To compute approximate KL-divergence, the following is preferred: ```python approx_kl_p_q = bf.monte_carlo_csiszar_f_divergence( f=bf.kl_reverse, p_log_prob=q.log_prob, q=p, num_draws=num_draws) ``` Args: f: Python callable which can return `f(samples)`. samples: `Tensor` of samples used to form the Monte-Carlo approximation of `E_p[f(X)]`. A batch of samples should be indexed by `axis` dimensions. log_prob: Python callable which can return `log_prob(samples)`. Must correspond to the natural-logarithm of the pdf/pmf of each sample. Only required/used if `use_reparametrization=False`. Default value: `None`. use_reparametrization: Python `bool` indicating that the approximation should use the fact that the gradient of samples is unbiased. Whether `True` or `False`, this arg only affects the gradient of the resulting `approx_expectation`. Default value: `True`. axis: The dimensions to average. If `None`, averages all dimensions. Default value: `0` (the left-most dimension). keep_dims: If True, retains averaged dimensions using size `1`. Default value: `False`. name: A `name_scope` for operations created by this function. Default value: `None` (which implies \"expectation\"). Returns: approx_expectation: `Tensor` corresponding to the Monte-Carlo approximation of `E_p[f(X)]`. Raises: ValueError: if `f` is not a Python `callable`. ValueError: if `use_reparametrization=False` and `log_prob` is not a Python `callable`.", "response": "def expectation(f, samples, log_prob=None, use_reparametrization=True,\n axis=0, keep_dims=False, name=None):\n \"\"\"Computes the Monte-Carlo approximation of `E_p[f(X)]`.\n\n This function computes the Monte-Carlo approximation of an expectation, i.e.,\n\n ```none\n E_p[f(X)] approx= m**-1 sum_i^m f(x_j), x_j ~iid p(X)\n ```\n\n where:\n\n - `x_j = samples[j, ...]`,\n - `log(p(samples)) = log_prob(samples)` and\n - `m = prod(shape(samples)[axis])`.\n\n Tricks: Reparameterization and Score-Gradient\n\n When p is \"reparameterized\", i.e., a diffeomorphic transformation of a\n parameterless distribution (e.g.,\n `Normal(Y; m, s) <=> Y = sX + m, X ~ Normal(0,1)`), we can swap gradient and\n expectation, i.e.,\n `grad[ Avg{ s_i : i=1...n } ] = Avg{ grad[s_i] : i=1...n }` where\n `S_n = Avg{s_i}` and `s_i = f(x_i), x_i ~ p`.\n\n However, if p is not reparameterized, TensorFlow's gradient will be incorrect\n since the chain-rule stops at samples of non-reparameterized distributions.\n (The non-differentiated result, `approx_expectation`, is the same regardless\n of `use_reparametrization`.) In this circumstance using the Score-Gradient\n trick results in an unbiased gradient, i.e.,\n\n ```none\n grad[ E_p[f(X)] ]\n = grad[ int dx p(x) f(x) ]\n = int dx grad[ p(x) f(x) ]\n = int dx [ p'(x) f(x) + p(x) f'(x) ]\n = int dx p(x) [p'(x) / p(x) f(x) + f'(x) ]\n = int dx p(x) grad[ f(x) p(x) / stop_grad[p(x)] ]\n = E_p[ grad[ f(x) p(x) / stop_grad[p(x)] ] ]\n ```\n\n Unless p is not reparametrized, it is usually preferable to\n `use_reparametrization = True`.\n\n Warning: users are responsible for verifying `p` is a \"reparameterized\"\n distribution.\n\n Example Use:\n\n ```python\n # Monte-Carlo approximation of a reparameterized distribution, e.g., Normal.\n\n num_draws = int(1e5)\n p = tfp.distributions.Normal(loc=0., scale=1.)\n q = tfp.distributions.Normal(loc=1., scale=2.)\n exact_kl_normal_normal = tfp.distributions.kl_divergence(p, q)\n # ==> 0.44314718\n approx_kl_normal_normal = tfp.monte_carlo.expectation(\n f=lambda x: p.log_prob(x) - q.log_prob(x),\n samples=p.sample(num_draws, seed=42),\n log_prob=p.log_prob,\n use_reparametrization=(p.reparameterization_type\n == tfp.distributions.FULLY_REPARAMETERIZED))\n # ==> 0.44632751\n # Relative Error: <1%\n\n # Monte-Carlo approximation of non-reparameterized distribution,\n # e.g., Bernoulli.\n\n num_draws = int(1e5)\n p = tfp.distributions.Bernoulli(probs=0.4)\n q = tfp.distributions.Bernoulli(probs=0.8)\n exact_kl_bernoulli_bernoulli = tfp.distributions.kl_divergence(p, q)\n # ==> 0.38190854\n approx_kl_bernoulli_bernoulli = tfp.monte_carlo.expectation(\n f=lambda x: p.log_prob(x) - q.log_prob(x),\n samples=p.sample(num_draws, seed=42),\n log_prob=p.log_prob,\n use_reparametrization=(p.reparameterization_type\n == tfp.distributions.FULLY_REPARAMETERIZED))\n # ==> 0.38336259\n # Relative Error: <1%\n\n # For comparing the gradients, see `expectation_test.py`.\n ```\n\n Note: The above example is for illustration only. To compute approximate\n KL-divergence, the following is preferred:\n\n ```python\n approx_kl_p_q = bf.monte_carlo_csiszar_f_divergence(\n f=bf.kl_reverse,\n p_log_prob=q.log_prob,\n q=p,\n num_draws=num_draws)\n ```\n\n Args:\n f: Python callable which can return `f(samples)`.\n samples: `Tensor` of samples used to form the Monte-Carlo approximation of\n `E_p[f(X)]`. A batch of samples should be indexed by `axis` dimensions.\n log_prob: Python callable which can return `log_prob(samples)`. Must\n correspond to the natural-logarithm of the pdf/pmf of each sample. Only\n required/used if `use_reparametrization=False`.\n Default value: `None`.\n use_reparametrization: Python `bool` indicating that the approximation\n should use the fact that the gradient of samples is unbiased. Whether\n `True` or `False`, this arg only affects the gradient of the resulting\n `approx_expectation`.\n Default value: `True`.\n axis: The dimensions to average. If `None`, averages all\n dimensions.\n Default value: `0` (the left-most dimension).\n keep_dims: If True, retains averaged dimensions using size `1`.\n Default value: `False`.\n name: A `name_scope` for operations created by this function.\n Default value: `None` (which implies \"expectation\").\n\n Returns:\n approx_expectation: `Tensor` corresponding to the Monte-Carlo approximation\n of `E_p[f(X)]`.\n\n Raises:\n ValueError: if `f` is not a Python `callable`.\n ValueError: if `use_reparametrization=False` and `log_prob` is not a Python\n `callable`.\n \"\"\"\n\n with tf.compat.v1.name_scope(name, 'expectation', [samples]):\n if not callable(f):\n raise ValueError('`f` must be a callable function.')\n if use_reparametrization:\n return tf.reduce_mean(\n input_tensor=f(samples), axis=axis, keepdims=keep_dims)\n else:\n if not callable(log_prob):\n raise ValueError('`log_prob` must be a callable function.')\n stop = tf.stop_gradient # For readability.\n x = stop(samples)\n logpx = log_prob(x)\n fx = f(x) # Call `f` once in case it has side-effects.\n # To achieve this, we use the fact that:\n # `h(x) - stop(h(x)) == zeros_like(h(x))`\n # but its gradient is grad[h(x)].\n #\n # This technique was published as:\n # Jakob Foerster, Greg Farquhar, Maruan Al-Shedivat, Tim Rocktaeschel,\n # Eric P. Xing, Shimon Whiteson (ICML 2018)\n # \"DiCE: The Infinitely Differentiable Monte-Carlo Estimator\"\n # https://arxiv.org/abs/1802.05098\n #\n # Unlike using:\n # fx = fx + stop(fx) * (logpx - stop(logpx)),\n # DiCE ensures that any order gradients of the objective\n # are unbiased gradient estimators.\n #\n # Note that IEEE754 specifies that `x - x == 0.` and `x + 0. == x`, hence\n # this trick loses no precision. For more discussion regarding the\n # relevant portions of the IEEE754 standard, see the StackOverflow\n # question,\n # \"Is there a floating point value of x, for which x-x == 0 is false?\"\n # http://stackoverflow.com/q/2686644\n dice = fx * tf.exp(logpx - stop(logpx))\n return tf.reduce_mean(input_tensor=dice, axis=axis, keepdims=keep_dims)"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _get_samples(dist, z, n, seed):\n with tf.compat.v1.name_scope('get_samples', values=[z, n]):\n if (n is None) == (z is None):\n raise ValueError(\n 'Must specify exactly one of arguments \"n\" and \"z\". Found: '\n 'n = %s, z = %s' % (n, z))\n if n is not None:\n return dist.sample(n, seed=seed)\n else:\n return tf.convert_to_tensor(value=z, name='z')", "response": "Check args and return samples."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef _choose_base_case(is_accepted,\n accepted,\n rejected,\n name=None):\n \"\"\"Helper to `choose` which expand_dims `is_accepted` and applies tf.where.\"\"\"\n def _expand_is_accepted_like(x):\n \"\"\"Helper to expand `is_accepted` like the shape of some input arg.\"\"\"\n with tf.compat.v1.name_scope('expand_is_accepted_like'):\n expand_shape = tf.concat([\n tf.shape(input=is_accepted),\n tf.ones([tf.rank(x) - tf.rank(is_accepted)], dtype=tf.int32),\n ],\n axis=0)\n multiples = tf.concat([\n tf.ones([tf.rank(is_accepted)], dtype=tf.int32),\n tf.shape(input=x)[tf.rank(is_accepted):],\n ],\n axis=0)\n m = tf.tile(tf.reshape(is_accepted, expand_shape),\n multiples)\n m.set_shape(m.shape.merge_with(x.shape))\n return m\n def _where(accepted, rejected):\n if accepted is rejected:\n return accepted\n accepted = tf.convert_to_tensor(value=accepted, name='accepted')\n rejected = tf.convert_to_tensor(value=rejected, name='rejected')\n r = tf.where(_expand_is_accepted_like(accepted), accepted, rejected)\n r.set_shape(r.shape.merge_with(accepted.shape.merge_with(rejected.shape)))\n return r\n\n with tf.compat.v1.name_scope(\n name, 'choose', values=[is_accepted, accepted, rejected]):\n if not is_list_like(accepted):\n return _where(accepted, rejected)\n return [(choose(is_accepted, a, r, name=name) if is_namedtuple_like(a)\n else _where(a, r))\n for a, r in zip(accepted, rejected)]", "response": "Helper function to choose base case."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef choose(is_accepted, accepted, rejected, name=None):\n if not is_namedtuple_like(accepted):\n return _choose_base_case(is_accepted, accepted, rejected, name=name)\n if not isinstance(accepted, type(rejected)):\n raise TypeError('Type of `accepted` ({}) must be identical to '\n 'type of `rejected` ({})'.format(\n type(accepted).__name__,\n type(rejected).__name__))\n return type(accepted)(**dict(\n [(fn,\n choose(is_accepted,\n getattr(accepted, fn),\n getattr(rejected, fn),\n name=name))\n for fn in accepted._fields]))", "response": "Helper which expand_dims is_accepted then applies tf. where."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef safe_sum(x, alt_value=-np.inf, name=None):\n with tf.compat.v1.name_scope(name, 'safe_sum', [x, alt_value]):\n if not is_list_like(x):\n raise TypeError('Expected list input.')\n if not x:\n raise ValueError('Input should not be empty.')\n in_shape = x[0].shape\n x = tf.stack(x, axis=-1)\n x = tf.reduce_sum(input_tensor=x, axis=-1)\n alt_value = np.array(alt_value, x.dtype.as_numpy_dtype)\n alt_fill = tf.fill(tf.shape(input=x), value=alt_value)\n x = tf.where(tf.math.is_finite(x), x, alt_fill)\n x.set_shape(x.shape.merge_with(in_shape))\n return x", "response": "Elementwise adds list members replacing non - finite results with alt_value."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\ncalling fn and computes the gradient of the result wrt args_list.", "response": "def maybe_call_fn_and_grads(fn,\n fn_arg_list,\n result=None,\n grads=None,\n check_non_none_grads=True,\n name=None):\n \"\"\"Calls `fn` and computes the gradient of the result wrt `args_list`.\"\"\"\n with tf.compat.v1.name_scope(name, 'maybe_call_fn_and_grads',\n [fn_arg_list, result, grads]):\n fn_arg_list = (list(fn_arg_list) if is_list_like(fn_arg_list)\n else [fn_arg_list])\n result, grads = _value_and_gradients(fn, fn_arg_list, result, grads)\n if not all(r.dtype.is_floating\n for r in (result if is_list_like(result) else [result])): # pylint: disable=superfluous-parens\n raise TypeError('Function result must be a `Tensor` with `float` '\n '`dtype`.')\n if len(fn_arg_list) != len(grads):\n raise ValueError('Function args must be in one-to-one correspondence '\n 'with grads.')\n if check_non_none_grads and any(g is None for g in grads):\n raise ValueError('Encountered `None` gradient.\\n'\n ' fn_arg_list: {}\\n'\n ' grads: {}'.format(fn_arg_list, grads))\n return result, grads"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef smart_for_loop(loop_num_iter, body_fn, initial_loop_vars,\n parallel_iterations=10, name=None):\n \"\"\"Construct a for loop, preferring a python loop if `n` is staticaly known.\n\n Given `loop_num_iter` and `body_fn`, return an op corresponding to executing\n `body_fn` `loop_num_iter` times, feeding previous outputs of `body_fn` into\n the next iteration.\n\n If `loop_num_iter` is statically known, the op is constructed via python for\n loop, and otherwise a `tf.while_loop` is used.\n\n Args:\n loop_num_iter: `Integer` `Tensor` representing the number of loop\n iterations.\n body_fn: Callable to be executed `loop_num_iter` times.\n initial_loop_vars: Listlike object of `Tensors` to be passed in to\n `body_fn`'s first execution.\n parallel_iterations: The number of iterations allowed to run in parallel.\n It must be a positive integer. See `tf.while_loop` for more details.\n Default value: `10`.\n name: Python `str` name prefixed to Ops created by this function.\n Default value: `None` (i.e., \"smart_for_loop\").\n Returns:\n result: `Tensor` representing applying `body_fn` iteratively `n` times.\n \"\"\"\n with tf.compat.v1.name_scope(name, 'smart_for_loop',\n [loop_num_iter, initial_loop_vars]):\n loop_num_iter_ = tf.get_static_value(loop_num_iter)\n if (loop_num_iter_ is None or tf.executing_eagerly() or\n control_flow_util.GraphOrParentsInXlaContext(\n tf.compat.v1.get_default_graph())):\n # Cast to int32 to run the comparison against i in host memory,\n # where while/LoopCond needs it.\n loop_num_iter = tf.cast(loop_num_iter, dtype=tf.int32)\n return tf.while_loop(\n cond=lambda i, *args: i < loop_num_iter,\n body=lambda i, *args: [i + 1] + list(body_fn(*args)),\n loop_vars=[np.int32(0)] + initial_loop_vars,\n parallel_iterations=parallel_iterations\n )[1:]\n result = initial_loop_vars\n for _ in range(loop_num_iter_):\n result = body_fn(*result)\n return result", "response": "Constructs a for loop."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef trace_scan(loop_fn,\n initial_state,\n elems,\n trace_fn,\n parallel_iterations=10,\n name=None):\n \"\"\"A simplified version of `tf.scan` that has configurable tracing.\n\n This function repeatedly calls `loop_fn(state, elem)`, where `state` is the\n `initial_state` during the first iteration, and the return value of `loop_fn`\n for every iteration thereafter. `elem` is a slice of `elements` along the\n first dimension, accessed in order. Additionally, it calls `trace_fn` on the\n return value of `loop_fn`. The `Tensor`s in return values of `trace_fn` are\n stacked and returned from this function, such that the first dimension of\n those `Tensor`s matches the size of `elems`.\n\n Args:\n loop_fn: A callable that takes in a `Tensor` or a nested collection of\n `Tensor`s with the same structure as `initial_state`, a slice of `elems`\n and returns the same structure as `initial_state`.\n initial_state: A `Tensor` or a nested collection of `Tensor`s passed to\n `loop_fn` in the first iteration.\n elems: A `Tensor` that is split along the first dimension and each element\n of which is passed to `loop_fn`.\n trace_fn: A callable that takes in the return value of `loop_fn` and returns\n a `Tensor` or a nested collection of `Tensor`s.\n parallel_iterations: Passed to the internal `tf.while_loop`.\n name: Name scope used in this function. Default: 'trace_scan'.\n\n Returns:\n final_state: The final return value of `loop_fn`.\n trace: The same structure as the return value of `trace_fn`, but with each\n `Tensor` being a stack of the corresponding `Tensors` in the return value\n of `trace_fn` for each slice of `elems`.\n \"\"\"\n with tf.compat.v1.name_scope(\n name, 'trace_scan', [initial_state, elems]), tf.compat.v1.variable_scope(\n tf.compat.v1.get_variable_scope()) as vs:\n if vs.caching_device is None and not tf.executing_eagerly():\n vs.set_caching_device(lambda op: op.device)\n\n initial_state = tf.nest.map_structure(\n lambda x: tf.convert_to_tensor(value=x, name='initial_state'),\n initial_state)\n elems = tf.convert_to_tensor(value=elems, name='elems')\n\n static_length = elems.shape[0]\n if tf.compat.dimension_value(static_length) is None:\n length = tf.shape(input=elems)[0]\n else:\n length = tf.convert_to_tensor(\n value=static_length, dtype=tf.int32, name='length')\n\n # This is an TensorArray in part because of XLA, which had trouble with\n # non-statically known indices. I.e. elems[i] errored, but\n # elems_array.read(i) worked.\n elems_array = tf.TensorArray(\n elems.dtype, size=length, element_shape=elems.shape[1:])\n elems_array = elems_array.unstack(elems)\n\n trace_arrays = tf.nest.map_structure(\n lambda x: tf.TensorArray(x.dtype, size=length, element_shape=x.shape),\n trace_fn(initial_state))\n\n def _body(i, state, trace_arrays):\n state = loop_fn(state, elems_array.read(i))\n trace_arrays = tf.nest.pack_sequence_as(trace_arrays, [\n a.write(i, v) for a, v in zip(\n tf.nest.flatten(trace_arrays), tf.nest.flatten(trace_fn(state)))\n ])\n return i + 1, state, trace_arrays\n\n _, final_state, trace_arrays = tf.while_loop(\n cond=lambda i, *args: i < length,\n body=_body,\n loop_vars=(0, initial_state, trace_arrays),\n parallel_iterations=parallel_iterations)\n\n stacked_trace = tf.nest.map_structure(lambda x: x.stack(), trace_arrays)\n\n # Restore the static length if we know it.\n def _merge_static_length(x):\n x.set_shape(tf.TensorShape(static_length).concatenate(x.shape[1:]))\n return x\n\n stacked_trace = tf.nest.map_structure(_merge_static_length, stacked_trace)\n return final_state, stacked_trace", "response": "A simplified version of tf. scan that has configurable tracing."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nwrapping a setter so it applies to the first n - kernel results in the first n - kernel result.", "response": "def make_innermost_setter(setter):\n \"\"\"Wraps a setter so it applies to the inner-most results in `kernel_results`.\n\n The wrapped setter unwraps `kernel_results` and applies `setter` to the first\n results without an `inner_results` attribute.\n\n Args:\n setter: A callable that takes the kernel results as well as some `*args` and\n `**kwargs` and returns a modified copy of those kernel results.\n\n Returns:\n new_setter: A wrapped `setter`.\n \"\"\"\n\n @functools.wraps(setter)\n def _new_setter(kernel_results, *args, **kwargs):\n \"\"\"Wrapped setter.\"\"\"\n results_stack = []\n while hasattr(kernel_results, 'inner_results'):\n results_stack.append(kernel_results)\n kernel_results = kernel_results.inner_results\n\n new_kernel_results = setter(kernel_results, *args, **kwargs)\n for outer_results in reversed(results_stack):\n new_kernel_results = outer_results._replace(\n inner_results=new_kernel_results)\n\n return new_kernel_results\n\n return _new_setter"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nwrap a getter so it applies to the first results in kernel_results.", "response": "def make_innermost_getter(getter):\n \"\"\"Wraps a getter so it applies to the inner-most results in `kernel_results`.\n\n The wrapped getter unwraps `kernel_results` and returns the return value of\n `getter` called with the first results without an `inner_results` attribute.\n\n Args:\n getter: A callable that takes Kernel results and returns some value.\n\n Returns:\n new_getter: A wrapped `getter`.\n \"\"\"\n\n @functools.wraps(getter)\n def _new_getter(kernel_results, *args, **kwargs):\n \"\"\"Wrapped getter.\"\"\"\n results_stack = []\n while hasattr(kernel_results, 'inner_results'):\n results_stack.append(kernel_results)\n kernel_results = kernel_results.inner_results\n\n return getter(kernel_results, *args, **kwargs)\n\n return _new_getter"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nenabling the store_parameters_in_results parameter in a chain of kernels.", "response": "def enable_store_parameters_in_results(kernel):\n \"\"\"Enables the `store_parameters_in_results` parameter in a chain of kernels.\n\n This is a temporary utility for use during the transition period of the\n parameter storage methods.\n\n Args:\n kernel: A TransitionKernel.\n\n Returns:\n kernel: The same kernel, but recreated with `store_parameters_in_results`\n recursively set to `True` in its parameters and its inner kernels (as\n appropriate).\n \"\"\"\n kernel_stack = []\n while hasattr(kernel, 'parameters') and 'inner_kernel' in kernel.parameters:\n kernel_stack.append(kernel)\n kernel = kernel.parameters['inner_kernel']\n\n def _recreate_kernel(kernel, parameters):\n new_parameters = kernel.parameters.copy()\n new_parameters.update(parameters)\n if 'store_parameters_in_results' in new_parameters:\n new_parameters['store_parameters_in_results'] = True\n with deprecation.silence():\n return type(kernel)(**new_parameters)\n\n if hasattr(kernel, 'parameters'):\n kernel = _recreate_kernel(kernel, {})\n\n for outer_kernel in reversed(kernel_stack):\n outer_kernel = _recreate_kernel(outer_kernel, {'inner_kernel': kernel})\n kernel = outer_kernel\n\n return kernel"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nreplace the rightmost dims in a Tensor with the same contents as input_shape except for the event dims in event_shape_in event_shape_out.", "response": "def _replace_event_shape_in_shape_tensor(\n input_shape, event_shape_in, event_shape_out, validate_args):\n \"\"\"Replaces the rightmost dims in a `Tensor` representing a shape.\n\n Args:\n input_shape: a rank-1 `Tensor` of integers\n event_shape_in: the event shape expected to be present in rightmost dims\n of `shape_in`.\n event_shape_out: the event shape with which to replace `event_shape_in` in\n the rightmost dims of `input_shape`.\n validate_args: Python `bool` indicating whether arguments should\n be checked for correctness.\n\n Returns:\n output_shape: A rank-1 integer `Tensor` with the same contents as\n `input_shape` except for the event dims, which are replaced with\n `event_shape_out`.\n \"\"\"\n output_tensorshape, is_validated = _replace_event_shape_in_tensorshape(\n tensorshape_util.constant_value_as_shape(input_shape),\n event_shape_in,\n event_shape_out)\n\n # TODO(b/124240153): Remove map(tf.identity, deps) once tf.function\n # correctly supports control_dependencies.\n validation_dependencies = (\n map(tf.identity, (event_shape_in, event_shape_out))\n if validate_args else ())\n\n if (tensorshape_util.is_fully_defined(output_tensorshape) and\n (is_validated or not validate_args)):\n with tf.control_dependencies(validation_dependencies):\n output_shape = tf.convert_to_tensor(\n value=output_tensorshape, name='output_shape', dtype_hint=tf.int32)\n return output_shape, output_tensorshape\n\n with tf.control_dependencies(validation_dependencies):\n event_shape_in_ndims = (\n tf.size(input=event_shape_in)\n if tensorshape_util.num_elements(event_shape_in.shape) is None else\n tensorshape_util.num_elements(event_shape_in.shape))\n input_non_event_shape, input_event_shape = tf.split(\n input_shape, num_or_size_splits=[-1, event_shape_in_ndims])\n\n additional_assertions = []\n if is_validated:\n pass\n elif validate_args:\n # Check that `input_event_shape` and `event_shape_in` are compatible in the\n # sense that they have equal entries in any position that isn't a `-1` in\n # `event_shape_in`. Note that our validations at construction time ensure\n # there is at most one such entry in `event_shape_in`.\n mask = event_shape_in >= 0\n explicit_input_event_shape = tf.boolean_mask(\n tensor=input_event_shape, mask=mask)\n explicit_event_shape_in = tf.boolean_mask(\n tensor=event_shape_in, mask=mask)\n additional_assertions.append(\n assert_util.assert_equal(\n explicit_input_event_shape,\n explicit_event_shape_in,\n message='Input `event_shape` does not match `event_shape_in`.'))\n # We don't explicitly additionally verify\n # `tf.size(input_shape) > tf.size(event_shape_in)` since `tf.split`\n # already makes this assertion.\n\n with tf.control_dependencies(additional_assertions):\n output_shape = tf.concat([input_non_event_shape, event_shape_out], axis=0,\n name='output_shape')\n\n return output_shape, output_tensorshape"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\ncheck that a shape Tensor is int - type and otherwise sane.", "response": "def _maybe_check_valid_shape(shape, validate_args):\n \"\"\"Check that a shape Tensor is int-type and otherwise sane.\"\"\"\n if not dtype_util.is_integer(shape.dtype):\n raise TypeError('{} dtype ({}) should be `int`-like.'.format(\n shape, dtype_util.name(shape.dtype)))\n\n assertions = []\n\n message = '`{}` rank should be <= 1.'\n if tensorshape_util.rank(shape.shape) is not None:\n if tensorshape_util.rank(shape.shape) > 1:\n raise ValueError(message.format(shape))\n elif validate_args:\n assertions.append(assert_util.assert_less(\n tf.rank(shape), 2, message=message.format(shape)))\n\n shape_ = tf.get_static_value(shape)\n\n message = '`{}` elements must have at most one `-1`.'\n if shape_ is not None:\n if sum(shape_ == -1) > 1:\n raise ValueError(message.format(shape))\n elif validate_args:\n assertions.append(assert_util.assert_less(\n tf.reduce_sum(input_tensor=tf.cast(tf.equal(shape, -1), tf.int32)),\n 2,\n message=message.format(shape)))\n\n message = '`{}` elements must be either positive integers or `-1`.'\n if shape_ is not None:\n if np.any(shape_ < -1):\n raise ValueError(message.format(shape))\n elif validate_args:\n assertions.append(assert_util.assert_greater(\n shape, -2, message=message.format(shape)))\n\n return assertions"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\ncalculate the batchwise KL divergence KL with d1 and d2 Beta.", "response": "def _kl_beta_beta(d1, d2, name=None):\n \"\"\"Calculate the batchwise KL divergence KL(d1 || d2) with d1 and d2 Beta.\n\n Args:\n d1: instance of a Beta distribution object.\n d2: instance of a Beta distribution object.\n name: (optional) Name to use for created operations.\n default is \"kl_beta_beta\".\n\n Returns:\n Batchwise KL(d1 || d2)\n \"\"\"\n def delta(fn, is_property=True):\n fn1 = getattr(d1, fn)\n fn2 = getattr(d2, fn)\n return (fn2 - fn1) if is_property else (fn2() - fn1())\n\n with tf.name_scope(name or \"kl_beta_beta\"):\n return (delta(\"_log_normalization\", is_property=False) -\n tf.math.digamma(d1.concentration1) * delta(\"concentration1\") -\n tf.math.digamma(d1.concentration0) * delta(\"concentration0\") +\n (tf.math.digamma(d1.total_concentration) *\n delta(\"total_concentration\")))"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _maybe_assert_valid_sample(self, x):\n if not self.validate_args:\n return x\n return distribution_util.with_dependencies([\n assert_util.assert_positive(x, message=\"sample must be positive\"),\n assert_util.assert_less(x, 1., message=\"sample must be less than `1`.\"),\n ], x)", "response": "Checks the validity of a sample."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef converged_any(converged, failed):\n return (tf.reduce_any(input_tensor=converged) |\n tf.reduce_all(input_tensor=failed))", "response": "Condition to stop when any batch member converges."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\nreturn a dictionary of initial state arguments for the current object.", "response": "def get_initial_state_args(value_and_gradients_function,\n initial_position,\n grad_tolerance,\n control_inputs=None):\n \"\"\"Returns a dictionary to populate the initial state of the search procedure.\n\n Performs an initial convergence check and the first evaluation of the\n objective function.\n\n Args:\n value_and_gradients_function: A Python callable that accepts a tensor and\n returns a tuple of two tensors: the objective function value and its\n derivative.\n initial_position: The starting point of the search procedure.\n grad_tolerance: The gradient tolerance for the procedure.\n control_inputs: Optional ops used to assert the validity of inputs, these\n are added as control dependencies to execute before the objective\n function is evaluated for the first time.\n\n Returns:\n An dictionary with values for the following keys:\n converged: True if the convergence check finds that the initial position\n is already an argmin of the objective function.\n failed: Initialized to False.\n num_objective_evaluations: Initialized to 1.\n position: Initialized to the initial position.\n objective_value: Initialized to the value of the objective function at\n the initial position.\n objective_gradient: Initialized to the gradient of the objective\n function at the initial position.\n \"\"\"\n if control_inputs:\n with tf.control_dependencies(control_inputs):\n f0, df0 = value_and_gradients_function(initial_position)\n else:\n f0, df0 = value_and_gradients_function(initial_position)\n converged = norm(df0, dims=1) < grad_tolerance\n return dict(\n converged=converged,\n failed=tf.zeros_like(converged), # i.e. False.\n num_iterations=tf.convert_to_tensor(value=0),\n num_objective_evaluations=tf.convert_to_tensor(value=1),\n position=initial_position,\n objective_value=f0,\n objective_gradient=df0)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nperforming the line search step of the BFGS search procedure. Uses hager_zhang line search procedure to compute a suitable step size to advance the current `state.position` along the given `search_direction`. Also, if the line search is successful, updates the `state.position` by taking the corresponding step. Args: state: A namedtuple instance holding values for the current state of the search procedure. The state must include the fields: `position`, `objective_value`, `objective_gradient`, `num_iterations`, `num_objective_evaluations`, `converged` and `failed`. value_and_gradients_function: A Python callable that accepts a point as a real `Tensor` of shape `[..., n]` and returns a tuple of two tensors of the same dtype: the objective function value, a real `Tensor` of shape `[...]`, and its derivative, another real `Tensor` of shape `[..., n]`. search_direction: A real `Tensor` of shape `[..., n]`. The direction along which to perform line search. grad_tolerance: Scalar `Tensor` of real dtype. Specifies the gradient tolerance for the procedure. f_relative_tolerance: Scalar `Tensor` of real dtype. Specifies the tolerance for the relative change in the objective value. x_tolerance: Scalar `Tensor` of real dtype. Specifies the tolerance for the change in the position. stopping_condition: A Python function that takes as input two Boolean tensors of shape `[...]`, and returns a Boolean scalar tensor. The input tensors are `converged` and `failed`, indicating the current status of each respective batch member; the return value states whether the algorithm should stop. Returns: A copy of the input state with the following fields updated: converged: a Boolean `Tensor` of shape `[...]` indicating whether the convergence criteria has been met. failed: a Boolean `Tensor` of shape `[...]` indicating whether the line search procedure failed to converge, or if either the updated gradient or objective function are no longer finite. num_iterations: Increased by 1. num_objective_evaluations: Increased by the number of times that the objective function got evaluated. position, objective_value, objective_gradient: If line search succeeded, updated by computing the new position and evaluating the objective function at that position.", "response": "def line_search_step(state, value_and_gradients_function, search_direction,\n grad_tolerance, f_relative_tolerance, x_tolerance,\n stopping_condition):\n \"\"\"Performs the line search step of the BFGS search procedure.\n\n Uses hager_zhang line search procedure to compute a suitable step size\n to advance the current `state.position` along the given `search_direction`.\n Also, if the line search is successful, updates the `state.position` by\n taking the corresponding step.\n\n Args:\n state: A namedtuple instance holding values for the current state of the\n search procedure. The state must include the fields: `position`,\n `objective_value`, `objective_gradient`, `num_iterations`,\n `num_objective_evaluations`, `converged` and `failed`.\n value_and_gradients_function: A Python callable that accepts a point as a\n real `Tensor` of shape `[..., n]` and returns a tuple of two tensors of\n the same dtype: the objective function value, a real `Tensor` of shape\n `[...]`, and its derivative, another real `Tensor` of shape `[..., n]`.\n search_direction: A real `Tensor` of shape `[..., n]`. The direction along\n which to perform line search.\n grad_tolerance: Scalar `Tensor` of real dtype. Specifies the gradient\n tolerance for the procedure.\n f_relative_tolerance: Scalar `Tensor` of real dtype. Specifies the\n tolerance for the relative change in the objective value.\n x_tolerance: Scalar `Tensor` of real dtype. Specifies the tolerance for the\n change in the position.\n stopping_condition: A Python function that takes as input two Boolean\n tensors of shape `[...]`, and returns a Boolean scalar tensor. The input\n tensors are `converged` and `failed`, indicating the current status of\n each respective batch member; the return value states whether the\n algorithm should stop.\n Returns:\n A copy of the input state with the following fields updated:\n converged: a Boolean `Tensor` of shape `[...]` indicating whether the\n convergence criteria has been met.\n failed: a Boolean `Tensor` of shape `[...]` indicating whether the line\n search procedure failed to converge, or if either the updated gradient\n or objective function are no longer finite.\n num_iterations: Increased by 1.\n num_objective_evaluations: Increased by the number of times that the\n objective function got evaluated.\n position, objective_value, objective_gradient: If line search succeeded,\n updated by computing the new position and evaluating the objective\n function at that position.\n \"\"\"\n line_search_value_grad_func = _restrict_along_direction(\n value_and_gradients_function, state.position, search_direction)\n derivative_at_start_pt = tf.reduce_sum(\n input_tensor=state.objective_gradient * search_direction, axis=-1)\n val_0 = ValueAndGradient(x=_broadcast(0, state.position),\n f=state.objective_value,\n df=derivative_at_start_pt,\n full_gradient=state.objective_gradient)\n inactive = state.failed | state.converged\n ls_result = linesearch.hager_zhang(\n line_search_value_grad_func,\n initial_step_size=_broadcast(1, state.position),\n value_at_zero=val_0,\n converged=inactive) # No search needed for these.\n\n state_after_ls = update_fields(\n state,\n failed=state.failed | ~ls_result.converged,\n num_iterations=state.num_iterations + 1,\n num_objective_evaluations=(\n state.num_objective_evaluations + ls_result.func_evals))\n\n def _do_update_position():\n # For inactive batch members `left.x` is zero. However, their\n # `search_direction` might also be undefined, so we can't rely on\n # multiplication by zero to produce a `position_delta` of zero.\n position_delta = tf.where(\n inactive,\n tf.zeros_like(search_direction),\n search_direction * tf.expand_dims(ls_result.left.x, axis=-1))\n return _update_position(\n state_after_ls,\n position_delta,\n ls_result.left.f,\n ls_result.left.full_gradient,\n grad_tolerance, f_relative_tolerance, x_tolerance)\n\n return prefer_static.cond(\n stopping_condition(state.converged, state.failed),\n true_fn=lambda: state_after_ls,\n false_fn=_do_update_position)"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _restrict_along_direction(value_and_gradients_function,\n position,\n direction):\n \"\"\"Restricts a function in n-dimensions to a given direction.\n\n Suppose f: R^n -> R. Then given a point x0 and a vector p0 in R^n, the\n restriction of the function along that direction is defined by:\n\n ```None\n g(t) = f(x0 + t * p0)\n ```\n\n This function performs this restriction on the given function. In addition, it\n also computes the gradient of the restricted function along the restriction\n direction. This is equivalent to computing `dg/dt` in the definition above.\n\n Args:\n value_and_gradients_function: Callable accepting a single real `Tensor`\n argument of shape `[..., n]` and returning a tuple of a real `Tensor` of\n shape `[...]` and a real `Tensor` of shape `[..., n]`. The multivariate\n function whose restriction is to be computed. The output values of the\n callable are the function value and the gradients at the input argument.\n position: `Tensor` of real dtype and shape consumable by\n `value_and_gradients_function`. Corresponds to `x0` in the definition\n above.\n direction: `Tensor` of the same dtype and shape as `position`. The direction\n along which to restrict the function. Note that the direction need not\n be a unit vector.\n\n Returns:\n restricted_value_and_gradients_func: A callable accepting a tensor of shape\n broadcastable to `[...]` and same dtype as `position` and returning a\n namedtuple of `Tensors`. The input tensor is the parameter along the\n direction labelled `t` above. The return value contains fields:\n x: A real `Tensor` of shape `[...]`. The input value `t` where the line\n function was evaluated, after any necessary broadcasting.\n f: A real `Tensor` of shape `[...]` containing the value of the\n function at the point `position + t * direction`.\n df: A real `Tensor` of shape `[...]` containing the derivative at\n `position + t * direction`.\n full_gradient: A real `Tensor` of shape `[..., n]`, the full gradient\n of the original `value_and_gradients_function`.\n \"\"\"\n def _restricted_func(t):\n t = _broadcast(t, position)\n pt = position + tf.expand_dims(t, axis=-1) * direction\n objective_value, gradient = value_and_gradients_function(pt)\n return ValueAndGradient(\n x=t,\n f=objective_value,\n df=tf.reduce_sum(input_tensor=gradient * direction, axis=-1),\n full_gradient=gradient)\n\n return _restricted_func", "response": "Restrict a function to a given direction."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef _update_position(state,\n position_delta,\n next_objective,\n next_gradient,\n grad_tolerance,\n f_relative_tolerance,\n x_tolerance):\n \"\"\"Updates the state advancing its position by a given position_delta.\"\"\"\n failed = state.failed | ~tf.math.is_finite(next_objective) | ~tf.reduce_all(\n input_tensor=tf.math.is_finite(next_gradient), axis=-1)\n\n next_position = state.position + position_delta\n converged = ~failed & _check_convergence(state.position,\n next_position,\n state.objective_value,\n next_objective,\n next_gradient,\n grad_tolerance,\n f_relative_tolerance,\n x_tolerance)\n return update_fields(\n state,\n converged=state.converged | converged,\n failed=failed,\n position=next_position,\n objective_value=next_objective,\n objective_gradient=next_gradient)", "response": "Updates the state advancing its position by a given position_delta."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef norm(value, dims, order=None):\n if dims == 0:\n return tf.math.abs(value)\n elif dims == 1:\n axis = -1\n elif dims == 2:\n axis = [-1, -2]\n else:\n ValueError(dims)\n if order is None:\n order = np.inf\n return tf.norm(tensor=value, axis=axis, ord=order)", "response": "Compute the norm of the given value."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef _check_convergence(current_position,\n next_position,\n current_objective,\n next_objective,\n next_gradient,\n grad_tolerance,\n f_relative_tolerance,\n x_tolerance):\n \"\"\"Checks if the algorithm satisfies the convergence criteria.\"\"\"\n grad_converged = norm(next_gradient, dims=1) <= grad_tolerance\n x_converged = norm(next_position - current_position, dims=1) <= x_tolerance\n f_converged = (norm(next_objective - current_objective, dims=0) <=\n f_relative_tolerance * current_objective)\n return grad_converged | x_converged | f_converged", "response": "Checks if the algorithm satisfies the convergence criteria."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef _broadcast(value, target):\n return tf.broadcast_to(\n tf.convert_to_tensor(value=value, dtype=target.dtype),\n distribution_util.prefer_static_shape(target)[:-1])", "response": "Broadcasts a value to match the batching dimensions of a target."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef _harmonic_number(x):\n one = tf.ones([], dtype=x.dtype)\n return tf.math.digamma(x + one) - tf.math.digamma(one)", "response": "Compute the harmonic number from its analytic continuation."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef _moment(self, n):\n total_concentration = self.concentration1 + self.concentration0\n expanded_concentration1 = tf.ones_like(\n total_concentration, dtype=self.dtype) * self.concentration1\n expanded_concentration0 = tf.ones_like(\n total_concentration, dtype=self.dtype) * self.concentration0\n beta_arg0 = 1 + n / expanded_concentration1\n beta_arg = tf.stack([beta_arg0, expanded_concentration0], -1)\n log_moment = tf.math.log(expanded_concentration0) + tf.math.lbeta(beta_arg)\n return tf.exp(log_moment)", "response": "Compute the n th uncentered moment."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _maybe_validate_target_accept_prob(target_accept_prob, validate_args):\n if not validate_args:\n return target_accept_prob\n with tf.control_dependencies([\n tf.compat.v1.assert_positive(\n target_accept_prob, message='`target_accept_prob` must be > 0.'),\n tf.compat.v1.assert_less(\n target_accept_prob,\n tf.ones_like(target_accept_prob),\n message='`target_accept_prob` must be < 1.')\n ]):\n return tf.identity(target_accept_prob)", "response": "Validates that target_accept_prob is in range 0 1."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\ndefault exchange proposal function for replica exchange MC.", "response": "def default_exchange_proposed_fn(prob_exchange):\n \"\"\"Default exchange proposal function, for replica exchange MC.\n\n With probability `prob_exchange` propose combinations of replica for exchange.\n When exchanging, create combinations of adjacent replicas in\n [Replica Exchange Monte Carlo](\n https://en.wikipedia.org/wiki/Parallel_tempering)\n\n ```\n exchange_fn = default_exchange_proposed_fn(prob_exchange=0.5)\n exchange_proposed = exchange_fn(num_replica=3)\n\n exchange_proposed.eval()\n ==> [[0, 1]] # 1 exchange, 0 <--> 1\n\n exchange_proposed.eval()\n ==> [] # 0 exchanges\n ```\n\n Args:\n prob_exchange: Scalar `Tensor` giving probability that any exchanges will\n be generated.\n\n Returns:\n default_exchange_proposed_fn_: Python callable which take a number of\n replicas (a Python integer), and return combinations of replicas for\n exchange as an [n, 2] integer `Tensor`, `0 <= n <= num_replica // 2`,\n with *unique* values in the set `{0, ..., num_replica}`.\n \"\"\"\n\n def default_exchange_proposed_fn_(num_replica, seed=None):\n \"\"\"Default function for `exchange_proposed_fn` of `kernel`.\"\"\"\n seed_stream = distributions.SeedStream(seed, 'default_exchange_proposed_fn')\n\n zero_start = tf.random.uniform([], seed=seed_stream()) > 0.5\n if num_replica % 2 == 0:\n\n def _exchange():\n flat_exchange = tf.range(num_replica)\n if num_replica > 2:\n start = tf.cast(~zero_start, dtype=tf.int32)\n end = num_replica - start\n flat_exchange = flat_exchange[start:end]\n return tf.reshape(flat_exchange, [tf.size(input=flat_exchange) // 2, 2])\n else:\n\n def _exchange():\n start = tf.cast(zero_start, dtype=tf.int32)\n end = num_replica - tf.cast(~zero_start, dtype=tf.int32)\n flat_exchange = tf.range(num_replica)[start:end]\n return tf.reshape(flat_exchange, [tf.size(input=flat_exchange) // 2, 2])\n\n def _null_exchange():\n return tf.reshape(tf.cast([], dtype=tf.int32), shape=[0, 2])\n\n return tf.cond(\n pred=tf.random.uniform([], seed=seed_stream()) < prob_exchange,\n true_fn=_exchange,\n false_fn=_null_exchange)\n\n return default_exchange_proposed_fn_"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _get_field(kernel_results, field_name):\n if hasattr(kernel_results, field_name):\n return getattr(kernel_results, field_name)\n if hasattr(kernel_results, 'accepted_results'):\n return getattr(kernel_results.accepted_results, field_name)\n raise TypeError('Cannot extract %s from %s' % (field_name, kernel_results))", "response": "Get a field from kernel_results or kernel_results. accepted_results."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\ntake one step of the TransitionKernel.", "response": "def one_step(self, current_state, previous_kernel_results):\n \"\"\"Takes one step of the TransitionKernel.\n\n Args:\n current_state: `Tensor` or Python `list` of `Tensor`s representing the\n current state(s) of the Markov chain(s).\n previous_kernel_results: A (possibly nested) `tuple`, `namedtuple` or\n `list` of `Tensor`s representing internal calculations made within the\n previous call to this function (or as returned by `bootstrap_results`).\n\n Returns:\n next_state: `Tensor` or Python `list` of `Tensor`s representing the\n next state(s) of the Markov chain(s).\n kernel_results: A (possibly nested) `tuple`, `namedtuple` or `list` of\n `Tensor`s representing internal calculations made within this function.\n This inculdes replica states.\n \"\"\"\n # Key difficulty: The type of exchanges differs from one call to the\n # next...even the number of exchanges can differ.\n # As a result, exchanges must happen dynamically, in while loops.\n with tf.compat.v1.name_scope(\n name=mcmc_util.make_name(self.name, 'remc', 'one_step'),\n values=[current_state, previous_kernel_results]):\n\n # Each replica does `one_step` to get pre-exchange states/KernelResults.\n sampled_replica_states, sampled_replica_results = zip(*[\n rk.one_step(previous_kernel_results.replica_states[i],\n previous_kernel_results.replica_results[i])\n for i, rk in enumerate(self.replica_kernels)\n ])\n sampled_replica_states = list(sampled_replica_states)\n sampled_replica_results = list(sampled_replica_results)\n\n states_are_lists = mcmc_util.is_list_like(sampled_replica_states[0])\n\n if not states_are_lists:\n sampled_replica_states = [[s] for s in sampled_replica_states]\n num_state_parts = len(sampled_replica_states[0])\n\n dtype = sampled_replica_states[0][0].dtype\n\n # Must put states into TensorArrays. Why? We will read/write states\n # dynamically with Tensor index `i`, and you cannot do this with lists.\n # old_states[k][i] is Tensor of (old) state part k, for replica i.\n # The `k` will be known statically, and `i` is a Tensor.\n old_states = [\n tf.TensorArray(\n dtype,\n size=self.num_replica,\n dynamic_size=False,\n clear_after_read=False,\n tensor_array_name='old_states',\n # State part k has same shape, regardless of replica. So use 0.\n element_shape=sampled_replica_states[0][k].shape)\n for k in range(num_state_parts)\n ]\n for k in range(num_state_parts):\n for i in range(self.num_replica):\n old_states[k] = old_states[k].write(i, sampled_replica_states[i][k])\n\n exchange_proposed = self.exchange_proposed_fn(\n self.num_replica, seed=self._seed_stream())\n exchange_proposed_n = tf.shape(input=exchange_proposed)[0]\n\n exchanged_states = self._get_exchanged_states(\n old_states, exchange_proposed, exchange_proposed_n,\n sampled_replica_states, sampled_replica_results)\n\n no_exchange_proposed, _ = tf.compat.v1.setdiff1d(\n tf.range(self.num_replica), tf.reshape(exchange_proposed, [-1]))\n\n exchanged_states = self._insert_old_states_where_no_exchange_was_proposed(\n no_exchange_proposed, old_states, exchanged_states)\n\n next_replica_states = []\n for i in range(self.num_replica):\n next_replica_states_i = []\n for k in range(num_state_parts):\n next_replica_states_i.append(exchanged_states[k].read(i))\n next_replica_states.append(next_replica_states_i)\n\n if not states_are_lists:\n next_replica_states = [s[0] for s in next_replica_states]\n sampled_replica_states = [s[0] for s in sampled_replica_states]\n\n # Now that states are/aren't exchanged, bootstrap next kernel_results.\n # The viewpoint is that after each exchange, we are starting anew.\n next_replica_results = [\n rk.bootstrap_results(state)\n for rk, state in zip(self.replica_kernels, next_replica_states)\n ]\n\n next_state = next_replica_states[0] # Replica 0 is the returned state(s).\n\n kernel_results = ReplicaExchangeMCKernelResults(\n replica_states=next_replica_states,\n replica_results=next_replica_results,\n sampled_replica_states=sampled_replica_states,\n sampled_replica_results=sampled_replica_results,\n )\n\n return next_state, kernel_results"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\ngetting the exchanged states.", "response": "def _get_exchanged_states(self, old_states, exchange_proposed,\n exchange_proposed_n, sampled_replica_states,\n sampled_replica_results):\n \"\"\"Get list of TensorArrays holding exchanged states, and zeros.\"\"\"\n with tf.compat.v1.name_scope('get_exchanged_states'):\n\n target_log_probs = []\n for replica in range(self.num_replica):\n replica_log_prob = _get_field(sampled_replica_results[replica],\n 'target_log_prob')\n inverse_temp = self.inverse_temperatures[replica]\n target_log_probs.append(replica_log_prob / inverse_temp)\n target_log_probs = tf.stack(target_log_probs, axis=0)\n\n dtype = target_log_probs.dtype\n num_state_parts = len(sampled_replica_states[0])\n # exchanged_states[k][i] is Tensor of (new) state part k, for replica i.\n # The `k` will be known statically, and `i` is a Tensor.\n # We will insert values into indices `i` for every replica with a proposed\n # exchange.\n exchanged_states = [\n tf.TensorArray(\n dtype,\n size=self.num_replica,\n dynamic_size=False,\n tensor_array_name='exchanged_states',\n # State part k has same shape, regardless of replica. So use 0.\n element_shape=sampled_replica_states[0][k].shape)\n for k in range(num_state_parts)\n ]\n\n # Draw random variables here, to avoid sampling in the loop (and losing\n # reproducibility). This may mean we sample too many, but we will always\n # have enough.\n sample_shape = tf.concat(\n ([self.num_replica // 2], tf.shape(input=target_log_probs)[1:]),\n axis=0)\n log_uniforms = tf.math.log(\n tf.random.uniform(\n shape=sample_shape, dtype=dtype, seed=self._seed_stream()))\n\n def _swap(is_exchange_accepted, x, y):\n \"\"\"Swap batches of x, y where accepted.\"\"\"\n with tf.compat.v1.name_scope('swap_where_exchange_accepted'):\n new_x = mcmc_util.choose(is_exchange_accepted, y, x)\n new_y = mcmc_util.choose(is_exchange_accepted, x, y)\n return new_x, new_y\n\n def cond(i, unused_exchanged_states):\n return i < exchange_proposed_n\n\n def body(i, exchanged_states):\n \"\"\"Body of while loop for exchanging states.\"\"\"\n # Propose exchange between replicas indexed by m and n.\n m, n = tf.unstack(exchange_proposed[i])\n\n # Construct log_accept_ratio: -temp_diff * target_log_prob_diff.\n # Note target_log_prob_diff = -EnergyDiff (common definition is in terms\n # of energy).\n temp_diff = self.inverse_temperatures[m] - self.inverse_temperatures[n]\n # Difference of target log probs may be +- Inf or NaN. We want the\n # product of this with the temperature difference to have \"alt value\" of\n # -Inf.\n log_accept_ratio = mcmc_util.safe_sum(\n [-temp_diff * target_log_probs[m], temp_diff * target_log_probs[n]])\n\n is_exchange_accepted = log_uniforms[i] < log_accept_ratio\n\n for k in range(num_state_parts):\n new_m, new_n = _swap(is_exchange_accepted, old_states[k].read(m),\n old_states[k].read(n))\n exchanged_states[k] = exchanged_states[k].write(m, new_m)\n exchanged_states[k] = exchanged_states[k].write(n, new_n)\n\n return i + 1, exchanged_states\n\n # At this point, exchanged_states[k] is a length num_replicas TensorArray.\n return tf.while_loop(\n cond=cond, body=body, loop_vars=[tf.constant(0),\n exchanged_states])[1]"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef bootstrap_results(self, init_state):\n with tf.compat.v1.name_scope(\n name=mcmc_util.make_name(self.name, 'remc', 'bootstrap_results'),\n values=[init_state]):\n replica_results = [\n self.replica_kernels[i].bootstrap_results(init_state)\n for i in range(self.num_replica)\n ]\n\n init_state_parts = (\n list(init_state)\n if mcmc_util.is_list_like(init_state) else [init_state])\n\n # Convert all states parts to tensor...\n replica_states = [[\n tf.convert_to_tensor(value=s) for s in init_state_parts\n ] for i in range(self.num_replica)]\n\n if not mcmc_util.is_list_like(init_state):\n replica_states = [s[0] for s in replica_states]\n\n return ReplicaExchangeMCKernelResults(\n replica_states=replica_states,\n replica_results=replica_results,\n sampled_replica_states=replica_states,\n sampled_replica_results=replica_results,\n )", "response": "Returns an object with the same type as returned by one_step."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef _variance_scale_term(self):\n # Expand back the last dim so the shape of _variance_scale_term matches the\n # shape of self.concentration.\n c0 = self.total_concentration[..., tf.newaxis]\n return tf.sqrt((1. + c0 / self.total_count[..., tf.newaxis]) / (1. + c0))", "response": "Helper to _covariance and _variance which computes a shared scale."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nchecks the validity of the concentration parameter.", "response": "def _maybe_assert_valid_concentration(self, concentration, validate_args):\n \"\"\"Checks the validity of the concentration parameter.\"\"\"\n if not validate_args:\n return concentration\n concentration = distribution_util.embed_check_categorical_event_shape(\n concentration)\n return distribution_util.with_dependencies([\n assert_util.assert_positive(\n concentration, message=\"Concentration parameter must be positive.\"),\n ], concentration)"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nchecks counts for proper shape values then return tensor version.", "response": "def _maybe_assert_valid_sample(self, counts):\n \"\"\"Check counts for proper shape, values, then return tensor version.\"\"\"\n if not self.validate_args:\n return counts\n counts = distribution_util.embed_check_nonnegative_integer_form(counts)\n return distribution_util.with_dependencies([\n assert_util.assert_equal(\n self.total_count,\n tf.reduce_sum(input_tensor=counts, axis=-1),\n message=\"counts last-dimension must sum to `self.total_count`\"),\n ], counts)"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nmaking a function which applies a list of Bijectors s log_det_jacobians.", "response": "def forward_log_det_jacobian_fn(bijector):\n \"\"\"Makes a function which applies a list of Bijectors' `log_det_jacobian`s.\"\"\"\n if not mcmc_util.is_list_like(bijector):\n bijector = [bijector]\n\n def fn(transformed_state_parts, event_ndims):\n return sum([\n b.forward_log_det_jacobian(sp, event_ndims=e)\n for b, e, sp in zip(bijector, event_ndims, transformed_state_parts)\n ])\n\n return fn"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef forward_transform_fn(bijector):\n if not mcmc_util.is_list_like(bijector):\n bijector = [bijector]\n\n def fn(transformed_state_parts):\n return [b.forward(sp) for b, sp in zip(bijector, transformed_state_parts)]\n\n return fn", "response": "Makes a function which applies a list of Bijectors s forward s."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nmakes a function which applies a list of Bijectors s inverse s.", "response": "def inverse_transform_fn(bijector):\n \"\"\"Makes a function which applies a list of Bijectors' `inverse`s.\"\"\"\n if not mcmc_util.is_list_like(bijector):\n bijector = [bijector]\n def fn(state_parts):\n return [b.inverse(sp)\n for b, sp in zip(bijector, state_parts)]\n return fn"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef one_step(self, current_state, previous_kernel_results):\n with tf.compat.v1.name_scope(\n name=mcmc_util.make_name(self.name, 'transformed_kernel', 'one_step'),\n values=[previous_kernel_results]):\n transformed_next_state, kernel_results = self._inner_kernel.one_step(\n previous_kernel_results.transformed_state,\n previous_kernel_results.inner_results)\n transformed_next_state_parts = (\n transformed_next_state\n if mcmc_util.is_list_like(transformed_next_state) else\n [transformed_next_state])\n next_state_parts = self._forward_transform(transformed_next_state_parts)\n next_state = (\n next_state_parts if mcmc_util.is_list_like(transformed_next_state)\n else next_state_parts[0])\n kernel_results = TransformedTransitionKernelResults(\n transformed_state=transformed_next_state,\n inner_results=kernel_results)\n return next_state, kernel_results", "response": "Runs one iteration of the Transformed Kernel."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nreturning a tuple containing the bootstrap results of the current state of the Markov chain.", "response": "def bootstrap_results(self, init_state=None, transformed_init_state=None):\n \"\"\"Returns an object with the same type as returned by `one_step`.\n\n Unlike other `TransitionKernel`s,\n `TransformedTransitionKernel.bootstrap_results` has the option of\n initializing the `TransformedTransitionKernelResults` from either an initial\n state, eg, requiring computing `bijector.inverse(init_state)`, or\n directly from `transformed_init_state`, i.e., a `Tensor` or list\n of `Tensor`s which is interpretted as the `bijector.inverse`\n transformed state.\n\n Args:\n init_state: `Tensor` or Python `list` of `Tensor`s representing the a\n state(s) of the Markov chain(s). Must specify `init_state` or\n `transformed_init_state` but not both.\n transformed_init_state: `Tensor` or Python `list` of `Tensor`s\n representing the a state(s) of the Markov chain(s). Must specify\n `init_state` or `transformed_init_state` but not both.\n\n Returns:\n kernel_results: A (possibly nested) `tuple`, `namedtuple` or `list` of\n `Tensor`s representing internal calculations made within this function.\n\n Raises:\n ValueError: if `inner_kernel` results doesn't contain the member\n \"target_log_prob\".\n\n #### Examples\n\n To use `transformed_init_state` in context of\n `tfp.mcmc.sample_chain`, you need to explicitly pass the\n `previous_kernel_results`, e.g.,\n\n ```python\n transformed_kernel = tfp.mcmc.TransformedTransitionKernel(...)\n init_state = ... # Doesnt matter.\n transformed_init_state = ... # Does matter.\n results, _ = tfp.mcmc.sample_chain(\n num_results=...,\n current_state=init_state,\n previous_kernel_results=transformed_kernel.bootstrap_results(\n transformed_init_state=transformed_init_state),\n kernel=transformed_kernel)\n ```\n \"\"\"\n if (init_state is None) == (transformed_init_state is None):\n raise ValueError('Must specify exactly one of `init_state` '\n 'or `transformed_init_state`.')\n with tf.compat.v1.name_scope(\n name=mcmc_util.make_name(self.name, 'transformed_kernel',\n 'bootstrap_results'),\n values=[init_state, transformed_init_state]):\n if transformed_init_state is None:\n init_state_parts = (init_state if mcmc_util.is_list_like(init_state)\n else [init_state])\n transformed_init_state_parts = self._inverse_transform(init_state_parts)\n transformed_init_state = (\n transformed_init_state_parts if mcmc_util.is_list_like(init_state)\n else transformed_init_state_parts[0])\n else:\n if mcmc_util.is_list_like(transformed_init_state):\n transformed_init_state = [\n tf.convert_to_tensor(value=s, name='transformed_init_state')\n for s in transformed_init_state\n ]\n else:\n transformed_init_state = tf.convert_to_tensor(\n value=transformed_init_state, name='transformed_init_state')\n kernel_results = TransformedTransitionKernelResults(\n transformed_state=transformed_init_state,\n inner_results=self._inner_kernel.bootstrap_results(\n transformed_init_state))\n return kernel_results"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nlikes tf. where but works on namedtuples.", "response": "def val_where(cond, tval, fval):\n \"\"\"Like tf.where but works on namedtuples.\"\"\"\n if isinstance(tval, tf.Tensor):\n return tf.where(cond, tval, fval)\n elif isinstance(tval, tuple):\n cls = type(tval)\n return cls(*(val_where(cond, t, f) for t, f in zip(tval, fval)))\n else:\n raise Exception(TypeError)"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef secant2(value_and_gradients_function,\n val_0,\n search_interval,\n f_lim,\n sufficient_decrease_param=0.1,\n curvature_param=0.9,\n name=None):\n \"\"\"Performs the secant square procedure of Hager Zhang.\n\n Given an interval that brackets a root, this procedure performs an update of\n both end points using two intermediate points generated using the secant\n interpolation. For details see the steps S1-S4 in [Hager and Zhang (2006)][2].\n\n The interval [a, b] must satisfy the opposite slope conditions described in\n the documentation for `update`.\n\n Args:\n value_and_gradients_function: A Python callable that accepts a real scalar\n tensor and returns an object that can be converted to a namedtuple.\n The namedtuple should have fields 'f' and 'df' that correspond to scalar\n tensors of real dtype containing the value of the function and its\n derivative at that point. The other namedtuple fields, if present,\n should be tensors or sequences (possibly nested) of tensors.\n In usual optimization application, this function would be generated by\n projecting the multivariate objective function along some specific\n direction. The direction is determined by some other procedure but should\n be a descent direction (i.e. the derivative of the projected univariate\n function must be negative at 0.).\n Alternatively, the function may represent the batching of `n` such line\n functions (e.g. projecting a single multivariate objective function along\n `n` distinct directions at once) accepting n points as input, i.e. a\n tensor of shape [n], and the fields 'f' and 'df' in the returned\n namedtuple should each be a tensor of shape [n], with the corresponding\n function values and derivatives at the input points.\n val_0: A namedtuple, as returned by value_and_gradients_function evaluated\n at `0.`.\n search_interval: A namedtuple describing the current search interval,\n must include the fields:\n - converged: Boolean `Tensor` of shape [n], indicating batch members\n where search has already converged. Interval for these batch members\n won't be modified.\n - failed: Boolean `Tensor` of shape [n], indicating batch members\n where search has already failed. Interval for these batch members\n wont be modified.\n - iterations: Scalar int32 `Tensor`. Number of line search iterations\n so far.\n - func_evals: Scalar int32 `Tensor`. Number of function evaluations\n so far.\n - left: A namedtuple, as returned by value_and_gradients_function,\n of the left end point of the current search interval.\n - right: A namedtuple, as returned by value_and_gradients_function,\n of the right end point of the current search interval.\n f_lim: Scalar `Tensor` of real dtype. The function value threshold for\n the approximate Wolfe conditions to be checked.\n sufficient_decrease_param: Positive scalar `Tensor` of real dtype.\n Bounded above by the curvature param. Corresponds to 'delta' in the\n terminology of [Hager and Zhang (2006)][2].\n curvature_param: Positive scalar `Tensor` of real dtype. Bounded above\n by `1.`. Corresponds to 'sigma' in the terminology of\n [Hager and Zhang (2006)][2].\n name: (Optional) Python str. The name prefixed to the ops created by this\n function. If not supplied, the default name 'secant2' is used.\n\n Returns:\n A namedtuple containing the following fields.\n active: A boolean `Tensor` of shape [n]. Used internally by the procedure\n to indicate batch members on which there is work left to do.\n converged: A boolean `Tensor` of shape [n]. Indicates whether a point\n satisfying the Wolfe conditions has been found. If this is True, the\n interval will be degenerate (i.e. `left` and `right` below will be\n identical).\n failed: A boolean `Tensor` of shape [n]. Indicates if invalid function or\n gradient values were encountered (i.e. infinity or NaNs).\n num_evals: A scalar int32 `Tensor`. The total number of function\n evaluations made.\n left: Return value of value_and_gradients_function at the updated left\n end point of the interval.\n right: Return value of value_and_gradients_function at the updated right\n end point of the interval.\n \"\"\"\n with tf.compat.v1.name_scope(name, 'secant2', [\n val_0, search_interval, f_lim, sufficient_decrease_param,\n curvature_param]):\n # This will always be s.t. left <= c <= right\n val_c = value_and_gradients_function(\n _secant(search_interval.left, search_interval.right))\n failed = search_interval.failed | ~is_finite(val_c)\n converged = search_interval.converged | (~failed & _satisfies_wolfe(\n val_0, val_c, f_lim, sufficient_decrease_param, curvature_param))\n new_converged = converged & ~search_interval.converged\n val_left = val_where(new_converged, val_c, search_interval.left)\n val_right = val_where(new_converged, val_c, search_interval.right)\n\n initial_args = _Secant2Result(\n active=~failed & ~converged,\n converged=converged,\n failed=failed,\n num_evals=search_interval.func_evals + 1,\n left=val_left,\n right=val_right)\n\n def _apply_secant2_inner():\n return _secant2_inner(\n value_and_gradients_function,\n initial_args,\n val_0,\n val_c,\n f_lim,\n sufficient_decrease_param,\n curvature_param)\n\n return prefer_static.cond(\n tf.reduce_any(input_tensor=initial_args.active),\n _apply_secant2_inner,\n lambda: initial_args)", "response": "This function performs the secant square procedure of Hager Zhang."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _secant2_inner(value_and_gradients_function,\n initial_args,\n val_0,\n val_c,\n f_lim,\n sufficient_decrease_param,\n curvature_param):\n \"\"\"Helper function for secant square.\"\"\"\n # Apply the `update` function on active branch members to squeeze their\n # bracketing interval.\n update_result = update(value_and_gradients_function,\n initial_args.left,\n initial_args.right,\n val_c,\n f_lim,\n active=initial_args.active)\n\n # Update active and failed flags, update left/right on non-failed entries.\n active = initial_args.active & ~update_result.failed\n failed = initial_args.failed | update_result.failed\n val_left = val_where(active, update_result.left, initial_args.left)\n val_right = val_where(active, update_result.right, initial_args.right)\n\n # Check if new `c` points should be generated.\n updated_left = active & tf.equal(val_left.x, val_c.x)\n updated_right = active & tf.equal(val_right.x, val_c.x)\n is_new = updated_left | updated_right\n\n next_c = tf.where(updated_left,\n _secant(initial_args.left, val_left),\n val_c.x)\n next_c = tf.where(updated_right,\n _secant(initial_args.right, val_right),\n next_c)\n in_range = (val_left.x <= next_c) & (next_c <= val_right.x)\n\n # Figure out if an extra function evaluation is needed for new `c` points.\n needs_extra_eval = tf.reduce_any(input_tensor=in_range & is_new)\n num_evals = initial_args.num_evals + update_result.num_evals\n num_evals = num_evals + tf.cast(needs_extra_eval, num_evals.dtype)\n\n next_args = _Secant2Result(\n active=active & in_range, # No longer active if `c` is out of range.\n converged=initial_args.converged,\n failed=failed,\n num_evals=num_evals,\n left=val_left,\n right=val_right)\n\n def _apply_inner_update():\n next_val_c = prefer_static.cond(\n needs_extra_eval,\n (lambda: value_and_gradients_function(next_c)),\n (lambda: val_c))\n return _secant2_inner_update(\n value_and_gradients_function, next_args, val_0, next_val_c, f_lim,\n sufficient_decrease_param, curvature_param)\n\n return prefer_static.cond(\n tf.reduce_any(input_tensor=next_args.active),\n _apply_inner_update,\n lambda: next_args)", "response": "Internal function for secant 2."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nsqueezes a bracketing interval containing the minimum. Given an interval which brackets a minimum and a point in that interval, finds a smaller nested interval which also brackets the minimum. If the supplied point does not lie in the bracketing interval, the current interval is returned. The following description is given in terms of individual points evaluated on a line function to be minimized. Note, however, the implementation also accepts batches of points allowing to minimize multiple line functions at once. See details on the docstring of `value_and_gradients_function` below. The requirement of the interval bracketing a minimum is expressed through the opposite slope conditions. Assume the left end point is 'a', the right end point is 'b', the function to be minimized is 'f' and the derivative is 'df'. The update procedure relies on the following conditions being satisfied: ''' f(a) <= f(0) + epsilon (1) df(a) < 0 (2) df(b) > 0 (3) ''' In the first condition, epsilon is a small positive constant. The condition demands that the function at the left end point be not much bigger than the starting point (i.e. 0). This is an easy to satisfy condition because by assumption, we are in a direction where the function value is decreasing. The second and third conditions together demand that there is at least one zero of the derivative in between a and b. In addition to the interval, the update algorithm requires a third point to be supplied. Usually, this point would lie within the interval [a, b]. If the point is outside this interval, the current interval is returned. If the point lies within the interval, the behaviour of the function and derivative value at this point is used to squeeze the original interval in a manner that preserves the opposite slope conditions. For further details of this component, see the procedure U0-U3 on page 123 of the [Hager and Zhang (2006)][2] article. Note that this function does not explicitly verify whether the opposite slope conditions are satisfied for the supplied interval. It is assumed that this is so. Args: value_and_gradients_function: A Python callable that accepts a real scalar tensor and returns an object that can be converted to a namedtuple. The namedtuple should have fields 'f' and 'df' that correspond to scalar tensors of real dtype containing the value of the function and its derivative at that point. The other namedtuple fields, if present, should be tensors or sequences (possibly nested) of tensors. In usual optimization application, this function would be generated by projecting the multivariate objective function along some specific direction. The direction is determined by some other procedure but should be a descent direction (i.e. the derivative of the projected univariate function must be negative at 0.). Alternatively, the function may represent the batching of `n` such line functions (e.g. projecting a single multivariate objective function along `n` distinct directions at once) accepting n points as input, i.e. a tensor of shape [n], and the fields 'f' and 'df' in the returned namedtuple should each be a tensor of shape [n], with the corresponding function values and derivatives at the input points. val_left: Return value of value_and_gradients_function at the left end point of the bracketing interval (labelles 'a' above). val_right: Return value of value_and_gradients_function at the right end point of the bracketing interval (labelles 'b' above). val_trial: Return value of value_and_gradients_function at the trial point to be used to shrink the interval (labelled 'c' above). f_lim: real `Tensor` of shape [n]. The function value threshold for the approximate Wolfe conditions to be checked for each batch member. active: optional boolean `Tensor` of shape [n]. Relevant in batching mode only, indicates batch members on which the update procedure should be applied. On non-active members the current left/right interval is returned unmodified. Returns: A namedtuple containing the following fields: iteration: An int32 scalar `Tensor`. The number of iterations performed by the bisect algorithm. stopped: A boolean `Tensor` of shape [n]. True for those batch members where the bisection algorithm terminated. failed: A boolean `Tensor` of shape [n]. True for those batch members where an error was encountered. num_evals: An int32 scalar `Tensor`. The number of times the objective function was evaluated. left: Return value of value_and_gradients_function at the updated left end point of the interval found. right: Return value of value_and_gradients_function at the updated right end point of the interval found.", "response": "def update(value_and_gradients_function, val_left, val_right, val_trial, f_lim,\n active=None):\n \"\"\"Squeezes a bracketing interval containing the minimum.\n\n Given an interval which brackets a minimum and a point in that interval,\n finds a smaller nested interval which also brackets the minimum. If the\n supplied point does not lie in the bracketing interval, the current interval\n is returned.\n\n The following description is given in terms of individual points evaluated on\n a line function to be minimized. Note, however, the implementation also\n accepts batches of points allowing to minimize multiple line functions at\n once. See details on the docstring of `value_and_gradients_function` below.\n\n The requirement of the interval bracketing a minimum is expressed through the\n opposite slope conditions. Assume the left end point is 'a', the right\n end point is 'b', the function to be minimized is 'f' and the derivative is\n 'df'. The update procedure relies on the following conditions being satisfied:\n\n '''\n f(a) <= f(0) + epsilon (1)\n df(a) < 0 (2)\n df(b) > 0 (3)\n '''\n\n In the first condition, epsilon is a small positive constant. The condition\n demands that the function at the left end point be not much bigger than the\n starting point (i.e. 0). This is an easy to satisfy condition because by\n assumption, we are in a direction where the function value is decreasing.\n The second and third conditions together demand that there is at least one\n zero of the derivative in between a and b.\n\n In addition to the interval, the update algorithm requires a third point to\n be supplied. Usually, this point would lie within the interval [a, b]. If the\n point is outside this interval, the current interval is returned. If the\n point lies within the interval, the behaviour of the function and derivative\n value at this point is used to squeeze the original interval in a manner that\n preserves the opposite slope conditions.\n\n For further details of this component, see the procedure U0-U3 on page 123 of\n the [Hager and Zhang (2006)][2] article.\n\n Note that this function does not explicitly verify whether the opposite slope\n conditions are satisfied for the supplied interval. It is assumed that this\n is so.\n\n Args:\n value_and_gradients_function: A Python callable that accepts a real scalar\n tensor and returns an object that can be converted to a namedtuple.\n The namedtuple should have fields 'f' and 'df' that correspond to scalar\n tensors of real dtype containing the value of the function and its\n derivative at that point. The other namedtuple fields, if present,\n should be tensors or sequences (possibly nested) of tensors.\n In usual optimization application, this function would be generated by\n projecting the multivariate objective function along some specific\n direction. The direction is determined by some other procedure but should\n be a descent direction (i.e. the derivative of the projected univariate\n function must be negative at 0.).\n Alternatively, the function may represent the batching of `n` such line\n functions (e.g. projecting a single multivariate objective function along\n `n` distinct directions at once) accepting n points as input, i.e. a\n tensor of shape [n], and the fields 'f' and 'df' in the returned\n namedtuple should each be a tensor of shape [n], with the corresponding\n function values and derivatives at the input points.\n val_left: Return value of value_and_gradients_function at the left\n end point of the bracketing interval (labelles 'a' above).\n val_right: Return value of value_and_gradients_function at the right\n end point of the bracketing interval (labelles 'b' above).\n val_trial: Return value of value_and_gradients_function at the trial point\n to be used to shrink the interval (labelled 'c' above).\n f_lim: real `Tensor` of shape [n]. The function value threshold for\n the approximate Wolfe conditions to be checked for each batch member.\n active: optional boolean `Tensor` of shape [n]. Relevant in batching mode\n only, indicates batch members on which the update procedure should be\n applied. On non-active members the current left/right interval is returned\n unmodified.\n\n Returns:\n A namedtuple containing the following fields:\n iteration: An int32 scalar `Tensor`. The number of iterations performed\n by the bisect algorithm.\n stopped: A boolean `Tensor` of shape [n]. True for those batch members\n where the bisection algorithm terminated.\n failed: A boolean `Tensor` of shape [n]. True for those batch members\n where an error was encountered.\n num_evals: An int32 scalar `Tensor`. The number of times the objective\n function was evaluated.\n left: Return value of value_and_gradients_function at the updated left\n end point of the interval found.\n right: Return value of value_and_gradients_function at the updated right\n end point of the interval found.\n \"\"\"\n # We should only update if the trial point is within the interval.\n within_range = (val_left.x < val_trial.x) & (val_trial.x < val_right.x)\n if active is not None:\n within_range = within_range & active\n\n # The new point is a valid left end point if it has negative slope\n # and the value at the point is not too large.\n valid_left = (val_trial.df < 0) & (val_trial.f <= f_lim)\n\n # If the trial point has a negative slope but the value at that point\n # is too high, bisect can narrow down an interval between the current left\n # and the trial point.\n needs_bisect = within_range & (val_trial.df < 0) & (val_trial.f > f_lim)\n\n # Note that if `~valid_left` it is because either:\n # - the slope at the trial point is positive, so it is a valid right\n # point, or\n # - the needs_bisect condition is true.\n # In both cases we want to keep the current left and replace right\n # with the trial point.\n left = val_where(within_range & valid_left, val_trial, val_left)\n right = val_where(within_range & ~valid_left, val_trial, val_right)\n\n bisect_args = _IntermediateResult(\n iteration=tf.convert_to_tensor(value=0),\n stopped=~needs_bisect,\n failed=tf.zeros_like(within_range), # i.e. all false.\n num_evals=tf.convert_to_tensor(value=0),\n left=left,\n right=right)\n return _bisect(value_and_gradients_function, bisect_args, f_lim)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nbracketing the minimum given an initial starting point. Applies the Hager Zhang bracketing algorithm to find an interval containing a region with points satisfying Wolfe conditions. Uses the supplied initial step size 'c', the right end point of the provided search interval, to find such an interval. The only condition on 'c' is that it should be positive. For more details see steps B0-B3 in [Hager and Zhang (2006)][2]. Args: value_and_gradients_function: A Python callable that accepts a real scalar tensor and returns a namedtuple containing the value filed `f` of the function and its derivative value field `df` at that point. Alternatively, the function may representthe batching of `n` such line functions (e.g. projecting a single multivariate objective function along `n` distinct directions at once) accepting n points as input, i.e. a tensor of shape [n], and return a tuple of two tensors of shape [n], the function values and the corresponding derivatives at the input points. search_interval: A namedtuple describing the current search interval, must include the fields: - converged: Boolean `Tensor` of shape [n], indicating batch members where search has already converged. Interval for these batch members wont be modified. - failed: Boolean `Tensor` of shape [n], indicating batch members where search has already failed. Interval for these batch members wont be modified. - iterations: Scalar int32 `Tensor`. Number of line search iterations so far. - func_evals: Scalar int32 `Tensor`. Number of function evaluations so far. - left: A namedtuple, as returned by value_and_gradients_function evaluated at 0, the left end point of the current interval. - right: A namedtuple, as returned by value_and_gradients_function, of the right end point of the current interval (labelled 'c' above). f_lim: real `Tensor` of shape [n]. The function value threshold for the approximate Wolfe conditions to be checked for each batch member. max_iterations: Int32 scalar `Tensor`. The maximum number of iterations permitted. The limit applies equally to all batch members. expansion_param: Scalar positive `Tensor` of real dtype. Must be greater than `1.`. Used to expand the initial interval in case it does not bracket a minimum. Returns: A namedtuple with the following fields. iteration: An int32 scalar `Tensor`. The number of iterations performed. Bounded above by `max_iterations` parameter. stopped: A boolean `Tensor` of shape [n]. True for those batch members where the algorithm terminated before reaching `max_iterations`. failed: A boolean `Tensor` of shape [n]. True for those batch members where an error was encountered during bracketing. num_evals: An int32 scalar `Tensor`. The number of times the objective function was evaluated. left: Return value of value_and_gradients_function at the updated left end point of the interval found. right: Return value of value_and_gradients_function at the updated right end point of the interval found.", "response": "def bracket(value_and_gradients_function,\n search_interval,\n f_lim,\n max_iterations,\n expansion_param=5.0):\n \"\"\"Brackets the minimum given an initial starting point.\n\n Applies the Hager Zhang bracketing algorithm to find an interval containing\n a region with points satisfying Wolfe conditions. Uses the supplied initial\n step size 'c', the right end point of the provided search interval, to find\n such an interval. The only condition on 'c' is that it should be positive.\n For more details see steps B0-B3 in [Hager and Zhang (2006)][2].\n\n Args:\n value_and_gradients_function: A Python callable that accepts a real scalar\n tensor and returns a namedtuple containing the value filed `f` of the\n function and its derivative value field `df` at that point.\n Alternatively, the function may representthe batching of `n` such line\n functions (e.g. projecting a single multivariate objective function along\n `n` distinct directions at once) accepting n points as input, i.e. a\n tensor of shape [n], and return a tuple of two tensors of shape [n], the\n function values and the corresponding derivatives at the input points.\n search_interval: A namedtuple describing the current search interval,\n must include the fields:\n - converged: Boolean `Tensor` of shape [n], indicating batch members\n where search has already converged. Interval for these batch members\n wont be modified.\n - failed: Boolean `Tensor` of shape [n], indicating batch members\n where search has already failed. Interval for these batch members\n wont be modified.\n - iterations: Scalar int32 `Tensor`. Number of line search iterations\n so far.\n - func_evals: Scalar int32 `Tensor`. Number of function evaluations\n so far.\n - left: A namedtuple, as returned by value_and_gradients_function\n evaluated at 0, the left end point of the current interval.\n - right: A namedtuple, as returned by value_and_gradients_function,\n of the right end point of the current interval (labelled 'c' above).\n f_lim: real `Tensor` of shape [n]. The function value threshold for\n the approximate Wolfe conditions to be checked for each batch member.\n max_iterations: Int32 scalar `Tensor`. The maximum number of iterations\n permitted. The limit applies equally to all batch members.\n expansion_param: Scalar positive `Tensor` of real dtype. Must be greater\n than `1.`. Used to expand the initial interval in case it does not bracket\n a minimum.\n\n Returns:\n A namedtuple with the following fields.\n iteration: An int32 scalar `Tensor`. The number of iterations performed.\n Bounded above by `max_iterations` parameter.\n stopped: A boolean `Tensor` of shape [n]. True for those batch members\n where the algorithm terminated before reaching `max_iterations`.\n failed: A boolean `Tensor` of shape [n]. True for those batch members\n where an error was encountered during bracketing.\n num_evals: An int32 scalar `Tensor`. The number of times the objective\n function was evaluated.\n left: Return value of value_and_gradients_function at the updated left\n end point of the interval found.\n right: Return value of value_and_gradients_function at the updated right\n end point of the interval found.\n \"\"\"\n already_stopped = search_interval.failed | search_interval.converged\n\n # If the slope at right end point is positive, step B1 in [2], then the given\n # initial points already bracket a minimum.\n bracketed = search_interval.right.df >= 0\n\n # Bisection is needed, step B2, if right end point almost works as a new left\n # end point but the objective value is too high.\n needs_bisect = (\n search_interval.right.df < 0) & (search_interval.right.f > f_lim)\n\n # In these three cases bracketing is already `stopped` and there is no need\n # to perform further evaluations. Otherwise the bracketing loop is needed to\n # expand the interval, step B3, until the conditions are met.\n initial_args = _IntermediateResult(\n iteration=search_interval.iterations,\n stopped=already_stopped | bracketed | needs_bisect,\n failed=search_interval.failed,\n num_evals=search_interval.func_evals,\n left=search_interval.left,\n right=search_interval.right)\n\n def _loop_cond(curr):\n return (curr.iteration <\n max_iterations) & ~tf.reduce_all(input_tensor=curr.stopped)\n\n def _loop_body(curr):\n \"\"\"Main body of bracketing loop.\"\"\"\n # The loop maintains the invariant that curr.stopped is true if we have\n # either: failed, successfully bracketed, or not yet bracketed but needs\n # bisect. On the only remaining case, step B3 in [2]. case we need to\n # expand and update the left/right values appropriately.\n new_right = value_and_gradients_function(expansion_param * curr.right.x)\n left = val_where(curr.stopped, curr.left, curr.right)\n right = val_where(curr.stopped, curr.right, new_right)\n\n # Updated the failed, bracketed, and needs_bisect conditions.\n failed = curr.failed | ~is_finite(right)\n bracketed = right.df >= 0\n needs_bisect = (right.df < 0) & (right.f > f_lim)\n return [_IntermediateResult(\n iteration=curr.iteration + 1,\n stopped=curr.stopped | failed | bracketed | needs_bisect,\n failed=failed,\n num_evals=curr.num_evals + 1,\n left=left,\n right=right)]\n\n bracket_result = tf.while_loop(\n cond=_loop_cond, body=_loop_body, loop_vars=[initial_args])[0]\n\n # For entries where bisect is still needed, mark them as not yet stopped,\n # reset the left end point, and run `_bisect` on them.\n needs_bisect = (\n (bracket_result.right.df < 0) & (bracket_result.right.f > f_lim))\n stopped = already_stopped | bracket_result.failed | ~needs_bisect\n left = val_where(stopped, bracket_result.left, search_interval.left)\n bisect_args = bracket_result._replace(stopped=stopped, left=left)\n return _bisect(value_and_gradients_function, bisect_args, f_lim)"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nbisect an interval and updates to satisfy opposite slope conditions.", "response": "def bisect(value_and_gradients_function,\n initial_left,\n initial_right,\n f_lim):\n \"\"\"Bisects an interval and updates to satisfy opposite slope conditions.\n\n Corresponds to the step U3 in [Hager and Zhang (2006)][2].\n\n Args:\n value_and_gradients_function: A Python callable that accepts a real scalar\n tensor and returns a namedtuple containing the value filed `f` of the\n function and its derivative value field `df` at that point.\n Alternatively, the function may representthe batching of `n` such line\n functions (e.g. projecting a single multivariate objective function along\n `n` distinct directions at once) accepting n points as input, i.e. a\n tensor of shape [n], and return a tuple of two tensors of shape [n], the\n function values and the corresponding derivatives at the input points.\n initial_left: Return value of value_and_gradients_function at the left end\n point of the current bracketing interval.\n initial_right: Return value of value_and_gradients_function at the right end\n point of the current bracketing interval.\n f_lim: real `Tensor` of shape [n]. The function value threshold for\n the approximate Wolfe conditions to be checked for each batch member.\n\n Returns:\n A namedtuple containing the following fields:\n iteration: An int32 scalar `Tensor`. The number of iterations performed.\n Bounded above by `max_iterations` parameter.\n stopped: A boolean scalar `Tensor`. True if the bisect algorithm\n terminated.\n failed: A scalar boolean tensor. Indicates whether the objective function\n failed to produce a finite value.\n num_evals: A scalar int32 tensor. The number of value and gradients\n function evaluations.\n left: Return value of value_and_gradients_function at the left end\n point of the bracketing interval found.\n right: Return value of value_and_gradients_function at the right end\n point of the bracketing interval found.\n \"\"\"\n failed = ~is_finite(initial_left, initial_right)\n needs_bisect = (initial_right.df < 0) & (initial_right.f > f_lim)\n bisect_args = _IntermediateResult(\n iteration=tf.convert_to_tensor(value=0),\n stopped=failed | ~needs_bisect,\n failed=failed,\n num_evals=tf.convert_to_tensor(value=0),\n left=initial_left,\n right=initial_right)\n return _bisect(value_and_gradients_function, bisect_args, f_lim)"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nchecking if the supplied values are finite.", "response": "def is_finite(val_1, val_2=None):\n \"\"\"Checks if the supplied values are finite.\n\n Args:\n val_1: A namedtuple instance with the function value and derivative,\n as returned e.g. by value_and_gradients_function evaluations.\n val_2: (Optional) A namedtuple instance with the function value and\n derivative, as returned e.g. by value_and_gradients_function evaluations.\n\n Returns:\n is_finite: Scalar boolean `Tensor` indicating whether the function value\n and the derivative in `val_1` (and optionally in `val_2`) are all finite.\n \"\"\"\n val_1_finite = tf.math.is_finite(val_1.f) & tf.math.is_finite(val_1.df)\n if val_2 is not None:\n return val_1_finite & tf.math.is_finite(val_2.f) & tf.math.is_finite(\n val_2.df)\n return val_1_finite"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nchecking whether the Wolfe or approx Wolfe conditions are satisfied.", "response": "def _satisfies_wolfe(val_0,\n val_c,\n f_lim,\n sufficient_decrease_param,\n curvature_param):\n \"\"\"Checks whether the Wolfe or approx Wolfe conditions are satisfied.\n\n The Wolfe conditions are a set of stopping criteria for an inexact line search\n algorithm. Let f(a) be the function value along the search direction and\n df(a) the derivative along the search direction evaluated a distance 'a'.\n Here 'a' is the distance along the search direction. The Wolfe conditions are:\n\n ```None\n f(a) <= f(0) + delta * a * df(0) (Armijo/Sufficient decrease condition)\n df(a) >= sigma * df(0) (Weak curvature condition)\n ```\n `delta` and `sigma` are two user supplied parameters satisfying:\n `0 < delta < sigma <= 1.`. In the following, delta is called\n `sufficient_decrease_param` and sigma is called `curvature_param`.\n\n On a finite precision machine, the Wolfe conditions are difficult to satisfy\n when one is close to the minimum. Hence, Hager-Zhang propose replacing\n the sufficient decrease condition with the following condition on the\n derivative in the vicinity of a minimum.\n\n ```None\n df(a) <= (2 * delta - 1) * df(0) (Approx Wolfe sufficient decrease)\n ```\n This condition is only used if one is near the minimum. This is tested using\n\n ```None\n f(a) <= f(0) + epsilon * |f(0)|\n ```\n The following function checks both the Wolfe and approx Wolfe conditions.\n Here, `epsilon` is a small positive constant. In the following, the argument\n `f_lim` corresponds to the product: epsilon * |f(0)|.\n\n Args:\n val_0: A namedtuple, as returned by value_and_gradients_function\n evaluated at 0.\n val_c: A namedtuple, as returned by value_and_gradients_function\n evaluated at the point to be tested.\n f_lim: Scalar `Tensor` of real dtype. The function value threshold for\n the approximate Wolfe conditions to be checked.\n sufficient_decrease_param: Positive scalar `Tensor` of real dtype.\n Bounded above by the curvature param. Corresponds to 'delta' in the\n terminology of [Hager and Zhang (2006)][2].\n curvature_param: Positive scalar `Tensor` of real dtype. Bounded above\n by `1.`. Corresponds to 'sigma' in the terminology of\n [Hager Zhang (2005)][1].\n\n Returns:\n is_satisfied: A scalar boolean `Tensor` which is True if either the\n Wolfe or approximate Wolfe conditions are satisfied.\n \"\"\"\n exact_wolfe_suff_dec = (sufficient_decrease_param * val_0.df >=\n (val_c.f - val_0.f) / val_c.x)\n wolfe_curvature = val_c.df >= curvature_param * val_0.df\n exact_wolfe = exact_wolfe_suff_dec & wolfe_curvature\n approx_wolfe_applies = val_c.f <= f_lim\n approx_wolfe_suff_dec = ((2 * sufficient_decrease_param - 1) * val_0.df\n >= val_c.df)\n approx_wolfe = approx_wolfe_applies & approx_wolfe_suff_dec & wolfe_curvature\n is_satisfied = exact_wolfe | approx_wolfe\n return is_satisfied"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _secant(val_a, val_b):\n return (val_a.x * val_b.df - val_b.x * val_a.df) / (val_b.df - val_a.df)", "response": "Calculates the secant interpolation for the minimum."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef make_simple_step_size_update_policy(num_adaptation_steps,\n target_rate=0.75,\n decrement_multiplier=0.01,\n increment_multiplier=0.01,\n step_counter=None):\n \"\"\"Create a function implementing a step-size update policy.\n\n The simple policy increases or decreases the `step_size_var` based on the\n average of `exp(minimum(0., log_accept_ratio))`. It is based on\n [Section 4.2 of Andrieu and Thoms (2008)](\n https://people.eecs.berkeley.edu/~jordan/sail/readings/andrieu-thoms.pdf).\n\n The `num_adaptation_steps` argument is set independently of any burnin\n for the overall chain. In general, adaptation prevents the chain from\n reaching a stationary distribution, so obtaining consistent samples requires\n `num_adaptation_steps` be set to a value [somewhat smaller](\n http://andrewgelman.com/2017/12/15/burn-vs-warm-iterative-simulation-algorithms/#comment-627745)\n than the number of burnin steps. However, it may sometimes be helpful to set\n `num_adaptation_steps` to a larger value during development in order to\n inspect the behavior of the chain during adaptation.\n\n Args:\n num_adaptation_steps: Scalar `int` `Tensor` number of initial steps to\n during which to adjust the step size. This may be greater, less than, or\n equal to the number of burnin steps. If `None`, the step size is adapted\n on every step (note this breaks stationarity of the chain!).\n target_rate: Scalar `Tensor` representing desired `accept_ratio`.\n Default value: `0.75` (i.e., [center of asymptotically optimal\n rate](https://arxiv.org/abs/1411.6669)).\n decrement_multiplier: `Tensor` representing amount to downscale current\n `step_size`.\n Default value: `0.01`.\n increment_multiplier: `Tensor` representing amount to upscale current\n `step_size`.\n Default value: `0.01`.\n step_counter: Scalar `int` `Variable` specifying the current step. The step\n size is adapted iff `step_counter < num_adaptation_steps`.\n Default value: if `None`, an internal variable\n `step_size_adaptation_step_counter` is created and initialized to `-1`.\n\n Returns:\n step_size_simple_update_fn: Callable that takes args\n `step_size_var, kernel_results` and returns updated step size(s).\n \"\"\"\n if step_counter is None and num_adaptation_steps is not None:\n step_counter = tf.compat.v1.get_variable(\n name='step_size_adaptation_step_counter',\n initializer=np.array(-1, dtype=np.int32),\n # Specify the dtype for variable sharing to work correctly\n # (b/120599991).\n dtype=tf.int32,\n trainable=False,\n use_resource=True)\n\n def step_size_simple_update_fn(step_size_var, kernel_results):\n \"\"\"Updates (list of) `step_size` using a standard adaptive MCMC procedure.\n\n Args:\n step_size_var: (List of) `tf.Variable`s representing the per `state_part`\n HMC `step_size`.\n kernel_results: `collections.namedtuple` containing `Tensor`s\n representing values from most recent call to `one_step`.\n\n Returns:\n step_size_assign: (List of) `Tensor`(s) representing updated\n `step_size_var`(s).\n \"\"\"\n\n if kernel_results is None:\n if mcmc_util.is_list_like(step_size_var):\n return [tf.identity(ss) for ss in step_size_var]\n return tf.identity(step_size_var)\n log_n = tf.math.log(\n tf.cast(\n tf.size(input=kernel_results.log_accept_ratio),\n kernel_results.log_accept_ratio.dtype))\n log_mean_accept_ratio = tf.reduce_logsumexp(\n input_tensor=tf.minimum(kernel_results.log_accept_ratio, 0.)) - log_n\n adjustment = tf.where(\n log_mean_accept_ratio < tf.cast(\n tf.math.log(target_rate), log_mean_accept_ratio.dtype),\n -decrement_multiplier / (1. + decrement_multiplier),\n increment_multiplier)\n\n def build_assign_op():\n if mcmc_util.is_list_like(step_size_var):\n return [\n ss.assign_add(ss * tf.cast(adjustment, ss.dtype))\n for ss in step_size_var\n ]\n return step_size_var.assign_add(\n step_size_var * tf.cast(adjustment, step_size_var.dtype))\n\n if num_adaptation_steps is None:\n return build_assign_op()\n else:\n with tf.control_dependencies([step_counter.assign_add(1)]):\n return tf.cond(\n pred=step_counter < num_adaptation_steps,\n true_fn=build_assign_op,\n false_fn=lambda: step_size_var)\n\n return step_size_simple_update_fn", "response": "Creates a function implementing a simple step - size update policy."} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\napplies num_leapfrog_steps of the leapfrog integrator.", "response": "def _leapfrog_integrator_one_step(\n target_log_prob_fn,\n independent_chain_ndims,\n step_sizes,\n current_momentum_parts,\n current_state_parts,\n current_target_log_prob,\n current_target_log_prob_grad_parts,\n state_gradients_are_stopped=False,\n name=None):\n \"\"\"Applies `num_leapfrog_steps` of the leapfrog integrator.\n\n Assumes a simple quadratic kinetic energy function: `0.5 ||momentum||**2`.\n\n #### Examples:\n\n ##### Simple quadratic potential.\n\n ```python\n import matplotlib.pyplot as plt\n %matplotlib inline\n import numpy as np\n import tensorflow as tf\n from tensorflow_probability.python.mcmc.hmc import _leapfrog_integrator_one_step # pylint: disable=line-too-long\n tfd = tfp.distributions\n\n dims = 10\n num_iter = int(1e3)\n dtype = np.float32\n\n position = tf.placeholder(np.float32)\n momentum = tf.placeholder(np.float32)\n\n target_log_prob_fn = tfd.MultivariateNormalDiag(\n loc=tf.zeros(dims, dtype)).log_prob\n\n def _leapfrog_one_step(*args):\n # Closure representing computation done during each leapfrog step.\n return _leapfrog_integrator_one_step(\n target_log_prob_fn=target_log_prob_fn,\n independent_chain_ndims=0,\n step_sizes=[0.1],\n current_momentum_parts=args[0],\n current_state_parts=args[1],\n current_target_log_prob=args[2],\n current_target_log_prob_grad_parts=args[3])\n\n # Do leapfrog integration.\n [\n [next_momentum],\n [next_position],\n next_target_log_prob,\n next_target_log_prob_grad_parts,\n ] = tf.while_loop(\n cond=lambda *args: True,\n body=_leapfrog_one_step,\n loop_vars=[\n [momentum],\n [position],\n target_log_prob_fn(position),\n tf.gradients(target_log_prob_fn(position), position),\n ],\n maximum_iterations=3)\n\n momentum_ = np.random.randn(dims).astype(dtype)\n position_ = np.random.randn(dims).astype(dtype)\n positions = np.zeros([num_iter, dims], dtype)\n\n with tf.Session() as sess:\n for i in xrange(num_iter):\n position_, momentum_ = sess.run(\n [next_momentum, next_position],\n feed_dict={position: position_, momentum: momentum_})\n positions[i] = position_\n\n plt.plot(positions[:, 0]); # Sinusoidal.\n ```\n\n Args:\n target_log_prob_fn: Python callable which takes an argument like\n `*current_state_parts` and returns its (possibly unnormalized) log-density\n under the target distribution.\n independent_chain_ndims: Scalar `int` `Tensor` representing the number of\n leftmost `Tensor` dimensions which index independent chains.\n step_sizes: Python `list` of `Tensor`s representing the step size for the\n leapfrog integrator. Must broadcast with the shape of\n `current_state_parts`. Larger step sizes lead to faster progress, but\n too-large step sizes make rejection exponentially more likely. When\n possible, it's often helpful to match per-variable step sizes to the\n standard deviations of the target distribution in each variable.\n current_momentum_parts: Tensor containing the value(s) of the momentum\n variable(s) to update.\n current_state_parts: Python `list` of `Tensor`s representing the current\n state(s) of the Markov chain(s). The first `independent_chain_ndims` of\n the `Tensor`(s) index different chains.\n current_target_log_prob: `Tensor` representing the value of\n `target_log_prob_fn(*current_state_parts)`. The only reason to specify\n this argument is to reduce TF graph size.\n current_target_log_prob_grad_parts: Python list of `Tensor`s representing\n gradient of `target_log_prob_fn(*current_state_parts`) wrt\n `current_state_parts`. Must have same shape as `current_state_parts`. The\n only reason to specify this argument is to reduce TF graph size.\n state_gradients_are_stopped: Python `bool` indicating that the proposed new\n state be run through `tf.stop_gradient`. This is particularly useful when\n combining optimization over samples from the HMC chain.\n Default value: `False` (i.e., do not apply `stop_gradient`).\n name: Python `str` name prefixed to Ops created by this function.\n Default value: `None` (i.e., 'hmc_leapfrog_integrator').\n\n Returns:\n proposed_momentum_parts: Updated value of the momentum.\n proposed_state_parts: Tensor or Python list of `Tensor`s representing the\n state(s) of the Markov chain(s) at each result step. Has same shape as\n input `current_state_parts`.\n proposed_target_log_prob: `Tensor` representing the value of\n `target_log_prob_fn` at `next_state`.\n proposed_target_log_prob_grad_parts: Gradient of `proposed_target_log_prob`\n wrt `next_state`.\n\n Raises:\n ValueError: if `len(momentum_parts) != len(state_parts)`.\n ValueError: if `len(state_parts) != len(step_sizes)`.\n ValueError: if `len(state_parts) != len(grads_target_log_prob)`.\n TypeError: if `not target_log_prob.dtype.is_floating`.\n \"\"\"\n # Note on per-variable step sizes:\n #\n # Using per-variable step sizes is equivalent to using the same step\n # size for all variables and adding a diagonal mass matrix in the\n # kinetic energy term of the Hamiltonian being integrated. This is\n # hinted at by Neal (2011) but not derived in detail there.\n #\n # Let x and v be position and momentum variables respectively.\n # Let g(x) be the gradient of `target_log_prob_fn(x)`.\n # Let S be a diagonal matrix of per-variable step sizes.\n # Let the Hamiltonian H(x, v) = -target_log_prob_fn(x) + 0.5 * ||v||**2.\n #\n # Using per-variable step sizes gives the updates\n # v' = v + 0.5 * matmul(S, g(x))\n # x'' = x + matmul(S, v')\n # v'' = v' + 0.5 * matmul(S, g(x''))\n #\n # Let u = matmul(inv(S), v).\n # Multiplying v by inv(S) in the updates above gives the transformed dynamics\n # u' = matmul(inv(S), v') = matmul(inv(S), v) + 0.5 * g(x)\n # = u + 0.5 * g(x)\n # x'' = x + matmul(S, v') = x + matmul(S**2, u')\n # u'' = matmul(inv(S), v'') = matmul(inv(S), v') + 0.5 * g(x'')\n # = u' + 0.5 * g(x'')\n #\n # These are exactly the leapfrog updates for the Hamiltonian\n # H'(x, u) = -target_log_prob_fn(x) + 0.5 * u^T S**2 u\n # = -target_log_prob_fn(x) + 0.5 * ||v||**2 = H(x, v).\n #\n # To summarize:\n #\n # * Using per-variable step sizes implicitly simulates the dynamics\n # of the Hamiltonian H' (which are energy-conserving in H'). We\n # keep track of v instead of u, but the underlying dynamics are\n # the same if we transform back.\n # * The value of the Hamiltonian H'(x, u) is the same as the value\n # of the original Hamiltonian H(x, v) after we transform back from\n # u to v.\n # * Sampling v ~ N(0, I) is equivalent to sampling u ~ N(0, S**-2).\n #\n # So using per-variable step sizes in HMC will give results that are\n # exactly identical to explicitly using a diagonal mass matrix.\n\n with tf.compat.v1.name_scope(name, 'hmc_leapfrog_integrator_one_step', [\n independent_chain_ndims, step_sizes, current_momentum_parts,\n current_state_parts, current_target_log_prob,\n current_target_log_prob_grad_parts\n ]):\n\n # Step 1: Update momentum.\n proposed_momentum_parts = [\n v + 0.5 * tf.cast(eps, v.dtype) * g\n for v, eps, g\n in zip(current_momentum_parts,\n step_sizes,\n current_target_log_prob_grad_parts)]\n\n # Step 2: Update state.\n proposed_state_parts = [\n x + tf.cast(eps, v.dtype) * v\n for x, eps, v\n in zip(current_state_parts,\n step_sizes,\n proposed_momentum_parts)]\n\n if state_gradients_are_stopped:\n proposed_state_parts = [tf.stop_gradient(x) for x in proposed_state_parts]\n\n # Step 3a: Re-evaluate target-log-prob (and grad) at proposed state.\n [\n proposed_target_log_prob,\n proposed_target_log_prob_grad_parts,\n ] = mcmc_util.maybe_call_fn_and_grads(\n target_log_prob_fn,\n proposed_state_parts)\n\n if not proposed_target_log_prob.dtype.is_floating:\n raise TypeError('`target_log_prob_fn` must produce a `Tensor` '\n 'with `float` `dtype`.')\n\n if any(g is None for g in proposed_target_log_prob_grad_parts):\n raise ValueError(\n 'Encountered `None` gradient. Does your target `target_log_prob_fn` '\n 'access all `tf.Variable`s via `tf.get_variable`?\\n'\n ' current_state_parts: {}\\n'\n ' proposed_state_parts: {}\\n'\n ' proposed_target_log_prob_grad_parts: {}'.format(\n current_state_parts,\n proposed_state_parts,\n proposed_target_log_prob_grad_parts))\n\n # Step 3b: Update momentum (again).\n proposed_momentum_parts = [\n v + 0.5 * tf.cast(eps, v.dtype) * g\n for v, eps, g\n in zip(proposed_momentum_parts,\n step_sizes,\n proposed_target_log_prob_grad_parts)]\n\n return [\n proposed_momentum_parts,\n proposed_state_parts,\n proposed_target_log_prob,\n proposed_target_log_prob_grad_parts,\n ]"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef _compute_log_acceptance_correction(current_momentums,\n proposed_momentums,\n independent_chain_ndims,\n name=None):\n \"\"\"Helper to `kernel` which computes the log acceptance-correction.\n\n A sufficient but not necessary condition for the existence of a stationary\n distribution, `p(x)`, is \"detailed balance\", i.e.:\n\n ```none\n p(x'|x) p(x) = p(x|x') p(x')\n ```\n\n In the Metropolis-Hastings algorithm, a state is proposed according to\n `g(x'|x)` and accepted according to `a(x'|x)`, hence\n `p(x'|x) = g(x'|x) a(x'|x)`.\n\n Inserting this into the detailed balance equation implies:\n\n ```none\n g(x'|x) a(x'|x) p(x) = g(x|x') a(x|x') p(x')\n ==> a(x'|x) / a(x|x') = p(x') / p(x) [g(x|x') / g(x'|x)] (*)\n ```\n\n One definition of `a(x'|x)` which satisfies (*) is:\n\n ```none\n a(x'|x) = min(1, p(x') / p(x) [g(x|x') / g(x'|x)])\n ```\n\n (To see that this satisfies (*), notice that under this definition only at\n most one `a(x'|x)` and `a(x|x') can be other than one.)\n\n We call the bracketed term the \"acceptance correction\".\n\n In the case of UncalibratedHMC, the log acceptance-correction is not the log\n proposal-ratio. UncalibratedHMC augments the state-space with momentum, z.\n Assuming a standard Gaussian distribution for momentums, the chain eventually\n converges to:\n\n ```none\n p([x, z]) propto= target_prob(x) exp(-0.5 z**2)\n ```\n\n Relating this back to Metropolis-Hastings parlance, for HMC we have:\n\n ```none\n p([x, z]) propto= target_prob(x) exp(-0.5 z**2)\n g([x, z] | [x', z']) = g([x', z'] | [x, z])\n ```\n\n In other words, the MH bracketed term is `1`. However, because we desire to\n use a general MH framework, we can place the momentum probability ratio inside\n the metropolis-correction factor thus getting an acceptance probability:\n\n ```none\n target_prob(x')\n accept_prob(x'|x) = ----------------- [exp(-0.5 z**2) / exp(-0.5 z'**2)]\n target_prob(x)\n ```\n\n (Note: we actually need to handle the kinetic energy change at each leapfrog\n step, but this is the idea.)\n\n Args:\n current_momentums: `Tensor` representing the value(s) of the current\n momentum(s) of the state (parts).\n proposed_momentums: `Tensor` representing the value(s) of the proposed\n momentum(s) of the state (parts).\n independent_chain_ndims: Scalar `int` `Tensor` representing the number of\n leftmost `Tensor` dimensions which index independent chains.\n name: Python `str` name prefixed to Ops created by this function.\n Default value: `None` (i.e., 'compute_log_acceptance_correction').\n\n Returns:\n log_acceptance_correction: `Tensor` representing the `log`\n acceptance-correction. (See docstring for mathematical definition.)\n \"\"\"\n with tf.compat.v1.name_scope(\n name, 'compute_log_acceptance_correction',\n [independent_chain_ndims, current_momentums, proposed_momentums]):\n log_current_kinetic, log_proposed_kinetic = [], []\n for current_momentum, proposed_momentum in zip(\n current_momentums, proposed_momentums):\n axis = tf.range(independent_chain_ndims, tf.rank(current_momentum))\n log_current_kinetic.append(_log_sum_sq(current_momentum, axis))\n log_proposed_kinetic.append(_log_sum_sq(proposed_momentum, axis))\n current_kinetic = 0.5 * tf.exp(\n tf.reduce_logsumexp(\n input_tensor=tf.stack(log_current_kinetic, axis=-1), axis=-1))\n proposed_kinetic = 0.5 * tf.exp(\n tf.reduce_logsumexp(\n input_tensor=tf.stack(log_proposed_kinetic, axis=-1), axis=-1))\n return mcmc_util.safe_sum([current_kinetic, -proposed_kinetic])", "response": "Helper function to compute the log acceptance correction."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef _log_sum_sq(x, axis=None):\n return tf.reduce_logsumexp(\n input_tensor=2. * tf.math.log(tf.abs(x)), axis=axis)", "response": "Computes log sum of x."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef one_step(self, current_state, previous_kernel_results):\n previous_step_size_assign = (\n [] if self.step_size_update_fn is None\n else (previous_kernel_results.extra.step_size_assign\n if mcmc_util.is_list_like(\n previous_kernel_results.extra.step_size_assign)\n else [previous_kernel_results.extra.step_size_assign]))\n\n with tf.control_dependencies(previous_step_size_assign):\n next_state, kernel_results = self._impl.one_step(\n current_state, previous_kernel_results)\n if self.step_size_update_fn is not None:\n step_size_assign = self.step_size_update_fn( # pylint: disable=not-callable\n self.step_size, kernel_results)\n kernel_results = kernel_results._replace(\n extra=HamiltonianMonteCarloExtraKernelResults(\n step_size_assign=step_size_assign))\n return next_state, kernel_results", "response": "Runs one iteration of Hamiltonian Monte Carlo."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef bootstrap_results(self, init_state):\n kernel_results = self._impl.bootstrap_results(init_state)\n if self.step_size_update_fn is not None:\n step_size_assign = self.step_size_update_fn(self.step_size, None) # pylint: disable=not-callable\n kernel_results = kernel_results._replace(\n extra=HamiltonianMonteCarloExtraKernelResults(\n step_size_assign=step_size_assign))\n return kernel_results", "response": "Creates initial kernel_results using a supplied state."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef bayesian_resnet(input_shape,\n num_classes=10,\n kernel_posterior_scale_mean=-9.0,\n kernel_posterior_scale_stddev=0.1,\n kernel_posterior_scale_constraint=0.2):\n \"\"\"Constructs a ResNet18 model.\n\n Args:\n input_shape: A `tuple` indicating the Tensor shape.\n num_classes: `int` representing the number of class labels.\n kernel_posterior_scale_mean: Python `int` number for the kernel\n posterior's scale (log variance) mean. The smaller the mean the closer\n is the initialization to a deterministic network.\n kernel_posterior_scale_stddev: Python `float` number for the initial kernel\n posterior's scale stddev.\n ```\n q(W|x) ~ N(mu, var),\n log_var ~ N(kernel_posterior_scale_mean, kernel_posterior_scale_stddev)\n ````\n kernel_posterior_scale_constraint: Python `float` number for the log value\n to constrain the log variance throughout training.\n i.e. log_var <= log(kernel_posterior_scale_constraint).\n\n Returns:\n tf.keras.Model.\n \"\"\"\n\n filters = [64, 128, 256, 512]\n kernels = [3, 3, 3, 3]\n strides = [1, 2, 2, 2]\n\n def _untransformed_scale_constraint(t):\n return tf.clip_by_value(t, -1000,\n tf.math.log(kernel_posterior_scale_constraint))\n\n kernel_posterior_fn = tfp.layers.default_mean_field_normal_fn(\n untransformed_scale_initializer=tf.compat.v1.initializers.random_normal(\n mean=kernel_posterior_scale_mean,\n stddev=kernel_posterior_scale_stddev),\n untransformed_scale_constraint=_untransformed_scale_constraint)\n\n image = tf.keras.layers.Input(shape=input_shape, dtype='float32')\n x = tfp.layers.Convolution2DFlipout(\n 64,\n 3,\n strides=1,\n padding='same',\n kernel_posterior_fn=kernel_posterior_fn)(image)\n\n for i in range(len(kernels)):\n x = _resnet_block(\n x,\n filters[i],\n kernels[i],\n strides[i],\n kernel_posterior_fn)\n\n x = tf.keras.layers.BatchNormalization()(x)\n x = tf.keras.layers.Activation('relu')(x)\n x = tf.keras.layers.AveragePooling2D(4, 1)(x)\n x = tf.keras.layers.Flatten()(x)\n\n x = tfp.layers.DenseFlipout(\n num_classes,\n kernel_posterior_fn=kernel_posterior_fn)(x)\n\n model = tf.keras.Model(inputs=image, outputs=x, name='resnet18')\n return model", "response": "Constructs a ResNet18 model."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _resnet_block(x, filters, kernel, stride, kernel_posterior_fn):\n x = tf.keras.layers.BatchNormalization()(x)\n x = tf.keras.layers.Activation('relu')(x)\n\n if stride != 1 or filters != x.shape[1]:\n shortcut = _projection_shortcut(x, filters, stride, kernel_posterior_fn)\n else:\n shortcut = x\n\n x = tfp.layers.Convolution2DFlipout(\n filters,\n kernel,\n strides=stride,\n padding='same',\n kernel_posterior_fn=kernel_posterior_fn)(x)\n x = tf.keras.layers.BatchNormalization()(x)\n x = tf.keras.layers.Activation('relu')(x)\n\n x = tfp.layers.Convolution2DFlipout(\n filters,\n kernel,\n strides=1,\n padding='same',\n kernel_posterior_fn=kernel_posterior_fn)(x)\n x = tf.keras.layers.add([x, shortcut])\n return x", "response": "Network block for ResNet."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef make_encoder(activation, num_topics, layer_sizes):\n encoder_net = tf.keras.Sequential()\n for num_hidden_units in layer_sizes:\n encoder_net.add(\n tf.keras.layers.Dense(\n num_hidden_units,\n activation=activation,\n kernel_initializer=tf.compat.v1.glorot_normal_initializer()))\n encoder_net.add(\n tf.keras.layers.Dense(\n num_topics,\n activation=tf.nn.softplus,\n kernel_initializer=tf.compat.v1.glorot_normal_initializer()))\n\n def encoder(bag_of_words):\n net = _clip_dirichlet_parameters(encoder_net(bag_of_words))\n return tfd.Dirichlet(concentration=net,\n name=\"topics_posterior\")\n\n return encoder", "response": "Create the encoder function."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\ncreates the decoder function.", "response": "def make_decoder(num_topics, num_words):\n \"\"\"Create the decoder function.\n\n Args:\n num_topics: The number of topics.\n num_words: The number of words.\n\n Returns:\n decoder: A `callable` mapping a `Tensor` of encodings to a\n `tfd.Distribution` instance over words.\n \"\"\"\n topics_words_logits = tf.compat.v1.get_variable(\n \"topics_words_logits\",\n shape=[num_topics, num_words],\n initializer=tf.compat.v1.glorot_normal_initializer())\n topics_words = tf.nn.softmax(topics_words_logits, axis=-1)\n\n def decoder(topics):\n word_probs = tf.matmul(topics, topics_words)\n # The observations are bag of words and therefore not one-hot. However,\n # log_prob of OneHotCategorical computes the probability correctly in\n # this case.\n return tfd.OneHotCategorical(probs=word_probs,\n name=\"bag_of_words\")\n\n return decoder, topics_words"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\ncreates the prior distribution.", "response": "def make_prior(num_topics, initial_value):\n \"\"\"Create the prior distribution.\n\n Args:\n num_topics: Number of topics.\n initial_value: The starting value for the prior parameters.\n\n Returns:\n prior: A `callable` that returns a `tf.distribution.Distribution`\n instance, the prior distribution.\n prior_variables: A `list` of `Variable` objects, the trainable parameters\n of the prior.\n \"\"\"\n def _softplus_inverse(x):\n return np.log(np.expm1(x))\n\n logit_concentration = tf.compat.v1.get_variable(\n \"logit_concentration\",\n shape=[1, num_topics],\n initializer=tf.compat.v1.initializers.constant(\n _softplus_inverse(initial_value)))\n concentration = _clip_dirichlet_parameters(\n tf.nn.softplus(logit_concentration))\n\n def prior():\n return tfd.Dirichlet(concentration=concentration,\n name=\"topics_prior\")\n\n prior_variables = [logit_concentration]\n\n return prior, prior_variables"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef model_fn(features, labels, mode, params, config):\n del labels, config\n\n encoder = make_encoder(params[\"activation\"],\n params[\"num_topics\"],\n params[\"layer_sizes\"])\n decoder, topics_words = make_decoder(params[\"num_topics\"],\n features.shape[1])\n prior, prior_variables = make_prior(params[\"num_topics\"],\n params[\"prior_initial_value\"])\n\n topics_prior = prior()\n alpha = topics_prior.concentration\n\n topics_posterior = encoder(features)\n topics = topics_posterior.sample()\n random_reconstruction = decoder(topics)\n\n reconstruction = random_reconstruction.log_prob(features)\n tf.compat.v1.summary.scalar(\"reconstruction\",\n tf.reduce_mean(input_tensor=reconstruction))\n\n # Compute the KL-divergence between two Dirichlets analytically.\n # The sampled KL does not work well for \"sparse\" distributions\n # (see Appendix D of [2]).\n kl = tfd.kl_divergence(topics_posterior, topics_prior)\n tf.compat.v1.summary.scalar(\"kl\", tf.reduce_mean(input_tensor=kl))\n\n # Ensure that the KL is non-negative (up to a very small slack).\n # Negative KL can happen due to numerical instability.\n with tf.control_dependencies(\n [tf.compat.v1.assert_greater(kl, -1e-3, message=\"kl\")]):\n kl = tf.identity(kl)\n\n elbo = reconstruction - kl\n avg_elbo = tf.reduce_mean(input_tensor=elbo)\n tf.compat.v1.summary.scalar(\"elbo\", avg_elbo)\n loss = -avg_elbo\n\n # Perform variational inference by minimizing the -ELBO.\n global_step = tf.compat.v1.train.get_or_create_global_step()\n optimizer = tf.compat.v1.train.AdamOptimizer(params[\"learning_rate\"])\n\n # This implements the \"burn-in\" for prior parameters (see Appendix D of [2]).\n # For the first prior_burn_in_steps steps they are fixed, and then trained\n # jointly with the other parameters.\n grads_and_vars = optimizer.compute_gradients(loss)\n grads_and_vars_except_prior = [\n x for x in grads_and_vars if x[1] not in prior_variables]\n\n def train_op_except_prior():\n return optimizer.apply_gradients(\n grads_and_vars_except_prior,\n global_step=global_step)\n\n def train_op_all():\n return optimizer.apply_gradients(\n grads_and_vars,\n global_step=global_step)\n\n train_op = tf.cond(\n pred=global_step < params[\"prior_burn_in_steps\"],\n true_fn=train_op_except_prior,\n false_fn=train_op_all)\n\n # The perplexity is an exponent of the average negative ELBO per word.\n words_per_document = tf.reduce_sum(input_tensor=features, axis=1)\n log_perplexity = -elbo / words_per_document\n tf.compat.v1.summary.scalar(\n \"perplexity\", tf.exp(tf.reduce_mean(input_tensor=log_perplexity)))\n (log_perplexity_tensor,\n log_perplexity_update) = tf.compat.v1.metrics.mean(log_perplexity)\n perplexity_tensor = tf.exp(log_perplexity_tensor)\n\n # Obtain the topics summary. Implemented as a py_func for simplicity.\n topics = tf.compat.v1.py_func(\n functools.partial(get_topics_strings, vocabulary=params[\"vocabulary\"]),\n [topics_words, alpha],\n tf.string,\n stateful=False)\n tf.compat.v1.summary.text(\"topics\", topics)\n\n return tf.estimator.EstimatorSpec(\n mode=mode,\n loss=loss,\n train_op=train_op,\n eval_metric_ops={\n \"elbo\": tf.compat.v1.metrics.mean(elbo),\n \"reconstruction\": tf.compat.v1.metrics.mean(reconstruction),\n \"kl\": tf.compat.v1.metrics.mean(kl),\n \"perplexity\": (perplexity_tensor, log_perplexity_update),\n \"topics\": (topics, tf.no_op()),\n },\n )", "response": "Builds the model function for use in an estimator."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef sample_chain(\n num_results,\n current_state,\n previous_kernel_results=None,\n kernel=None,\n num_burnin_steps=0,\n num_steps_between_results=0,\n trace_fn=lambda current_state, kernel_results: kernel_results,\n return_final_kernel_results=False,\n parallel_iterations=10,\n name=None,\n):\n \"\"\"Implements Markov chain Monte Carlo via repeated `TransitionKernel` steps.\n\n This function samples from an Markov chain at `current_state` and whose\n stationary distribution is governed by the supplied `TransitionKernel`\n instance (`kernel`).\n\n This function can sample from multiple chains, in parallel. (Whether or not\n there are multiple chains is dictated by the `kernel`.)\n\n The `current_state` can be represented as a single `Tensor` or a `list` of\n `Tensors` which collectively represent the current state.\n\n Since MCMC states are correlated, it is sometimes desirable to produce\n additional intermediate states, and then discard them, ending up with a set of\n states with decreased autocorrelation. See [Owen (2017)][1]. Such \"thinning\"\n is made possible by setting `num_steps_between_results > 0`. The chain then\n takes `num_steps_between_results` extra steps between the steps that make it\n into the results. The extra steps are never materialized (in calls to\n `sess.run`), and thus do not increase memory requirements.\n\n Warning: when setting a `seed` in the `kernel`, ensure that `sample_chain`'s\n `parallel_iterations=1`, otherwise results will not be reproducible.\n\n In addition to returning the chain state, this function supports tracing of\n auxiliary variables used by the kernel. The traced values are selected by\n specifying `trace_fn`. By default, all kernel results are traced but in the\n future the default will be changed to no results being traced, so plan\n accordingly. See below for some examples of this feature.\n\n Args:\n num_results: Integer number of Markov chain draws.\n current_state: `Tensor` or Python `list` of `Tensor`s representing the\n current state(s) of the Markov chain(s).\n previous_kernel_results: A `Tensor` or a nested collection of `Tensor`s\n representing internal calculations made within the previous call to this\n function (or as returned by `bootstrap_results`).\n kernel: An instance of `tfp.mcmc.TransitionKernel` which implements one step\n of the Markov chain.\n num_burnin_steps: Integer number of chain steps to take before starting to\n collect results.\n Default value: 0 (i.e., no burn-in).\n num_steps_between_results: Integer number of chain steps between collecting\n a result. Only one out of every `num_steps_between_samples + 1` steps is\n included in the returned results. The number of returned chain states is\n still equal to `num_results`. Default value: 0 (i.e., no thinning).\n trace_fn: A callable that takes in the current chain state and the previous\n kernel results and return a `Tensor` or a nested collection of `Tensor`s\n that is then traced along with the chain state.\n return_final_kernel_results: If `True`, then the final kernel results are\n returned alongside the chain state and the trace specified by the\n `trace_fn`.\n parallel_iterations: The number of iterations allowed to run in parallel. It\n must be a positive integer. See `tf.while_loop` for more details.\n name: Python `str` name prefixed to Ops created by this function.\n Default value: `None` (i.e., \"mcmc_sample_chain\").\n\n Returns:\n checkpointable_states_and_trace: if `return_final_kernel_results` is\n `True`. The return value is an instance of\n `CheckpointableStatesAndTrace`.\n all_states: if `return_final_kernel_results` is `False` and `trace_fn` is\n `None`. The return value is a `Tensor` or Python list of `Tensor`s\n representing the state(s) of the Markov chain(s) at each result step. Has\n same shape as input `current_state` but with a prepended\n `num_results`-size dimension.\n states_and_trace: if `return_final_kernel_results` is `False` and\n `trace_fn` is not `None`. The return value is an instance of\n `StatesAndTrace`.\n\n #### Examples\n\n ##### Sample from a diagonal-variance Gaussian.\n\n I.e.,\n\n ```none\n for i=1..n:\n x[i] ~ MultivariateNormal(loc=0, scale=diag(true_stddev)) # likelihood\n ```\n\n ```python\n import tensorflow as tf\n import tensorflow_probability as tfp\n tfd = tfp.distributions\n\n dims = 10\n true_stddev = np.sqrt(np.linspace(1., 3., dims))\n likelihood = tfd.MultivariateNormalDiag(loc=0., scale_diag=true_stddev)\n\n states = tfp.mcmc.sample_chain(\n num_results=1000,\n num_burnin_steps=500,\n current_state=tf.zeros(dims),\n kernel=tfp.mcmc.HamiltonianMonteCarlo(\n target_log_prob_fn=likelihood.log_prob,\n step_size=0.5,\n num_leapfrog_steps=2),\n trace_fn=None)\n\n sample_mean = tf.reduce_mean(states, axis=0)\n # ==> approx all zeros\n\n sample_stddev = tf.sqrt(tf.reduce_mean(\n tf.squared_difference(states, sample_mean),\n axis=0))\n # ==> approx equal true_stddev\n ```\n\n ##### Sampling from factor-analysis posteriors with known factors.\n\n I.e.,\n\n ```none\n # prior\n w ~ MultivariateNormal(loc=0, scale=eye(d))\n for i=1..n:\n # likelihood\n x[i] ~ Normal(loc=w^T F[i], scale=1)\n ```\n\n where `F` denotes factors.\n\n ```python\n import tensorflow as tf\n import tensorflow_probability as tfp\n tfd = tfp.distributions\n\n # Specify model.\n def make_prior(dims):\n return tfd.MultivariateNormalDiag(\n loc=tf.zeros(dims))\n\n def make_likelihood(weights, factors):\n return tfd.MultivariateNormalDiag(\n loc=tf.matmul(weights, factors, adjoint_b=True))\n\n def joint_log_prob(num_weights, factors, x, w):\n return (make_prior(num_weights).log_prob(w) +\n make_likelihood(w, factors).log_prob(x))\n\n def unnormalized_log_posterior(w):\n # Posterior is proportional to: `p(W, X=x | factors)`.\n return joint_log_prob(num_weights, factors, x, w)\n\n # Setup data.\n num_weights = 10 # == d\n num_factors = 40 # == n\n num_chains = 100\n\n weights = make_prior(num_weights).sample(1)\n factors = tf.random_normal([num_factors, num_weights])\n x = make_likelihood(weights, factors).sample()\n\n # Sample from Hamiltonian Monte Carlo Markov Chain.\n\n # Get `num_results` samples from `num_chains` independent chains.\n chains_states, kernels_results = tfp.mcmc.sample_chain(\n num_results=1000,\n num_burnin_steps=500,\n current_state=tf.zeros([num_chains, num_weights], name='init_weights'),\n kernel=tfp.mcmc.HamiltonianMonteCarlo(\n target_log_prob_fn=unnormalized_log_posterior,\n step_size=0.1,\n num_leapfrog_steps=2))\n\n # Compute sample stats.\n sample_mean = tf.reduce_mean(chains_states, axis=[0, 1])\n # ==> approx equal to weights\n\n sample_var = tf.reduce_mean(\n tf.squared_difference(chains_states, sample_mean),\n axis=[0, 1])\n # ==> less than 1\n ```\n\n ##### Custom tracing functions.\n\n ```python\n import tensorflow as tf\n import tensorflow_probability as tfp\n tfd = tfp.distributions\n\n likelihood = tfd.Normal(loc=0., scale=1.)\n\n def sample_chain(trace_fn):\n return tfp.mcmc.sample_chain(\n num_results=1000,\n num_burnin_steps=500,\n current_state=0.,\n kernel=tfp.mcmc.HamiltonianMonteCarlo(\n target_log_prob_fn=likelihood.log_prob,\n step_size=0.5,\n num_leapfrog_steps=2),\n trace_fn=trace_fn)\n\n def trace_log_accept_ratio(states, previous_kernel_results):\n return previous_kernel_results.log_accept_ratio\n\n def trace_everything(states, previous_kernel_results):\n return previous_kernel_results\n\n _, log_accept_ratio = sample_chain(trace_fn=trace_log_accept_ratio)\n _, kernel_results = sample_chain(trace_fn=trace_everything)\n\n acceptance_prob = tf.exp(tf.minimum(log_accept_ratio_, 0.))\n # Equivalent to, but more efficient than:\n acceptance_prob = tf.exp(tf.minimum(kernel_results.log_accept_ratio_, 0.))\n ```\n\n #### References\n\n [1]: Art B. Owen. Statistically efficient thinning of a Markov chain sampler.\n _Technical Report_, 2017.\n http://statweb.stanford.edu/~owen/reports/bestthinning.pdf\n \"\"\"\n if not kernel.is_calibrated:\n warnings.warn(\"supplied `TransitionKernel` is not calibrated. Markov \"\n \"chain may not converge to intended target distribution.\")\n with tf.compat.v1.name_scope(\n name, \"mcmc_sample_chain\",\n [num_results, num_burnin_steps, num_steps_between_results]):\n num_results = tf.convert_to_tensor(\n value=num_results, dtype=tf.int32, name=\"num_results\")\n num_burnin_steps = tf.convert_to_tensor(\n value=num_burnin_steps, dtype=tf.int32, name=\"num_burnin_steps\")\n num_steps_between_results = tf.convert_to_tensor(\n value=num_steps_between_results,\n dtype=tf.int32,\n name=\"num_steps_between_results\")\n current_state = tf.nest.map_structure(\n lambda x: tf.convert_to_tensor(value=x, name=\"current_state\"),\n current_state)\n if previous_kernel_results is None:\n previous_kernel_results = kernel.bootstrap_results(current_state)\n\n if trace_fn is None:\n # It simplifies the logic to use a dummy function here.\n trace_fn = lambda *args: ()\n no_trace = True\n else:\n no_trace = False\n if trace_fn is sample_chain.__defaults__[4]:\n warnings.warn(\"Tracing all kernel results by default is deprecated. Set \"\n \"the `trace_fn` argument to None (the future default \"\n \"value) or an explicit callback that traces the values \"\n \"you are interested in.\")\n\n def _trace_scan_fn(state_and_results, num_steps):\n next_state, current_kernel_results = mcmc_util.smart_for_loop(\n loop_num_iter=num_steps,\n body_fn=kernel.one_step,\n initial_loop_vars=list(state_and_results),\n parallel_iterations=parallel_iterations)\n return next_state, current_kernel_results\n\n (_, final_kernel_results), (all_states, trace) = mcmc_util.trace_scan(\n loop_fn=_trace_scan_fn,\n initial_state=(current_state, previous_kernel_results),\n elems=tf.one_hot(\n indices=0,\n depth=num_results,\n on_value=1 + num_burnin_steps,\n off_value=1 + num_steps_between_results,\n dtype=tf.int32),\n # pylint: disable=g-long-lambda\n trace_fn=lambda state_and_results: (state_and_results[0],\n trace_fn(*state_and_results)),\n # pylint: enable=g-long-lambda\n parallel_iterations=parallel_iterations)\n\n if return_final_kernel_results:\n return CheckpointableStatesAndTrace(\n all_states=all_states,\n trace=trace,\n final_kernel_results=final_kernel_results)\n else:\n if no_trace:\n return all_states\n else:\n return StatesAndTrace(all_states=all_states, trace=trace)", "response": "Implements Markov chain Monte Carlo via repeated TransitionKernel steps."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef trainable_gamma(shape, min_concentration=1e-3, min_scale=1e-5, name=None):\n with tf.compat.v1.variable_scope(None, default_name=\"trainable_gamma\"):\n unconstrained_concentration = tf.compat.v1.get_variable(\n \"unconstrained_concentration\",\n shape,\n initializer=tf.compat.v1.initializers.random_normal(\n mean=0.5, stddev=0.1))\n unconstrained_scale = tf.compat.v1.get_variable(\n \"unconstrained_scale\",\n shape,\n initializer=tf.compat.v1.initializers.random_normal(stddev=0.1))\n concentration = tf.maximum(tf.nn.softplus(unconstrained_concentration),\n min_concentration)\n rate = tf.maximum(1. / tf.nn.softplus(unconstrained_scale), 1. / min_scale)\n rv = ed.Gamma(concentration=concentration, rate=rate, name=name)\n return rv", "response": "Learnable Gamma via concentration and scale parameterization."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nloading NIPS 2011 conference papers from CSV file.", "response": "def load_nips2011_papers(path):\n \"\"\"Loads NIPS 2011 conference papers.\n\n The NIPS 1987-2015 data set is in the form of a 11,463 x 5,812 matrix of\n per-paper word counts, containing 11,463 words and 5,811 NIPS conference\n papers (Perrone et al., 2016). We subset to papers in 2011 and words appearing\n in at least two documents and having a total word count of at least 10.\n\n Built from the Observations Python package.\n\n Args:\n path: str.\n Path to directory which either stores file or otherwise file will\n be downloaded and extracted there. Filename is `NIPS_1987-2015.csv`.\n\n Returns:\n bag_of_words: np.ndarray of shape [num_documents, num_words]. Each element\n denotes the number of occurrences of a specific word in a specific\n document.\n words: List of strings, denoting the words for `bag_of_words`'s columns.\n \"\"\"\n path = os.path.expanduser(path)\n filename = \"NIPS_1987-2015.csv\"\n filepath = os.path.join(path, filename)\n if not os.path.exists(filepath):\n url = (\"https://archive.ics.uci.edu/ml/machine-learning-databases/\"\n \"00371/NIPS_1987-2015.csv\")\n if not tf.io.gfile.exists(path):\n tf.io.gfile.makedirs(path)\n print(\"Downloading %s to %s\" % (url, filepath))\n urllib.request.urlretrieve(url, filepath)\n\n with open(filepath) as f:\n iterator = csv.reader(f)\n documents = next(iterator)[1:]\n words = []\n x_train = []\n for row in iterator:\n words.append(row[0])\n x_train.append(row[1:])\n\n x_train = np.array(x_train, dtype=np.int)\n\n # Subset to documents in 2011 and words appearing in at least two documents\n # and have a total word count of at least 10.\n doc_idx = [i for i, document in enumerate(documents)\n if document.startswith(\"2011\")]\n documents = [documents[doc] for doc in doc_idx]\n x_train = x_train[:, doc_idx]\n word_idx = np.logical_and(np.sum(x_train != 0, 1) >= 2,\n np.sum(x_train, 1) >= 10)\n words = [word for word, idx in zip(words, word_idx) if idx]\n bag_of_words = x_train[word_idx, :].T\n return bag_of_words, words"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nsharing init logic for amplitude and length_scale params.", "response": "def _init_params(self, amplitude, length_scale, validate_args):\n \"\"\"Shared init logic for `amplitude` and `length_scale` params.\n\n Args:\n amplitude: `Tensor` (or convertible) or `None` to convert, validate.\n length_scale: `Tensor` (or convertible) or `None` to convert, validate.\n validate_args: If `True`, parameters are checked for validity despite\n possibly degrading runtime performance\n\n Returns:\n dtype: The common `DType` of the parameters.\n \"\"\"\n dtype = util.maybe_get_common_dtype(\n [amplitude, length_scale])\n if amplitude is not None:\n amplitude = tf.convert_to_tensor(\n value=amplitude, name='amplitude', dtype=dtype)\n self._amplitude = _validate_arg_if_not_none(\n amplitude, tf.compat.v1.assert_positive, validate_args)\n if length_scale is not None:\n length_scale = tf.convert_to_tensor(\n value=length_scale, name='length_scale', dtype=dtype)\n self._length_scale = _validate_arg_if_not_none(\n length_scale, tf.compat.v1.assert_positive, validate_args)\n return dtype"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\ngets the KL function registered for classes a and b.", "response": "def _registered_kl(type_a, type_b):\n \"\"\"Get the KL function registered for classes a and b.\"\"\"\n hierarchy_a = tf_inspect.getmro(type_a)\n hierarchy_b = tf_inspect.getmro(type_b)\n dist_to_children = None\n kl_fn = None\n for mro_to_a, parent_a in enumerate(hierarchy_a):\n for mro_to_b, parent_b in enumerate(hierarchy_b):\n candidate_dist = mro_to_a + mro_to_b\n candidate_kl_fn = _DIVERGENCES.get((parent_a, parent_b), None)\n if not kl_fn or (candidate_kl_fn and candidate_dist < dist_to_children):\n dist_to_children = candidate_dist\n kl_fn = candidate_kl_fn\n return kl_fn"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nreturning the batchwise KL - divergence between two distributions.", "response": "def kl_divergence(distribution_a, distribution_b,\n allow_nan_stats=True, name=None):\n \"\"\"Get the KL-divergence KL(distribution_a || distribution_b).\n\n If there is no KL method registered specifically for `type(distribution_a)`\n and `type(distribution_b)`, then the class hierarchies of these types are\n searched.\n\n If one KL method is registered between any pairs of classes in these two\n parent hierarchies, it is used.\n\n If more than one such registered method exists, the method whose registered\n classes have the shortest sum MRO paths to the input types is used.\n\n If more than one such shortest path exists, the first method\n identified in the search is used (favoring a shorter MRO distance to\n `type(distribution_a)`).\n\n Args:\n distribution_a: The first distribution.\n distribution_b: The second distribution.\n allow_nan_stats: Python `bool`, default `True`. When `True`,\n statistics (e.g., mean, mode, variance) use the value \"`NaN`\" to\n indicate the result is undefined. When `False`, an exception is raised\n if one or more of the statistic's batch members are undefined.\n name: Python `str` name prefixed to Ops created by this class.\n\n Returns:\n A Tensor with the batchwise KL-divergence between `distribution_a`\n and `distribution_b`.\n\n Raises:\n NotImplementedError: If no KL method is defined for distribution types\n of `distribution_a` and `distribution_b`.\n \"\"\"\n kl_fn = _registered_kl(type(distribution_a), type(distribution_b))\n if kl_fn is None:\n raise NotImplementedError(\n \"No KL(distribution_a || distribution_b) registered for distribution_a \"\n \"type {} and distribution_b type {}\".format(\n type(distribution_a).__name__, type(distribution_b).__name__))\n\n with tf.name_scope(\"KullbackLeibler\"):\n kl_t = kl_fn(distribution_a, distribution_b, name=name)\n if allow_nan_stats:\n return kl_t\n\n # Check KL for NaNs\n kl_t = tf.identity(kl_t, name=\"kl\")\n\n with tf.control_dependencies([\n tf.Assert(\n tf.logical_not(tf.reduce_any(input_tensor=tf.math.is_nan(kl_t))),\n [\n (\"KL calculation between {} and {} returned NaN values \"\n \"(and was called with allow_nan_stats=False). Values:\".format(\n distribution_a.name, distribution_b.name)),\n kl_t\n ])\n ]):\n return tf.identity(kl_t, name=\"checked_kl\")"} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\ncomputes the cross entropy of two distributions by P and Q.", "response": "def cross_entropy(ref, other,\n allow_nan_stats=True, name=None):\n \"\"\"Computes the (Shannon) cross entropy.\n\n Denote two distributions by `P` (`ref`) and `Q` (`other`). Assuming `P, Q`\n are absolutely continuous with respect to one another and permit densities\n `p(x) dr(x)` and `q(x) dr(x)`, (Shanon) cross entropy is defined as:\n\n ```none\n H[P, Q] = E_p[-log q(X)] = -int_F p(x) log q(x) dr(x)\n ```\n\n where `F` denotes the support of the random variable `X ~ P`.\n\n Args:\n ref: `tfd.Distribution` instance.\n other: `tfd.Distribution` instance.\n allow_nan_stats: Python `bool`, default `True`. When `True`,\n statistics (e.g., mean, mode, variance) use the value \"`NaN`\" to\n indicate the result is undefined. When `False`, an exception is raised\n if one or more of the statistic's batch members are undefined.\n name: Python `str` prepended to names of ops created by this function.\n\n Returns:\n cross_entropy: `ref.dtype` `Tensor` with shape `[B1, ..., Bn]`\n representing `n` different calculations of (Shanon) cross entropy.\n \"\"\"\n with tf.name_scope(name or \"cross_entropy\"):\n return ref.entropy() + kl_divergence(\n ref, other, allow_nan_stats=allow_nan_stats)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef read_image(filepath):\n im_bytes = tf.io.read_file(filepath)\n im = tf.image.decode_image(im_bytes, channels=CHANNELS)\n im = tf.image.convert_image_dtype(im, tf.float32)\n return im", "response": "Reads an image tensor from a file."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef download_sprites():\n filepath = os.path.join(FLAGS.data_dir, DATA_SPRITES_DIR)\n if not tf.io.gfile.exists(filepath):\n if not tf.io.gfile.exists(FLAGS.data_dir):\n tf.io.gfile.makedirs(FLAGS.data_dir)\n zip_name = \"{}.zip\".format(filepath)\n urllib.request.urlretrieve(DATA_SPRITES_URL, zip_name)\n with zipfile.ZipFile(zip_name, \"r\") as zip_file:\n zip_file.extractall(FLAGS.data_dir)\n tf.io.gfile.remove(zip_name)\n return filepath", "response": "Downloads the sprites data and returns the saved filepath."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\ncreates a character sprite from a set of attribute sprites.", "response": "def create_character(skin, hair, top, pants):\n \"\"\"Creates a character sprite from a set of attribute sprites.\"\"\"\n dtype = skin.dtype\n hair_mask = tf.cast(hair[..., -1:] <= 0, dtype)\n top_mask = tf.cast(top[..., -1:] <= 0, dtype)\n pants_mask = tf.cast(pants[..., -1:] <= 0, dtype)\n char = (skin * hair_mask) + hair\n char = (char * top_mask) + top\n char = (char * pants_mask) + pants\n return char"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\ncreate a sequence tensor for the given character sprite.", "response": "def create_seq(character, action_metadata, direction, length=8, start=0):\n \"\"\"Creates a sequence.\n\n Args:\n character: A character sprite tensor.\n action_metadata: An action metadata tuple.\n direction: An integer representing the direction, i.e., the row\n offset within each action group corresponding to a particular\n direction.\n length: Desired length of the sequence. If this is longer than\n the number of available frames, it will roll over to the\n beginning.\n start: Index of possible frames at which to start the sequence.\n\n Returns:\n A sequence tensor.\n \"\"\"\n sprite_start = (action_metadata[0]+direction) * FRAME_SIZE\n sprite_end = (action_metadata[0]+direction+1) * FRAME_SIZE\n sprite_line = character[sprite_start:sprite_end, ...]\n\n # Extract 64x64 patches that are side-by-side in the sprite, and limit\n # to the actual number of frames for the given action.\n frames = tf.stack(tf.split(sprite_line, 13, axis=1)) # 13 is a hack\n frames = frames[0:action_metadata[1]]\n\n # Extract a slice of the desired length.\n # NOTE: Length could be longer than the number of frames, so tile as needed.\n frames = tf.roll(frames, shift=-start, axis=0)\n frames = tf.tile(frames, [2, 1, 1, 1]) # 2 is a hack\n frames = frames[:length]\n frames = tf.cast(frames, dtype=tf.float32)\n frames.set_shape([length, FRAME_SIZE, FRAME_SIZE, CHANNELS])\n return frames"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\ncreate a random sequence.", "response": "def create_random_seq(character, action_metadata, direction, length=8):\n \"\"\"Creates a random sequence.\"\"\"\n start = tf.random.uniform([], maxval=action_metadata[1], dtype=tf.int32)\n return create_seq(character, action_metadata, direction, length, start)"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\ncreating a tf. data. Dataset for the sprites dataset.", "response": "def create_sprites_dataset(characters, actions, directions, channels=3,\n length=8, shuffle=False, fake_data=False):\n \"\"\"Creates a tf.data pipeline for the sprites dataset.\n\n Args:\n characters: A list of (skin, hair, top, pants) tuples containing\n relative paths to the sprite png image for each attribute.\n actions: A list of Actions.\n directions: A list of Directions.\n channels: Number of image channels to yield.\n length: Desired length of the sequences.\n shuffle: Whether or not to shuffle the characters and sequences\n start frame.\n fake_data: Boolean for whether or not to yield synthetic data.\n\n Returns:\n A tf.data.Dataset yielding (seq, skin label index, hair label index,\n top label index, pants label index, action label index, skin label\n name, hair label_name, top label name, pants label name, action\n label name) tuples.\n \"\"\"\n if fake_data:\n dummy_image = tf.random.normal([HEIGHT, WIDTH, CHANNELS])\n else:\n basedir = download_sprites()\n\n action_names = [action.name for action in actions]\n action_metadata = [(action.start_row, action.frames) for action in actions]\n\n direction_rows = [direction.row_offset for direction in directions]\n\n chars = tf.data.Dataset.from_tensor_slices(characters)\n act_names = tf.data.Dataset.from_tensor_slices(action_names).repeat()\n acts_metadata = tf.data.Dataset.from_tensor_slices(action_metadata).repeat()\n dir_rows = tf.data.Dataset.from_tensor_slices(direction_rows).repeat()\n\n if shuffle:\n chars = chars.shuffle(len(characters))\n\n dataset = tf.data.Dataset.zip((chars, act_names, acts_metadata, dir_rows))\n\n skin_table = tf.contrib.lookup.index_table_from_tensor(sorted(SKIN_COLORS))\n hair_table = tf.contrib.lookup.index_table_from_tensor(sorted(HAIRSTYLES))\n top_table = tf.contrib.lookup.index_table_from_tensor(sorted(TOPS))\n pants_table = tf.contrib.lookup.index_table_from_tensor(sorted(PANTS))\n action_table = tf.contrib.lookup.index_table_from_tensor(sorted(action_names))\n\n def process_example(attrs, act_name, act_metadata, dir_row_offset):\n \"\"\"Processes a dataset row.\"\"\"\n skin_name = attrs[0]\n hair_name = attrs[1]\n top_name = attrs[2]\n pants_name = attrs[3]\n\n if fake_data:\n char = dummy_image\n else:\n skin = read_image(basedir + os.sep + skin_name)\n hair = read_image(basedir + os.sep + hair_name)\n top = read_image(basedir + os.sep + top_name)\n pants = read_image(basedir + os.sep + pants_name)\n char = create_character(skin, hair, top, pants)\n\n if shuffle:\n seq = create_random_seq(char, act_metadata, dir_row_offset, length)\n else:\n seq = create_seq(char, act_metadata, dir_row_offset, length)\n seq = seq[..., :channels] # limit output channels\n\n skin_idx = skin_table.lookup(skin_name)\n hair_idx = hair_table.lookup(hair_name)\n top_idx = top_table.lookup(top_name)\n pants_idx = pants_table.lookup(pants_name)\n act_idx = action_table.lookup(act_name)\n\n return (seq, skin_idx, hair_idx, top_idx, pants_idx, act_idx,\n skin_name, hair_name, top_name, pants_name, act_name)\n\n dataset = dataset.map(process_example)\n return dataset"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _maybe_validate_distributions(distributions, dtype_override, validate_args):\n assertions = []\n\n if not _is_iterable(distributions) or not distributions:\n raise ValueError('`distributions` must be a list of one or more '\n 'distributions.')\n\n if dtype_override is None:\n dts = [\n dtype_util.base_dtype(d.dtype)\n for d in distributions\n if d.dtype is not None\n ]\n if dts[1:] != dts[:-1]:\n raise TypeError('Distributions must have same dtype; found: {}.'.format(\n set(dtype_util.name(dt) for dt in dts)))\n\n # Validate event_ndims.\n for d in distributions:\n if tensorshape_util.rank(d.event_shape) is not None:\n if tensorshape_util.rank(d.event_shape) != 1:\n raise ValueError('`Distribution` must be vector variate, '\n 'found event nimds: {}.'.format(\n tensorshape_util.rank(d.event_shape)))\n elif validate_args:\n assertions.append(\n assert_util.assert_equal(\n 1, tf.size(input=d.event_shape_tensor()),\n message='`Distribution` must be vector variate.'))\n\n batch_shapes = [d.batch_shape for d in distributions]\n if all(tensorshape_util.is_fully_defined(b) for b in batch_shapes):\n if batch_shapes[1:] != batch_shapes[:-1]:\n raise ValueError('Distributions must have the same `batch_shape`; '\n 'found: {}.'.format(batch_shapes))\n elif validate_args:\n batch_shapes = [\n tensorshape_util.as_list(d.batch_shape) # pylint: disable=g-complex-comprehension\n if tensorshape_util.is_fully_defined(d.batch_shape) else\n d.batch_shape_tensor() for d in distributions\n ]\n assertions.extend(\n assert_util.assert_equal( # pylint: disable=g-complex-comprehension\n b1, b2,\n message='Distribution `batch_shape`s must be identical.')\n for b1, b2 in zip(batch_shapes[1:], batch_shapes[:-1]))\n\n return assertions", "response": "Checks that the given distributions satisfy all assumptions."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef _kl_blockwise_blockwise(b0, b1, name=None):\n if len(b0.distributions) != len(b1.distributions):\n raise ValueError(\n 'Can only compute KL divergence between Blockwise distributions with '\n 'the same number of component distributions.')\n\n # We also need to check that the event shapes match for each one.\n b0_event_sizes = [_event_size(d) for d in b0.distributions]\n b1_event_sizes = [_event_size(d) for d in b1.distributions]\n\n assertions = []\n message = ('Can only compute KL divergence between Blockwise distributions '\n 'with the same pairwise event shapes.')\n\n if (all(isinstance(event_size, int) for event_size in b0_event_sizes) and\n all(isinstance(event_size, int) for event_size in b1_event_sizes)):\n if b0_event_sizes != b1_event_sizes:\n raise ValueError(message)\n else:\n if b0.validate_args or b1.validate_args:\n assertions.extend(\n assert_util.assert_equal( # pylint: disable=g-complex-comprehension\n e1, e2, message=message)\n for e1, e2 in zip(b0_event_sizes, b1_event_sizes))\n\n with tf.name_scope(name or 'kl_blockwise_blockwise'):\n with tf.control_dependencies(assertions):\n return sum([\n kullback_leibler.kl_divergence(d1, d2) for d1, d2 in zip(\n b0.distributions, b1.distributions)])", "response": "Calculate the batched KL divergence between two Blockwise distributions."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _kl_half_normal_half_normal(a, b, name=None):\n with tf.name_scope(name or \"kl_half_normal_half_normal\"):\n # Consistent with\n # http://www.mast.queensu.ca/~communications/Papers/gil-msc11.pdf, page 119\n return (tf.math.log(b.scale) - tf.math.log(a.scale) +\n (a.scale**2 - b.scale**2) / (2 * b.scale**2))", "response": "Calculate the batched KL divergence KL with a and b HalfNormal distribution."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nflatten a list of kernels which may contain _SumKernel instances.", "response": "def _flatten_summand_list(kernels):\n \"\"\"Flatten a list of kernels which may contain _SumKernel instances.\n\n Args:\n kernels: Python list of `PositiveSemidefiniteKernel` instances\n\n Returns:\n Python list containing the elements of kernels, with any _SumKernel\n instances replaced by their `kernels` property contents.\n \"\"\"\n flattened = []\n for k in kernels:\n if isinstance(k, _SumKernel):\n flattened += k.kernels\n else:\n flattened.append(k)\n return flattened"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _flatten_multiplicand_list(kernels):\n flattened = []\n for k in kernels:\n if isinstance(k, _ProductKernel):\n flattened += k.kernels\n else:\n flattened.append(k)\n return flattened", "response": "Flatten a list of kernels which may contain _ProductKernel instances."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef build_input_pipeline(x_train, x_test, y_train, y_test,\n batch_size, valid_size):\n \"\"\"Build an Iterator switching between train and heldout data.\"\"\"\n\n x_train = x_train.astype(\"float32\")\n x_test = x_test.astype(\"float32\")\n\n x_train /= 255\n x_test /= 255\n\n y_train = y_train.flatten()\n y_test = y_test.flatten()\n\n if FLAGS.subtract_pixel_mean:\n x_train_mean = np.mean(x_train, axis=0)\n x_train -= x_train_mean\n x_test -= x_train_mean\n\n print(\"x_train shape:\" + str(x_train.shape))\n print(str(x_train.shape[0]) + \" train samples\")\n print(str(x_test.shape[0]) + \" test samples\")\n\n # Build an iterator over training batches.\n training_dataset = tf.data.Dataset.from_tensor_slices(\n (x_train, np.int32(y_train)))\n training_batches = training_dataset.shuffle(\n 50000, reshuffle_each_iteration=True).repeat().batch(batch_size)\n training_iterator = tf.compat.v1.data.make_one_shot_iterator(training_batches)\n\n # Build a iterator over the heldout set with batch_size=heldout_size,\n # i.e., return the entire heldout set as a constant.\n heldout_dataset = tf.data.Dataset.from_tensor_slices(\n (x_test, np.int32(y_test)))\n heldout_batches = heldout_dataset.repeat().batch(valid_size)\n heldout_iterator = tf.compat.v1.data.make_one_shot_iterator(heldout_batches)\n\n # Combine these into a feedable iterator that can switch between training\n # and validation inputs.\n handle = tf.compat.v1.placeholder(tf.string, shape=[])\n feedable_iterator = tf.compat.v1.data.Iterator.from_string_handle(\n handle, training_batches.output_types, training_batches.output_shapes)\n images, labels = feedable_iterator.get_next()\n\n return images, labels, handle, training_iterator, heldout_iterator", "response": "Builds a pipeline that can be used to switch between train and heldout data."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef build_fake_data():\n num_examples = 10\n x_train = np.random.rand(num_examples, *IMAGE_SHAPE).astype(np.float32)\n y_train = np.random.permutation(np.arange(num_examples)).astype(np.int32)\n x_test = np.random.rand(num_examples, *IMAGE_SHAPE).astype(np.float32)\n y_test = np.random.permutation(np.arange(num_examples)).astype(np.int32)\n return (x_train, y_train), (x_test, y_test)", "response": "Build fake CIFAR10 - style data for unit testing."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\ncount the number of occurrences of each value in an integer array.", "response": "def count_integers(arr,\n weights=None,\n minlength=None,\n maxlength=None,\n axis=None,\n dtype=tf.int32,\n name=None):\n \"\"\"Counts the number of occurrences of each value in an integer array `arr`.\n\n Works like `tf.math.bincount`, but provides an `axis` kwarg that specifies\n dimensions to reduce over. With\n `~axis = [i for i in range(arr.ndim) if i not in axis]`,\n this function returns a `Tensor` of shape `[K] + arr.shape[~axis]`.\n\n If `minlength` and `maxlength` are not given, `K = tf.reduce_max(arr) + 1`\n if `arr` is non-empty, and 0 otherwise.\n If `weights` are non-None, then index `i` of the output stores the sum of the\n value in `weights` at each index where the corresponding value in `arr` is\n `i`.\n\n Args:\n arr: An `int32` `Tensor` of non-negative values.\n weights: If non-None, must be the same shape as arr. For each value in\n `arr`, the bin will be incremented by the corresponding weight instead of\n 1.\n minlength: If given, ensures the output has length at least `minlength`,\n padding with zeros at the end if necessary.\n maxlength: If given, skips values in `arr` that are equal or greater than\n `maxlength`, ensuring that the output has length at most `maxlength`.\n axis: A `0-D` or `1-D` `int32` `Tensor` (with static values) designating\n dimensions in `arr` to reduce over.\n `Default value:` `None`, meaning reduce over all dimensions.\n dtype: If `weights` is None, determines the type of the output bins.\n name: A name scope for the associated operations (optional).\n\n Returns:\n A vector with the same dtype as `weights` or the given `dtype`. The bin\n values.\n \"\"\"\n with tf.compat.v1.name_scope(\n name, 'count_integers', values=[arr, weights, minlength, maxlength,\n axis]):\n if axis is None:\n return tf.math.bincount(\n arr,\n weights=weights,\n minlength=minlength,\n maxlength=maxlength,\n dtype=dtype)\n\n arr = tf.convert_to_tensor(value=arr, dtype=tf.int32, name='arr')\n arr_ndims = _get_static_ndims(arr, expect_static=True)\n\n axis = _make_static_axis_non_negative_list(axis, arr_ndims)\n\n # ~axis from docstring. Dims in arr that are not in axis.\n not_axis = sorted(set(range(arr_ndims)).difference(axis))\n\n # If we're reducing over everything, just use standard bincount.\n if not not_axis:\n return tf.math.bincount(\n arr,\n weights=weights,\n minlength=minlength,\n maxlength=maxlength,\n dtype=dtype)\n\n # Move dims in ~axis to the left, so we can tf.map_fn bincount over them,\n # Producing counts for every index I in ~axis.\n # Thus, flat_arr is not totally flat, it just has the dims in ~axis\n # flattened.\n flat_arr = _move_dims_to_flat_end(arr, not_axis, arr_ndims, right_end=False)\n\n # tf.map_fn over dim 0.\n if weights is None:\n\n def one_bincount(arr_slice):\n return tf.math.bincount(\n arr_slice,\n weights=None,\n minlength=minlength,\n maxlength=maxlength,\n dtype=dtype)\n\n flat_counts = tf.map_fn(one_bincount, elems=flat_arr, dtype=dtype)\n else:\n weights = tf.convert_to_tensor(value=weights, name='weights')\n _get_static_ndims(weights, expect_static=True, expect_ndims=arr_ndims)\n flat_weights = _move_dims_to_flat_end(\n weights, not_axis, arr_ndims, right_end=False)\n\n def one_bincount(arr_and_weights_slices):\n arr_slice, weights_slice = arr_and_weights_slices\n return tf.math.bincount(\n arr_slice,\n weights=weights_slice,\n minlength=minlength,\n maxlength=maxlength,\n dtype=dtype)\n\n flat_counts = tf.map_fn(\n one_bincount, elems=[flat_arr, flat_weights], dtype=weights.dtype)\n\n # flat_counts.shape = [prod(~axis), K], because map_fn stacked on axis 0.\n # bincount needs to have the K bins in axis 0, so transpose...\n flat_counts_t = tf.transpose(a=flat_counts, perm=[1, 0])\n\n # Throw in this assert, to ensure shape assumptions are correct.\n _get_static_ndims(flat_counts_t, expect_ndims=2, expect_static=True)\n\n # not_axis_shape = arr.shape[~axis]\n not_axis_shape = tf.gather(tf.shape(input=arr), indices=not_axis)\n\n # The first index of flat_counts_t indexes bins 0,..,K-1, the rest are ~axis\n out_shape = tf.concat([[-1], not_axis_shape], axis=0)\n\n return tf.reshape(flat_counts_t, out_shape)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nreturn a list of bin values into discrete intervals.", "response": "def find_bins(x,\n edges,\n extend_lower_interval=False,\n extend_upper_interval=False,\n dtype=None,\n name=None):\n \"\"\"Bin values into discrete intervals.\n\n Given `edges = [c0, ..., cK]`, defining intervals\n `I0 = [c0, c1)`, `I1 = [c1, c2)`, ..., `I_{K-1} = [c_{K-1}, cK]`,\n This function returns `bins`, such that:\n `edges[bins[i]] <= x[i] < edges[bins[i] + 1]`.\n\n Args:\n x: Numeric `N-D` `Tensor` with `N > 0`.\n edges: `Tensor` of same `dtype` as `x`. The first dimension indexes edges\n of intervals. Must either be `1-D` or have\n `x.shape[1:] == edges.shape[1:]`. If `rank(edges) > 1`, `edges[k]`\n designates a shape `edges.shape[1:]` `Tensor` of bin edges for the\n corresponding dimensions of `x`.\n extend_lower_interval: Python `bool`. If `True`, extend the lowest\n interval `I0` to `(-inf, c1]`.\n extend_upper_interval: Python `bool`. If `True`, extend the upper\n interval `I_{K-1}` to `[c_{K-1}, +inf)`.\n dtype: The output type (`int32` or `int64`). `Default value:` `x.dtype`.\n This effects the output values when `x` is below/above the intervals,\n which will be `-1/K+1` for `int` types and `NaN` for `float`s.\n At indices where `x` is `NaN`, the output values will be `0` for `int`\n types and `NaN` for floats.\n name: A Python string name to prepend to created ops. Default: 'find_bins'\n\n Returns:\n bins: `Tensor` with same `shape` as `x` and `dtype`.\n Has whole number values. `bins[i] = k` means the `x[i]` falls into the\n `kth` bin, ie, `edges[bins[i]] <= x[i] < edges[bins[i] + 1]`.\n\n Raises:\n ValueError: If `edges.shape[0]` is determined to be less than 2.\n\n #### Examples\n\n Cut a `1-D` array\n\n ```python\n x = [0., 5., 6., 10., 20.]\n edges = [0., 5., 10.]\n tfp.stats.find_bins(x, edges)\n ==> [0., 0., 1., 1., np.nan]\n ```\n\n Cut `x` into its deciles\n\n ```python\n x = tf.random_uniform(shape=(100, 200))\n decile_edges = tfp.stats.quantiles(x, num_quantiles=10)\n bins = tfp.stats.find_bins(x, edges=decile_edges)\n bins.shape\n ==> (100, 200)\n tf.reduce_mean(bins == 0.)\n ==> approximately 0.1\n tf.reduce_mean(bins == 1.)\n ==> approximately 0.1\n ```\n\n \"\"\"\n # TFP users may be surprised to see the \"action\" in the leftmost dim of\n # edges, rather than the rightmost (event) dim. Why?\n # 1. Most likely you created edges by getting quantiles over samples, and\n # quantile/percentile return these edges in the leftmost (sample) dim.\n # 2. Say you have event_shape = [5], then we expect the bin will be different\n # for all 5 events, so the index of the bin should not be in the event dim.\n with tf.compat.v1.name_scope(\n name, default_name='find_bins', values=[x, edges]):\n in_type = dtype_util.common_dtype([x, edges],\n preferred_dtype=tf.float32)\n edges = tf.convert_to_tensor(value=edges, name='edges', dtype=in_type)\n x = tf.convert_to_tensor(value=x, name='x', dtype=in_type)\n\n if (tf.compat.dimension_value(edges.shape[0]) is not None and\n tf.compat.dimension_value(edges.shape[0]) < 2):\n raise ValueError(\n 'First dimension of `edges` must have length > 1 to index 1 or '\n 'more bin. Found: {}'.format(edges.shape))\n\n flattening_x = edges.shape.ndims == 1 and x.shape.ndims > 1\n\n if flattening_x:\n x_orig_shape = tf.shape(input=x)\n x = tf.reshape(x, [-1])\n\n if dtype is None:\n dtype = in_type\n dtype = tf.as_dtype(dtype)\n\n # Move first dims into the rightmost.\n x_permed = distribution_util.rotate_transpose(x, shift=-1)\n edges_permed = distribution_util.rotate_transpose(edges, shift=-1)\n\n # If...\n # x_permed = [0, 1, 6., 10]\n # edges = [0, 5, 10.]\n # ==> almost_output = [0, 1, 2, 2]\n searchsorted_type = dtype if dtype in [tf.int32, tf.int64] else None\n almost_output_permed = tf.searchsorted(\n sorted_sequence=edges_permed,\n values=x_permed,\n side='right',\n out_type=searchsorted_type)\n # Move the rightmost dims back to the leftmost.\n almost_output = tf.cast(\n distribution_util.rotate_transpose(almost_output_permed, shift=1),\n dtype)\n\n # In above example, we want [0, 0, 1, 1], so correct this here.\n bins = tf.clip_by_value(almost_output - 1, tf.cast(0, dtype),\n tf.cast(tf.shape(input=edges)[0] - 2, dtype))\n\n if not extend_lower_interval:\n low_fill = np.nan if dtype.is_floating else -1\n bins = tf.where(x < tf.expand_dims(edges[0], 0),\n tf.fill(tf.shape(input=x), tf.cast(low_fill, dtype)),\n bins)\n\n if not extend_upper_interval:\n up_fill = np.nan if dtype.is_floating else tf.shape(input=edges)[0] - 1\n bins = tf.where(x > tf.expand_dims(edges[-1], 0),\n tf.fill(tf.shape(input=x), tf.cast(up_fill, dtype)), bins)\n\n if flattening_x:\n bins = tf.reshape(bins, x_orig_shape)\n\n return bins"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\nreturns a histogram of the number of events in a given axis.", "response": "def histogram(x,\n edges,\n axis=None,\n extend_lower_interval=False,\n extend_upper_interval=False,\n dtype=None,\n name=None):\n \"\"\"Count how often `x` falls in intervals defined by `edges`.\n\n Given `edges = [c0, ..., cK]`, defining intervals\n `I0 = [c0, c1)`, `I1 = [c1, c2)`, ..., `I_{K-1} = [c_{K-1}, cK]`,\n This function counts how often `x` falls into each interval.\n\n Values of `x` outside of the intervals cause errors. Consider using\n `extend_lower_interval`, `extend_upper_interval` to deal with this.\n\n Args:\n x: Numeric `N-D` `Tensor` with `N > 0`. If `axis` is not\n `None`, must have statically known number of dimensions. The\n `axis` kwarg determines which dimensions index iid samples.\n Other dimensions of `x` index \"events\" for which we will compute different\n histograms.\n edges: `Tensor` of same `dtype` as `x`. The first dimension indexes edges\n of intervals. Must either be `1-D` or have `edges.shape[1:]` the same\n as the dimensions of `x` excluding `axis`.\n If `rank(edges) > 1`, `edges[k]` designates a shape `edges.shape[1:]`\n `Tensor` of interval edges for the corresponding dimensions of `x`.\n axis: Optional `0-D` or `1-D` integer `Tensor` with constant\n values. The axis in `x` that index iid samples.\n `Default value:` `None` (treat every dimension as sample dimension).\n extend_lower_interval: Python `bool`. If `True`, extend the lowest\n interval `I0` to `(-inf, c1]`.\n extend_upper_interval: Python `bool`. If `True`, extend the upper\n interval `I_{K-1}` to `[c_{K-1}, +inf)`.\n dtype: The output type (`int32` or `int64`). `Default value:` `x.dtype`.\n name: A Python string name to prepend to created ops.\n `Default value:` 'histogram'\n\n Returns:\n counts: `Tensor` of type `dtype` and, with\n `~axis = [i for i in range(arr.ndim) if i not in axis]`,\n `counts.shape = [edges.shape[0]] + x.shape[~axis]`.\n With `I` a multi-index into `~axis`, `counts[k][I]` is the number of times\n event(s) fell into the `kth` interval of `edges`.\n\n #### Examples\n\n ```python\n # x.shape = [1000, 2]\n # x[:, 0] ~ Uniform(0, 1), x[:, 1] ~ Uniform(1, 2).\n x = tf.stack([tf.random_uniform([1000]), 1 + tf.random_uniform([1000])],\n axis=-1)\n\n # edges ==> bins [0, 0.5), [0.5, 1.0), [1.0, 1.5), [1.5, 2.0].\n edges = [0., 0.5, 1.0, 1.5, 2.0]\n\n tfp.stats.histogram(x, edges)\n ==> approximately [500, 500, 500, 500]\n\n tfp.stats.histogram(x, edges, axis=0)\n ==> approximately [[500, 500, 0, 0], [0, 0, 500, 500]]\n ```\n\n \"\"\"\n with tf.compat.v1.name_scope(name, 'histogram', values=[x, edges, axis]):\n\n # Tensor conversions.\n in_dtype = dtype_util.common_dtype([x, edges], preferred_dtype=tf.float32)\n\n x = tf.convert_to_tensor(value=x, name='x', dtype=in_dtype)\n edges = tf.convert_to_tensor(value=edges, name='edges', dtype=in_dtype)\n\n # Move dims in axis to the left end as one flattened dim.\n # After this, x.shape = [n_samples] + E.\n if axis is None:\n x = tf.reshape(x, shape=[-1])\n else:\n x_ndims = _get_static_ndims(\n x, expect_static=True, expect_ndims_at_least=1)\n axis = _make_static_axis_non_negative_list(axis, x_ndims)\n if not axis:\n raise ValueError('`axis` cannot be empty. Found: {}'.format(axis))\n x = _move_dims_to_flat_end(x, axis, x_ndims, right_end=False)\n\n # bins.shape = x.shape = [n_samples] + E,\n # and bins[i] is a shape E Tensor of the bins that sample `i` fell into.\n # E is the \"event shape\", which is [] if axis is None.\n bins = find_bins(\n x,\n edges=edges,\n # If not extending intervals, then values outside the edges will return\n # -1, which gives an error when fed to bincount.\n extend_lower_interval=extend_lower_interval,\n extend_upper_interval=extend_upper_interval,\n dtype=tf.int32)\n\n # TODO(b/124015136) Use standard tf.math.bincount once it supports `axis`.\n counts = count_integers(\n bins,\n # Ensure we get correct output, even if x did not fall into every bin\n minlength=tf.shape(input=edges)[0] - 1,\n maxlength=tf.shape(input=edges)[0] - 1,\n axis=0,\n dtype=dtype or in_dtype)\n n_edges = tf.compat.dimension_value(edges.shape[0])\n if n_edges is not None:\n counts.set_shape(\n tf.TensorShape([n_edges - 1]).concatenate(counts.shape[1:]))\n return counts"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef percentile(x,\n q,\n axis=None,\n interpolation=None,\n keep_dims=False,\n validate_args=False,\n preserve_gradients=True,\n name=None):\n \"\"\"Compute the `q`-th percentile(s) of `x`.\n\n Given a vector `x`, the `q`-th percentile of `x` is the value `q / 100` of the\n way from the minimum to the maximum in a sorted copy of `x`.\n\n The values and distances of the two nearest neighbors as well as the\n `interpolation` parameter will determine the percentile if the normalized\n ranking does not match the location of `q` exactly.\n\n This function is the same as the median if `q = 50`, the same as the minimum\n if `q = 0` and the same as the maximum if `q = 100`.\n\n Multiple percentiles can be computed at once by using `1-D` vector `q`.\n Dimension zero of the returned `Tensor` will index the different percentiles.\n\n Compare to `numpy.percentile`.\n\n Args:\n x: Numeric `N-D` `Tensor` with `N > 0`. If `axis` is not `None`,\n `x` must have statically known number of dimensions.\n q: Scalar or vector `Tensor` with values in `[0, 100]`. The percentile(s).\n axis: Optional `0-D` or `1-D` integer `Tensor` with constant values. The\n axis that index independent samples over which to return the desired\n percentile. If `None` (the default), treat every dimension as a sample\n dimension, returning a scalar.\n interpolation : {'nearest', 'linear', 'lower', 'higher', 'midpoint'}.\n Default value: 'nearest'. This specifies the interpolation method to\n use when the desired quantile lies between two data points `i < j`:\n * linear: i + (j - i) * fraction, where fraction is the fractional part\n of the index surrounded by i and j.\n * lower: `i`.\n * higher: `j`.\n * nearest: `i` or `j`, whichever is nearest.\n * midpoint: (i + j) / 2.\n `linear` and `midpoint` interpolation do not work with integer dtypes.\n keep_dims: Python `bool`. If `True`, the last dimension is kept with size 1\n If `False`, the last dimension is removed from the output shape.\n validate_args: Whether to add runtime checks of argument validity. If\n False, and arguments are incorrect, correct behavior is not guaranteed.\n preserve_gradients: Python `bool`. If `True`, ensure that gradient w.r.t\n the percentile `q` is preserved in the case of linear interpolation.\n If `False`, the gradient will be (incorrectly) zero when `q` corresponds\n to a point in `x`.\n name: A Python string name to give this `Op`. Default is 'percentile'\n\n Returns:\n A `(rank(q) + N - len(axis))` dimensional `Tensor` of same dtype as `x`, or,\n if `axis` is `None`, a `rank(q)` `Tensor`. The first `rank(q)` dimensions\n index quantiles for different values of `q`.\n\n Raises:\n ValueError: If argument 'interpolation' is not an allowed type.\n ValueError: If interpolation type not compatible with `dtype`.\n\n #### Examples\n\n ```python\n # Get 30th percentile with default ('nearest') interpolation.\n x = [1., 2., 3., 4.]\n tfp.stats.percentile(x, q=30.)\n ==> 2.0\n\n # Get 30th percentile with 'linear' interpolation.\n x = [1., 2., 3., 4.]\n tfp.stats.percentile(x, q=30., interpolation='linear')\n ==> 1.9\n\n # Get 30th and 70th percentiles with 'lower' interpolation\n x = [1., 2., 3., 4.]\n tfp.stats.percentile(x, q=[30., 70.], interpolation='lower')\n ==> [1., 3.]\n\n # Get 100th percentile (maximum). By default, this is computed over every dim\n x = [[1., 2.]\n [3., 4.]]\n tfp.stats.percentile(x, q=100.)\n ==> 4.\n\n # Treat the leading dim as indexing samples, and find the 100th quantile (max)\n # over all such samples.\n x = [[1., 2.]\n [3., 4.]]\n tfp.stats.percentile(x, q=100., axis=[0])\n ==> [3., 4.]\n ```\n\n \"\"\"\n name = name or 'percentile'\n allowed_interpolations = {'linear', 'lower', 'higher', 'nearest', 'midpoint'}\n\n if interpolation is None:\n interpolation = 'nearest'\n else:\n if interpolation not in allowed_interpolations:\n raise ValueError('Argument `interpolation` must be in %s. Found %s' %\n (allowed_interpolations, interpolation))\n\n with tf.compat.v1.name_scope(name, values=[x, q]):\n x = tf.convert_to_tensor(value=x, name='x')\n\n if interpolation in {'linear', 'midpoint'} and x.dtype.is_integer:\n raise TypeError('{} interpolation not allowed with dtype {}'.format(\n interpolation, x.dtype))\n\n # Double is needed here and below, else we get the wrong index if the array\n # is huge along axis.\n q = tf.cast(q, tf.float64)\n _get_static_ndims(q, expect_ndims_no_more_than=1)\n\n if validate_args:\n q = distribution_util.with_dependencies([\n tf.compat.v1.assert_rank_in(q, [0, 1]),\n tf.compat.v1.assert_greater_equal(q, tf.cast(0., tf.float64)),\n tf.compat.v1.assert_less_equal(q, tf.cast(100., tf.float64))\n ], q)\n\n # Move `axis` dims of `x` to the rightmost, call it `y`.\n if axis is None:\n y = tf.reshape(x, [-1])\n else:\n x_ndims = _get_static_ndims(\n x, expect_static=True, expect_ndims_at_least=1)\n axis = _make_static_axis_non_negative_list(axis, x_ndims)\n y = _move_dims_to_flat_end(x, axis, x_ndims, right_end=True)\n\n frac_at_q_or_above = 1. - q / 100.\n\n # Sort everything, not just the top 'k' entries, which allows multiple calls\n # to sort only once (under the hood) and use CSE.\n sorted_y = _sort_tensor(y)\n\n d = tf.cast(tf.shape(input=y)[-1], tf.float64)\n\n def _get_indices(interp_type):\n \"\"\"Get values of y at the indices implied by interp_type.\"\"\"\n # Note `lower` <--> ceiling. Confusing, huh? Due to the fact that\n # _sort_tensor sorts highest to lowest, tf.ceil corresponds to the higher\n # index, but the lower value of y!\n if interp_type == 'lower':\n indices = tf.math.ceil((d - 1) * frac_at_q_or_above)\n elif interp_type == 'higher':\n indices = tf.floor((d - 1) * frac_at_q_or_above)\n elif interp_type == 'nearest':\n indices = tf.round((d - 1) * frac_at_q_or_above)\n # d - 1 will be distinct from d in int32, but not necessarily double.\n # So clip to avoid out of bounds errors.\n return tf.clip_by_value(\n tf.cast(indices, tf.int32), 0,\n tf.shape(input=y)[-1] - 1)\n\n if interpolation in ['nearest', 'lower', 'higher']:\n gathered_y = tf.gather(sorted_y, _get_indices(interpolation), axis=-1)\n elif interpolation == 'midpoint':\n gathered_y = 0.5 * (\n tf.gather(sorted_y, _get_indices('lower'), axis=-1) +\n tf.gather(sorted_y, _get_indices('higher'), axis=-1))\n elif interpolation == 'linear':\n # Copy-paste of docstring on interpolation:\n # linear: i + (j - i) * fraction, where fraction is the fractional part\n # of the index surrounded by i and j.\n larger_y_idx = _get_indices('lower')\n exact_idx = (d - 1) * frac_at_q_or_above\n if preserve_gradients:\n # If q corresponds to a point in x, we will initially have\n # larger_y_idx == smaller_y_idx.\n # This results in the gradient w.r.t. fraction being zero (recall `q`\n # enters only through `fraction`...and see that things cancel).\n # The fix is to ensure that smaller_y_idx and larger_y_idx are always\n # separated by exactly 1.\n smaller_y_idx = tf.maximum(larger_y_idx - 1, 0)\n larger_y_idx = tf.minimum(smaller_y_idx + 1, tf.shape(input=y)[-1] - 1)\n fraction = tf.cast(larger_y_idx, tf.float64) - exact_idx\n else:\n smaller_y_idx = _get_indices('higher')\n fraction = tf.math.ceil((d - 1) * frac_at_q_or_above) - exact_idx\n\n fraction = tf.cast(fraction, y.dtype)\n gathered_y = (\n tf.gather(sorted_y, larger_y_idx, axis=-1) * (1 - fraction) +\n tf.gather(sorted_y, smaller_y_idx, axis=-1) * fraction)\n\n # Propagate NaNs\n if x.dtype in (tf.bfloat16, tf.float16, tf.float32, tf.float64):\n # Apparently tf.is_nan doesn't like other dtypes\n nan_batch_members = tf.reduce_any(\n input_tensor=tf.math.is_nan(x), axis=axis)\n right_rank_matched_shape = tf.pad(\n tensor=tf.shape(input=nan_batch_members),\n paddings=[[0, tf.rank(input=q)]],\n constant_values=1)\n nan_batch_members = tf.reshape(\n nan_batch_members, shape=right_rank_matched_shape)\n shape_gathered_y = tf.shape(input=gathered_y)\n nan = np.array(np.nan, gathered_y.dtype.as_numpy_dtype)\n gathered_y = tf.where(\n tf.broadcast_to(nan_batch_members, shape_gathered_y),\n tf.fill(shape_gathered_y, nan),\n gathered_y)\n\n # Expand dimensions if requested\n if keep_dims:\n if axis is None:\n ones_vec = tf.ones(\n shape=[_get_best_effort_ndims(x) + _get_best_effort_ndims(q)],\n dtype=tf.int32)\n gathered_y *= tf.ones(ones_vec, dtype=x.dtype)\n else:\n gathered_y = _insert_back_keep_dims(gathered_y, axis)\n\n # If q is a scalar, then result has the right shape.\n # If q is a vector, then result has trailing dim of shape q.shape, which\n # needs to be rotated to dim 0.\n return distribution_util.rotate_transpose(gathered_y, tf.rank(q))", "response": "Compute the percentile of x along a given axis."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\ncomputes the quantiles of a distribution along a specified axis.", "response": "def quantiles(x,\n num_quantiles,\n axis=None,\n interpolation=None,\n keep_dims=False,\n validate_args=False,\n name=None):\n \"\"\"Compute quantiles of `x` along `axis`.\n\n The quantiles of a distribution are cut points dividing the range into\n intervals with equal probabilities.\n\n Given a vector `x` of samples, this function estimates the cut points by\n returning `num_quantiles + 1` cut points, `(c0, ..., cn)`, such that, roughly\n speaking, equal number of sample points lie in the `num_quantiles` intervals\n `[c0, c1), [c1, c2), ..., [c_{n-1}, cn]`. That is,\n\n * About `1 / n` fraction of the data lies in `[c_{k-1}, c_k)`, `k = 1, ..., n`\n * About `k / n` fraction of the data lies below `c_k`.\n * `c0` is the sample minimum and `cn` is the maximum.\n\n The exact number of data points in each interval depends on the size of\n `x` (e.g. whether the size is divisible by `n`) and the `interpolation` kwarg.\n\n Args:\n x: Numeric `N-D` `Tensor` with `N > 0`. If `axis` is not `None`,\n `x` must have statically known number of dimensions.\n num_quantiles: Scalar `integer` `Tensor`. The number of intervals the\n returned `num_quantiles + 1` cut points divide the range into.\n axis: Optional `0-D` or `1-D` integer `Tensor` with constant values. The\n axis that index independent samples over which to return the desired\n percentile. If `None` (the default), treat every dimension as a sample\n dimension, returning a scalar.\n interpolation : {'nearest', 'linear', 'lower', 'higher', 'midpoint'}.\n Default value: 'nearest'. This specifies the interpolation method to\n use when the fractions `k / n` lie between two data points `i < j`:\n * linear: i + (j - i) * fraction, where fraction is the fractional part\n of the index surrounded by i and j.\n * lower: `i`.\n * higher: `j`.\n * nearest: `i` or `j`, whichever is nearest.\n * midpoint: (i + j) / 2. `linear` and `midpoint` interpolation do not\n work with integer dtypes.\n keep_dims: Python `bool`. If `True`, the last dimension is kept with size 1\n If `False`, the last dimension is removed from the output shape.\n validate_args: Whether to add runtime checks of argument validity. If\n False, and arguments are incorrect, correct behavior is not guaranteed.\n name: A Python string name to give this `Op`. Default is 'percentile'\n\n Returns:\n cut_points: A `rank(x) + 1 - len(axis)` dimensional `Tensor` with same\n `dtype` as `x` and shape `[num_quantiles + 1, ...]` where the trailing shape\n is that of `x` without the dimensions in `axis` (unless `keep_dims is True`)\n\n Raises:\n ValueError: If argument 'interpolation' is not an allowed type.\n ValueError: If interpolation type not compatible with `dtype`.\n\n #### Examples\n\n ```python\n # Get quartiles of x with various interpolation choices.\n x = [0., 1., 2., 3., 4., 5., 6., 7., 8., 9., 10.]\n\n tfp.stats.quantiles(x, num_quantiles=4, interpolation='nearest')\n ==> [ 0., 2., 5., 8., 10.]\n\n tfp.stats.quantiles(x, num_quantiles=4, interpolation='linear')\n ==> [ 0. , 2.5, 5. , 7.5, 10. ]\n\n tfp.stats.quantiles(x, num_quantiles=4, interpolation='lower')\n ==> [ 0., 2., 5., 7., 10.]\n\n # Get deciles of columns of an R x C data set.\n data = load_my_columnar_data(...)\n tfp.stats.quantiles(data, num_quantiles=10)\n ==> Shape [11, C] Tensor\n ```\n\n \"\"\"\n with tf.compat.v1.name_scope(\n name, 'quantiles', values=[x, num_quantiles, axis]):\n x = tf.convert_to_tensor(value=x, name='x')\n return percentile(\n x,\n q=tf.linspace(\n # percentile casts q to float64 before using it...so may as well use\n # float64 here. Note that using x.dtype won't work with linspace\n # if x is integral type (which is anothe motivation for hard-coding\n # float64).\n tf.convert_to_tensor(value=0, dtype=tf.float64),\n tf.convert_to_tensor(value=100, dtype=tf.float64),\n num=num_quantiles + 1),\n axis=axis,\n interpolation=interpolation,\n keep_dims=keep_dims,\n validate_args=validate_args,\n preserve_gradients=False)"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\ngets the number of dimensions and assert that some expectations are met.", "response": "def _get_static_ndims(x,\n expect_static=False,\n expect_ndims=None,\n expect_ndims_no_more_than=None,\n expect_ndims_at_least=None):\n \"\"\"Get static number of dimensions and assert that some expectations are met.\n\n This function returns the number of dimensions 'ndims' of x, as a Python int.\n\n The optional expect arguments are used to check the ndims of x, but this is\n only done if the static ndims of x is not None.\n\n Args:\n x: A Tensor.\n expect_static: Expect `x` to have statically defined `ndims`.\n expect_ndims: Optional Python integer. If provided, assert that x has\n number of dimensions equal to this.\n expect_ndims_no_more_than: Optional Python integer. If provided, assert\n that x has no more than this many dimensions.\n expect_ndims_at_least: Optional Python integer. If provided, assert that x\n has at least this many dimensions.\n\n Returns:\n ndims: A Python integer.\n\n Raises:\n ValueError: If any of the expectations above are violated.\n \"\"\"\n ndims = x.shape.ndims\n if ndims is None:\n shape_const = tf.get_static_value(tf.shape(input=x))\n if shape_const is not None:\n ndims = shape_const.ndim\n\n if ndims is None:\n if expect_static:\n raise ValueError(\n 'Expected argument `x` to have statically defined `ndims`. Found: ' %\n x)\n return\n\n if expect_ndims is not None:\n ndims_message = ('Expected argument `x` to have ndims %s. Found tensor %s'\n % (expect_ndims, x))\n if ndims != expect_ndims:\n raise ValueError(ndims_message)\n\n if expect_ndims_at_least is not None:\n ndims_at_least_message = (\n 'Expected argument `x` to have ndims >= %d. Found tensor %s' %\n (expect_ndims_at_least, x))\n if ndims < expect_ndims_at_least:\n raise ValueError(ndims_at_least_message)\n\n if expect_ndims_no_more_than is not None:\n ndims_no_more_than_message = (\n 'Expected argument `x` to have ndims <= %d. Found tensor %s' %\n (expect_ndims_no_more_than, x))\n if ndims > expect_ndims_no_more_than:\n raise ValueError(ndims_no_more_than_message)\n\n return ndims"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nget ndims if possible.", "response": "def _get_best_effort_ndims(x,\n expect_ndims=None,\n expect_ndims_at_least=None,\n expect_ndims_no_more_than=None):\n \"\"\"Get static ndims if possible. Fallback on `tf.rank(x)`.\"\"\"\n ndims_static = _get_static_ndims(\n x,\n expect_ndims=expect_ndims,\n expect_ndims_at_least=expect_ndims_at_least,\n expect_ndims_no_more_than=expect_ndims_no_more_than)\n if ndims_static is not None:\n return ndims_static\n return tf.rank(x)"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\ninsert the dims in axis back as singletons after being removed.", "response": "def _insert_back_keep_dims(x, axis):\n \"\"\"Insert the dims in `axis` back as singletons after being removed.\n\n Args:\n x: `Tensor`.\n axis: Python list of integers.\n\n Returns:\n `Tensor` with same values as `x`, but additional singleton dimensions.\n \"\"\"\n for i in sorted(axis):\n x = tf.expand_dims(x, axis=i)\n return x"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nconverts possibly negative indexed axis to non - negative list of ints.", "response": "def _make_static_axis_non_negative_list(axis, ndims):\n \"\"\"Convert possibly negatively indexed axis to non-negative list of ints.\n\n Args:\n axis: Integer Tensor.\n ndims: Number of dimensions into which axis indexes.\n\n Returns:\n A list of non-negative Python integers.\n\n Raises:\n ValueError: If `axis` is not statically defined.\n \"\"\"\n axis = distribution_util.make_non_negative_axis(axis, ndims)\n\n axis_const = tf.get_static_value(axis)\n if axis_const is None:\n raise ValueError(\n 'Expected argument `axis` to be statically available. Found: %s' %\n axis)\n\n # Make at least 1-D.\n axis = axis_const + np.zeros([1], dtype=axis_const.dtype)\n\n return list(int(dim) for dim in axis)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nmove dims corresponding to axis in x to the end then flatten.", "response": "def _move_dims_to_flat_end(x, axis, x_ndims, right_end=True):\n \"\"\"Move dims corresponding to `axis` in `x` to the end, then flatten.\n\n Args:\n x: `Tensor` with shape `[B0,B1,...,Bb]`.\n axis: Python list of indices into dimensions of `x`.\n x_ndims: Python integer holding number of dimensions in `x`.\n right_end: Python bool. Whether to move dims to the right end (else left).\n\n Returns:\n `Tensor` with value from `x` and dims in `axis` moved to end into one single\n dimension.\n \"\"\"\n\n if not axis:\n return x\n\n # Suppose x.shape = [a, b, c, d]\n # Suppose axis = [1, 3]\n\n # other_dims = [0, 2] in example above.\n other_dims = sorted(set(range(x_ndims)).difference(axis))\n # x_permed.shape = [a, c, b, d]\n perm = other_dims + list(axis) if right_end else list(axis) + other_dims\n x_permed = tf.transpose(a=x, perm=perm)\n\n if x.shape.is_fully_defined():\n x_shape = x.shape.as_list()\n # other_shape = [a, c], end_shape = [b * d]\n other_shape = [x_shape[i] for i in other_dims]\n end_shape = [np.prod([x_shape[i] for i in axis])]\n full_shape = (\n other_shape + end_shape if right_end else end_shape + other_shape)\n else:\n other_shape = tf.gather(tf.shape(input=x), other_dims)\n full_shape = tf.concat(\n [other_shape, [-1]] if right_end else [[-1], other_shape], axis=0)\n return tf.reshape(x_permed, shape=full_shape)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nusing top_k to sort a Tensor along the last dimension.", "response": "def _sort_tensor(tensor):\n \"\"\"Use `top_k` to sort a `Tensor` along the last dimension.\"\"\"\n sorted_, _ = tf.nn.top_k(tensor, k=tf.shape(input=tensor)[-1])\n sorted_.set_shape(tensor.shape)\n return sorted_"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nbuild a list of LinearGaussianStateSpaceModel objects for each component in the current state space.", "response": "def make_component_state_space_models(self,\n num_timesteps,\n param_vals,\n initial_step=0):\n \"\"\"Build an ordered list of Distribution instances for component models.\n\n Args:\n num_timesteps: Python `int` number of timesteps to model.\n param_vals: a list of `Tensor` parameter values in order corresponding to\n `self.parameters`, or a dict mapping from parameter names to values.\n initial_step: optional `int` specifying the initial timestep to model.\n This is relevant when the model contains time-varying components,\n e.g., holidays or seasonality.\n\n Returns:\n component_ssms: a Python list of `LinearGaussianStateSpaceModel`\n Distribution objects, in order corresponding to `self.components`.\n \"\"\"\n\n with tf.compat.v1.name_scope('make_component_state_space_models'):\n\n # List the model parameters in canonical order\n param_map = self._canonicalize_param_vals_as_map(param_vals)\n param_vals_list = [param_map[p.name] for p in self.parameters]\n\n # Build SSMs for each component model. We process the components in\n # canonical order, extracting the parameters for each component from the\n # (ordered) list of parameters.\n remaining_param_vals = param_vals_list[1:]\n component_ssms = []\n for component in self.components:\n num_parameters = len(component.parameters)\n component_param_vals = remaining_param_vals[:num_parameters]\n remaining_param_vals = remaining_param_vals[num_parameters:]\n\n component_ssms.append(\n component.make_state_space_model(\n num_timesteps,\n param_vals=component_param_vals,\n initial_step=initial_step))\n\n return component_ssms"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef kl_reverse(logu, self_normalized=False, name=None):\n\n with tf.compat.v1.name_scope(name, \"kl_reverse\", [logu]):\n return amari_alpha(logu, alpha=0., self_normalized=self_normalized)", "response": "A function that computes the inverse Kullback - Leibler Csiszar - function at logu."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef jensen_shannon(logu, self_normalized=False, name=None):\n\n with tf.compat.v1.name_scope(name, \"jensen_shannon\", [logu]):\n logu = tf.convert_to_tensor(value=logu, name=\"logu\")\n npdt = logu.dtype.as_numpy_dtype\n y = tf.nn.softplus(logu)\n if self_normalized:\n y -= np.log(2).astype(npdt)\n return tf.exp(logu) * logu - (1. + tf.exp(logu)) * y", "response": "A function that computes the Jensen - Shannon Csiszar - function in log - space."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef triangular(logu, name=None):\n\n with tf.compat.v1.name_scope(name, \"triangular\", [logu]):\n logu = tf.convert_to_tensor(value=logu, name=\"logu\")\n return pearson(logu) / (1. + tf.exp(logu))", "response": "A function that evaluates a Triangular Csiszar - function at logu."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef t_power(logu, t, self_normalized=False, name=None):\n with tf.compat.v1.name_scope(name, \"t_power\", [logu, t]):\n logu = tf.convert_to_tensor(value=logu, name=\"logu\")\n t = tf.convert_to_tensor(value=t, dtype=logu.dtype.base_dtype, name=\"t\")\n fu = tf.math.expm1(t * logu)\n if self_normalized:\n fu -= t * tf.math.expm1(logu)\n fu *= tf.where(tf.logical_and(0. < t, t < 1.),\n -tf.ones_like(t),\n tf.ones_like(t))\n return fu", "response": "A function that evaluates the T - Power Csiszar - function at logu at t."} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef modified_gan(logu, self_normalized=False, name=None):\n\n with tf.compat.v1.name_scope(name, \"chi_square\", [logu]):\n logu = tf.convert_to_tensor(value=logu, name=\"logu\")\n y = tf.nn.softplus(logu) - logu\n if self_normalized:\n y += 0.5 * tf.math.expm1(logu)\n return y", "response": "The Modified - GAN Csiszar - function in log - space."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef dual_csiszar_function(logu, csiszar_function, name=None):\n\n with tf.compat.v1.name_scope(name, \"dual_csiszar_function\", [logu]):\n return tf.exp(logu) * csiszar_function(-logu)", "response": "Calculates the dual of a Csiszar - function over log - space."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef symmetrized_csiszar_function(logu, csiszar_function, name=None):\n\n with tf.compat.v1.name_scope(name, \"symmetrized_csiszar_function\", [logu]):\n logu = tf.convert_to_tensor(value=logu, name=\"logu\")\n return 0.5 * (csiszar_function(logu)\n + dual_csiszar_function(logu, csiszar_function))", "response": "This function symmetrizes a Csiszar - function in log - space."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef monte_carlo_csiszar_f_divergence(\n f,\n p_log_prob,\n q,\n num_draws,\n use_reparametrization=None,\n seed=None,\n name=None):\n \"\"\"Monte-Carlo approximation of the Csiszar f-Divergence.\n\n A Csiszar-function is a member of,\n\n ```none\n F = { f:R_+ to R : f convex }.\n ```\n\n The Csiszar f-Divergence for Csiszar-function f is given by:\n\n ```none\n D_f[p(X), q(X)] := E_{q(X)}[ f( p(X) / q(X) ) ]\n ~= m**-1 sum_j^m f( p(x_j) / q(x_j) ),\n where x_j ~iid q(X)\n ```\n\n Tricks: Reparameterization and Score-Gradient\n\n When q is \"reparameterized\", i.e., a diffeomorphic transformation of a\n parameterless distribution (e.g.,\n `Normal(Y; m, s) <=> Y = sX + m, X ~ Normal(0,1)`), we can swap gradient and\n expectation, i.e.,\n `grad[Avg{ s_i : i=1...n }] = Avg{ grad[s_i] : i=1...n }` where `S_n=Avg{s_i}`\n and `s_i = f(x_i), x_i ~iid q(X)`.\n\n However, if q is not reparameterized, TensorFlow's gradient will be incorrect\n since the chain-rule stops at samples of unreparameterized distributions. In\n this circumstance using the Score-Gradient trick results in an unbiased\n gradient, i.e.,\n\n ```none\n grad[ E_q[f(X)] ]\n = grad[ int dx q(x) f(x) ]\n = int dx grad[ q(x) f(x) ]\n = int dx [ q'(x) f(x) + q(x) f'(x) ]\n = int dx q(x) [q'(x) / q(x) f(x) + f'(x) ]\n = int dx q(x) grad[ f(x) q(x) / stop_grad[q(x)] ]\n = E_q[ grad[ f(x) q(x) / stop_grad[q(x)] ] ]\n ```\n\n Unless `q.reparameterization_type != tfd.FULLY_REPARAMETERIZED` it is\n usually preferable to set `use_reparametrization = True`.\n\n Example Application:\n\n The Csiszar f-Divergence is a useful framework for variational inference.\n I.e., observe that,\n\n ```none\n f(p(x)) = f( E_{q(Z | x)}[ p(x, Z) / q(Z | x) ] )\n <= E_{q(Z | x)}[ f( p(x, Z) / q(Z | x) ) ]\n := D_f[p(x, Z), q(Z | x)]\n ```\n\n The inequality follows from the fact that the \"perspective\" of `f`, i.e.,\n `(s, t) |-> t f(s / t))`, is convex in `(s, t)` when `s/t in domain(f)` and\n `t` is a real. Since the above framework includes the popular Evidence Lower\n BOund (ELBO) as a special case, i.e., `f(u) = -log(u)`, we call this framework\n \"Evidence Divergence Bound Optimization\" (EDBO).\n\n Args:\n f: Python `callable` representing a Csiszar-function in log-space, i.e.,\n takes `p_log_prob(q_samples) - q.log_prob(q_samples)`.\n p_log_prob: Python `callable` taking (a batch of) samples from `q` and\n returning the natural-log of the probability under distribution `p`.\n (In variational inference `p` is the joint distribution.)\n q: `tf.Distribution`-like instance; must implement:\n `reparameterization_type`, `sample(n, seed)`, and `log_prob(x)`.\n (In variational inference `q` is the approximate posterior distribution.)\n num_draws: Integer scalar number of draws used to approximate the\n f-Divergence expectation.\n use_reparametrization: Python `bool`. When `None` (the default),\n automatically set to:\n `q.reparameterization_type == tfd.FULLY_REPARAMETERIZED`.\n When `True` uses the standard Monte-Carlo average. When `False` uses the\n score-gradient trick. (See above for details.) When `False`, consider\n using `csiszar_vimco`.\n seed: Python `int` seed for `q.sample`.\n name: Python `str` name prefixed to Ops created by this function.\n\n Returns:\n monte_carlo_csiszar_f_divergence: `float`-like `Tensor` Monte Carlo\n approximation of the Csiszar f-Divergence.\n\n Raises:\n ValueError: if `q` is not a reparameterized distribution and\n `use_reparametrization = True`. A distribution `q` is said to be\n \"reparameterized\" when its samples are generated by transforming the\n samples of another distribution which does not depend on the\n parameterization of `q`. This property ensures the gradient (with respect\n to parameters) is valid.\n TypeError: if `p_log_prob` is not a Python `callable`.\n \"\"\"\n reparameterization_types = tf.nest.flatten(q.reparameterization_type)\n with tf.compat.v1.name_scope(name, \"monte_carlo_csiszar_f_divergence\",\n [num_draws]):\n if use_reparametrization is None:\n use_reparametrization = all(\n reparameterization_type == tfd.FULLY_REPARAMETERIZED\n for reparameterization_type in reparameterization_types)\n elif (use_reparametrization and\n any(reparameterization_type != tfd.FULLY_REPARAMETERIZED\n for reparameterization_type in reparameterization_types)):\n # TODO(jvdillon): Consider only raising an exception if the gradient is\n # requested.\n raise ValueError(\n \"Distribution `q` must be reparameterized, i.e., a diffeomorphic \"\n \"transformation of a parameterless distribution. (Otherwise this \"\n \"function has a biased gradient.)\")\n if not callable(p_log_prob):\n raise TypeError(\"`p_log_prob` must be a Python `callable` function.\")\n return monte_carlo.expectation(\n f=lambda q_samples: f(p_log_prob(q_samples) - q.log_prob(q_samples)),\n samples=q.sample(num_draws, seed=seed),\n log_prob=q.log_prob, # Only used if use_reparametrization=False.\n use_reparametrization=use_reparametrization)", "response": "Monte - Carlo approximation of the Csiszar - function f - Divergence."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef csiszar_vimco(f,\n p_log_prob,\n q,\n num_draws,\n num_batch_draws=1,\n seed=None,\n name=None):\n \"\"\"Use VIMCO to lower the variance of gradient[csiszar_function(Avg(logu))].\n\n This function generalizes VIMCO [(Mnih and Rezende, 2016)][1] to Csiszar\n f-Divergences.\n\n Note: if `q.reparameterization_type = tfd.FULLY_REPARAMETERIZED`,\n consider using `monte_carlo_csiszar_f_divergence`.\n\n The VIMCO loss is:\n\n ```none\n vimco = f(Avg{logu[i] : i=0,...,m-1})\n where,\n logu[i] = log( p(x, h[i]) / q(h[i] | x) )\n h[i] iid~ q(H | x)\n ```\n\n Interestingly, the VIMCO gradient is not the naive gradient of `vimco`.\n Rather, it is characterized by:\n\n ```none\n grad[vimco] - variance_reducing_term\n where,\n variance_reducing_term = Sum{ grad[log q(h[i] | x)] *\n (vimco - f(log Avg{h[j;i] : j=0,...,m-1}))\n : i=0, ..., m-1 }\n h[j;i] = { u[j] j!=i\n { GeometricAverage{ u[k] : k!=i} j==i\n ```\n\n (We omitted `stop_gradient` for brevity. See implementation for more details.)\n\n The `Avg{h[j;i] : j}` term is a kind of \"swap-out average\" where the `i`-th\n element has been replaced by the leave-`i`-out Geometric-average.\n\n This implementation prefers numerical precision over efficiency, i.e.,\n `O(num_draws * num_batch_draws * prod(batch_shape) * prod(event_shape))`.\n (The constant may be fairly large, perhaps around 12.)\n\n Args:\n f: Python `callable` representing a Csiszar-function in log-space.\n p_log_prob: Python `callable` representing the natural-log of the\n probability under distribution `p`. (In variational inference `p` is the\n joint distribution.)\n q: `tf.Distribution`-like instance; must implement: `sample(n, seed)`, and\n `log_prob(x)`. (In variational inference `q` is the approximate posterior\n distribution.)\n num_draws: Integer scalar number of draws used to approximate the\n f-Divergence expectation.\n num_batch_draws: Integer scalar number of draws used to approximate the\n f-Divergence expectation.\n seed: Python `int` seed for `q.sample`.\n name: Python `str` name prefixed to Ops created by this function.\n\n Returns:\n vimco: The Csiszar f-Divergence generalized VIMCO objective.\n\n Raises:\n ValueError: if `num_draws < 2`.\n\n #### References\n\n [1]: Andriy Mnih and Danilo Rezende. Variational Inference for Monte Carlo\n objectives. In _International Conference on Machine Learning_, 2016.\n https://arxiv.org/abs/1602.06725\n \"\"\"\n with tf.compat.v1.name_scope(name, \"csiszar_vimco\",\n [num_draws, num_batch_draws]):\n if num_draws < 2:\n raise ValueError(\"Must specify num_draws > 1.\")\n stop = tf.stop_gradient # For readability.\n x = stop(q.sample(sample_shape=[num_draws, num_batch_draws],\n seed=seed))\n logqx = q.log_prob(x)\n logu = p_log_prob(x) - logqx\n f_log_avg_u, f_log_sooavg_u = [f(r) for r in csiszar_vimco_helper(logu)]\n dotprod = tf.reduce_sum(\n input_tensor=logqx * stop(f_log_avg_u - f_log_sooavg_u),\n axis=0) # Sum over iid samples.\n # We now rewrite f_log_avg_u so that:\n # `grad[f_log_avg_u] := grad[f_log_avg_u + dotprod]`.\n # To achieve this, we use a trick that\n # `f(x) - stop(f(x)) == zeros_like(f(x))`\n # but its gradient is grad[f(x)].\n # Note that IEEE754 specifies that `x - x == 0.` and `x + 0. == x`, hence\n # this trick loses no precision. For more discussion regarding the relevant\n # portions of the IEEE754 standard, see the StackOverflow question,\n # \"Is there a floating point value of x, for which x-x == 0 is false?\"\n # http://stackoverflow.com/q/2686644\n f_log_avg_u += dotprod - stop(dotprod) # Add zeros_like(dot_prod).\n return tf.reduce_mean(input_tensor=f_log_avg_u, axis=0)", "response": "This function is used to lower the variance of gradient of vimco."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef csiszar_vimco_helper(logu, name=None):\n with tf.compat.v1.name_scope(name, \"csiszar_vimco_helper\", [logu]):\n logu = tf.convert_to_tensor(value=logu, name=\"logu\")\n\n n = tf.compat.dimension_value(logu.shape.with_rank_at_least(1)[0])\n if n is None:\n n = tf.shape(input=logu)[0]\n log_n = tf.math.log(tf.cast(n, dtype=logu.dtype))\n nm1 = tf.cast(n - 1, dtype=logu.dtype)\n else:\n log_n = np.log(n).astype(logu.dtype.as_numpy_dtype)\n nm1 = np.asarray(n - 1, dtype=logu.dtype.as_numpy_dtype)\n\n # Throughout we reduce across axis=0 since this is presumed to be iid\n # samples.\n\n log_max_u = tf.reduce_max(input_tensor=logu, axis=0)\n log_sum_u_minus_log_max_u = tf.reduce_logsumexp(\n input_tensor=logu - log_max_u, axis=0)\n\n # log_loosum_u[i] =\n # = logsumexp(logu[j] : j != i)\n # = log( exp(logsumexp(logu)) - exp(logu[i]) )\n # = log( exp(logsumexp(logu - logu[i])) exp(logu[i]) - exp(logu[i]))\n # = logu[i] + log(exp(logsumexp(logu - logu[i])) - 1)\n # = logu[i] + log(exp(logsumexp(logu) - logu[i]) - 1)\n # = logu[i] + softplus_inverse(logsumexp(logu) - logu[i])\n d = log_sum_u_minus_log_max_u + (log_max_u - logu)\n # We use `d != 0` rather than `d > 0.` because `d < 0.` should never\n # happens; if it does we want to complain loudly (which `softplus_inverse`\n # will).\n d_ok = tf.not_equal(d, 0.)\n safe_d = tf.where(d_ok, d, tf.ones_like(d))\n d_ok_result = logu + tfd.softplus_inverse(safe_d)\n\n inf = np.array(np.inf, dtype=logu.dtype.as_numpy_dtype)\n\n # When not(d_ok) and is_positive_and_largest then we manually compute the\n # log_loosum_u. (We can efficiently do this for any one point but not all,\n # hence we still need the above calculation.) This is good because when\n # this condition is met, we cannot use the above calculation; its -inf.\n # We now compute the log-leave-out-max-sum, replicate it to every\n # point and make sure to select it only when we need to.\n is_positive_and_largest = tf.logical_and(\n logu > 0.,\n tf.equal(logu, log_max_u[tf.newaxis, ...]))\n log_lomsum_u = tf.reduce_logsumexp(\n input_tensor=tf.where(is_positive_and_largest,\n tf.fill(tf.shape(input=logu), -inf), logu),\n axis=0,\n keepdims=True)\n log_lomsum_u = tf.tile(\n log_lomsum_u,\n multiples=1 + tf.pad(tensor=[n - 1], paddings=[[0, tf.rank(logu) - 1]]))\n\n d_not_ok_result = tf.where(is_positive_and_largest, log_lomsum_u,\n tf.fill(tf.shape(input=d), -inf))\n\n log_loosum_u = tf.where(d_ok, d_ok_result, d_not_ok_result)\n\n # The swap-one-out-sum (\"soosum\") is n different sums, each of which\n # replaces the i-th item with the i-th-left-out average, i.e.,\n # soo_sum_u[i] = [exp(logu) - exp(logu[i])] + exp(mean(logu[!=i]))\n # = exp(log_loosum_u[i]) + exp(looavg_logu[i])\n looavg_logu = (tf.reduce_sum(input_tensor=logu, axis=0) - logu) / nm1\n log_soosum_u = tf.reduce_logsumexp(\n input_tensor=tf.stack([log_loosum_u, looavg_logu]), axis=0)\n\n log_avg_u = log_sum_u_minus_log_max_u + log_max_u - log_n\n log_sooavg_u = log_soosum_u - log_n\n\n log_avg_u.set_shape(logu.shape.with_rank_at_least(1)[1:])\n log_sooavg_u.set_shape(logu.shape)\n\n return log_avg_u, log_sooavg_u", "response": "Helper to `csiszar_vimco`; computes `log_avg_u`, `log_sooavg_u`.\n\n `axis = 0` of `logu` is presumed to correspond to iid samples from `q`, i.e.,\n\n ```none\n logu[j] = log(u[j])\n u[j] = p(x, h[j]) / q(h[j] | x)\n h[j] iid~ q(H | x)\n ```\n\n Args:\n logu: Floating-type `Tensor` representing `log(p(x, h) / q(h | x))`.\n name: Python `str` name prefixed to Ops created by this function.\n\n Returns:\n log_avg_u: `logu.dtype` `Tensor` corresponding to the natural-log of the\n average of `u`. The sum of the gradient of `log_avg_u` is `1`.\n log_sooavg_u: `logu.dtype` `Tensor` characterized by the natural-log of the\n average of `u`` except that the average swaps-out `u[i]` for the\n leave-`i`-out Geometric-average. The mean of the gradient of\n `log_sooavg_u` is `1`. Mathematically `log_sooavg_u` is,\n ```none\n log_sooavg_u[i] = log(Avg{h[j ; i] : j=0, ..., m-1})\n h[j ; i] = { u[j] j!=i\n { GeometricAverage{u[k] : k != i} j==i\n ```"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef interp_regular_1d_grid(x,\n x_ref_min,\n x_ref_max,\n y_ref,\n axis=-1,\n fill_value='constant_extension',\n fill_value_below=None,\n fill_value_above=None,\n grid_regularizing_transform=None,\n name=None):\n \"\"\"Linear `1-D` interpolation on a regular (constant spacing) grid.\n\n Given reference values, this function computes a piecewise linear interpolant\n and evaluates it on a new set of `x` values.\n\n The interpolant is built from `C` reference values indexed by one dimension\n of `y_ref` (specified by the `axis` kwarg).\n\n If `y_ref` is a vector, then each value `y_ref[i]` is considered to be equal\n to `f(x_ref[i])`, for `C` (implicitly defined) reference values between\n `x_ref_min` and `x_ref_max`:\n\n ```none\n x_ref[i] = x_ref_min + i * (x_ref_max - x_ref_min) / (C - 1),\n i = 0, ..., C - 1.\n ```\n\n If `rank(y_ref) > 1`, then dimension `axis` indexes `C` reference values of\n a shape `y_ref.shape[:axis] + y_ref.shape[axis + 1:]` `Tensor`.\n\n If `rank(x) > 1`, then the output is obtained by effectively flattening `x`,\n interpolating along `axis`, then expanding the result to shape\n `y_ref.shape[:axis] + x.shape + y_ref.shape[axis + 1:]`.\n\n These shape semantics are equivalent to `scipy.interpolate.interp1d`.\n\n Args:\n x: Numeric `Tensor` The x-coordinates of the interpolated output values.\n x_ref_min: Scalar `Tensor` of same `dtype` as `x`. The minimum value of\n the (implicitly defined) reference `x_ref`.\n x_ref_max: Scalar `Tensor` of same `dtype` as `x`. The maximum value of\n the (implicitly defined) reference `x_ref`.\n y_ref: `N-D` `Tensor` (`N > 0`) of same `dtype` as `x`. The reference\n output values.\n axis: Scalar `Tensor` designating the dimension of `y_ref` that indexes\n values of the interpolation table.\n Default value: `-1`, the rightmost axis.\n fill_value: Determines what values output should take for `x` values that\n are below `x_ref_min` or above `x_ref_max`. `Tensor` or one of the strings\n \"constant_extension\" ==> Extend as constant function. \"extrapolate\" ==>\n Extrapolate in a linear fashion.\n Default value: `\"constant_extension\"`\n fill_value_below: Optional override of `fill_value` for `x < x_ref_min`.\n fill_value_above: Optional override of `fill_value` for `x > x_ref_max`.\n grid_regularizing_transform: Optional transformation `g` which regularizes\n the implied spacing of the x reference points. In other words, if\n provided, we assume `g(x_ref_i)` is a regular grid between `g(x_ref_min)`\n and `g(x_ref_max)`.\n name: A name to prepend to created ops.\n Default value: `\"interp_regular_1d_grid\"`.\n\n Returns:\n y_interp: Interpolation between members of `y_ref`, at points `x`.\n `Tensor` of same `dtype` as `x`, and shape\n `y.shape[:axis] + x.shape + y.shape[axis + 1:]`\n\n Raises:\n ValueError: If `fill_value` is not an allowed string.\n ValueError: If `axis` is not a scalar.\n\n #### Examples\n\n Interpolate a function of one variable:\n\n ```python\n y_ref = tf.exp(tf.linspace(start=0., stop=10., num=200))\n\n interp_regular_1d_grid(\n x=[6.0, 0.5, 3.3], x_ref_min=0., x_ref_max=1., y_ref=y_ref)\n ==> approx [exp(6.0), exp(0.5), exp(3.3)]\n ```\n\n Interpolate a matrix-valued function of one variable:\n\n ```python\n mat_0 = [[1., 0.], [0., 1.]]\n mat_1 = [[0., -1], [1, 0]]\n y_ref = [mat_0, mat_1]\n\n # Get three output matrices at once.\n tfp.math.interp_regular_1d_grid(\n x=[0., 0.5, 1.], x_ref_min=0., x_ref_max=1., y_ref=y_ref, axis=0)\n ==> [mat_0, 0.5 * mat_0 + 0.5 * mat_1, mat_1]\n ```\n\n Interpolate a scalar valued function, and get a matrix of results:\n\n ```python\n y_ref = tf.exp(tf.linspace(start=0., stop=10., num=200))\n x = [[1.1, 1.2], [2.1, 2.2]]\n tfp.math.interp_regular_1d_grid(x, x_ref_min=0., x_ref_max=10., y_ref=y_ref)\n ==> tf.exp(x)\n ```\n\n Interpolate a function of one variable on a log-spaced grid:\n\n ```python\n x_ref = tf.exp(tf.linspace(tf.log(1.), tf.log(100000.), num_pts))\n y_ref = tf.log(x_ref + x_ref**2)\n\n interp_regular_1d_grid(x=[1.1, 2.2], x_ref_min=1., x_ref_max=100000., y_ref,\n grid_regularizing_transform=tf.log)\n ==> [tf.log(1.1 + 1.1**2), tf.log(2.2 + 2.2**2)]\n ```\n\n \"\"\"\n\n return _interp_regular_1d_grid_impl(\n x,\n x_ref_min,\n x_ref_max,\n y_ref,\n axis=axis,\n batch_y_ref=False,\n fill_value=fill_value,\n fill_value_below=fill_value_below,\n fill_value_above=fill_value_above,\n grid_regularizing_transform=grid_regularizing_transform,\n name=name or 'interp_regular_1d_grid')", "response": "Linear 1 - D interpolation on regular grids."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef batch_interp_regular_1d_grid(x,\n x_ref_min,\n x_ref_max,\n y_ref,\n axis=-1,\n fill_value='constant_extension',\n fill_value_below=None,\n fill_value_above=None,\n grid_regularizing_transform=None,\n name=None):\n \"\"\"Linear `1-D` interpolation on a regular (constant spacing) grid.\n\n Given [batch of] reference values, this function computes a piecewise linear\n interpolant and evaluates it on a [batch of] of new `x` values.\n\n The interpolant is built from `C` reference values indexed by one dimension\n of `y_ref` (specified by the `axis` kwarg).\n\n If `y_ref` is a vector, then each value `y_ref[i]` is considered to be equal\n to `f(x_ref[i])`, for `C` (implicitly defined) reference values between\n `x_ref_min` and `x_ref_max`:\n\n ```none\n x_ref[i] = x_ref_min + i * (x_ref_max - x_ref_min) / (C - 1),\n i = 0, ..., C - 1.\n ```\n\n In the general case, dimensions to the left of `axis` in `y_ref` are broadcast\n with leading dimensions in `x`, `x_ref_min`, `x_ref_max`.\n\n Args:\n x: Numeric `Tensor` The x-coordinates of the interpolated output values for\n each batch. Shape broadcasts with `[A1, ..., AN, D]`, `N >= 0`.\n x_ref_min: `Tensor` of same `dtype` as `x`. The minimum value of the each\n batch of the (implicitly defined) reference `x_ref`. Shape broadcasts with\n `[A1, ..., AN]`, `N >= 0`.\n x_ref_max: `Tensor` of same `dtype` as `x`. The maximum value of the each\n batch of the (implicitly defined) reference `x_ref`. Shape broadcasts with\n `[A1, ..., AN]`, `N >= 0`.\n y_ref: `Tensor` of same `dtype` as `x`. The reference output values.\n `y_ref.shape[:axis]` broadcasts with the batch shape `[A1, ..., AN]`, and\n `y_ref.shape[axis:]` is `[C, B1, ..., BM]`, so the trailing dimensions\n index `C` reference values of a rank `M` `Tensor` (`M >= 0`).\n axis: Scalar `Tensor` designating the dimension of `y_ref` that indexes\n values of the interpolation table.\n Default value: `-1`, the rightmost axis.\n fill_value: Determines what values output should take for `x` values that\n are below `x_ref_min` or above `x_ref_max`. `Tensor` or one of the strings\n \"constant_extension\" ==> Extend as constant function. \"extrapolate\" ==>\n Extrapolate in a linear fashion.\n Default value: `\"constant_extension\"`\n fill_value_below: Optional override of `fill_value` for `x < x_ref_min`.\n fill_value_above: Optional override of `fill_value` for `x > x_ref_max`.\n grid_regularizing_transform: Optional transformation `g` which regularizes\n the implied spacing of the x reference points. In other words, if\n provided, we assume `g(x_ref_i)` is a regular grid between `g(x_ref_min)`\n and `g(x_ref_max)`.\n name: A name to prepend to created ops.\n Default value: `\"batch_interp_regular_1d_grid\"`.\n\n Returns:\n y_interp: Interpolation between members of `y_ref`, at points `x`.\n `Tensor` of same `dtype` as `x`, and shape `[A1, ..., AN, D, B1, ..., BM]`\n\n Raises:\n ValueError: If `fill_value` is not an allowed string.\n ValueError: If `axis` is not a scalar.\n\n #### Examples\n\n Interpolate a function of one variable:\n\n ```python\n y_ref = tf.exp(tf.linspace(start=0., stop=10., 20))\n\n batch_interp_regular_1d_grid(\n x=[6.0, 0.5, 3.3], x_ref_min=0., x_ref_max=1., y_ref=y_ref)\n ==> approx [exp(6.0), exp(0.5), exp(3.3)]\n ```\n\n Interpolate a batch of functions of one variable.\n\n ```python\n # First batch member is an exponential function, second is a log.\n implied_x_ref = [tf.linspace(-3., 3.2, 200), tf.linspace(0.5, 3., 200)]\n y_ref = tf.stack( # Shape [2, 200], 2 batches, 200 reference values per batch\n [tf.exp(implied_x_ref[0]), tf.log(implied_x_ref[1])], axis=0)\n\n x = [[-1., 1., 0.], # Shape [2, 3], 2 batches, 3 values per batch.\n [1., 2., 3.]]\n\n y = tfp.math.batch_interp_regular_1d_grid( # Shape [2, 3]\n x,\n x_ref_min=[-3., 0.5],\n x_ref_max=[3.2, 3.],\n y_ref=y_ref,\n axis=-1)\n\n # y[0] approx tf.exp(x[0])\n # y[1] approx tf.log(x[1])\n ```\n\n Interpolate a function of one variable on a log-spaced grid:\n\n ```python\n x_ref = tf.exp(tf.linspace(tf.log(1.), tf.log(100000.), num_pts))\n y_ref = tf.log(x_ref + x_ref**2)\n\n batch_interp_regular_1d_grid(x=[1.1, 2.2], x_ref_min=1., x_ref_max=100000.,\n y_ref, grid_regularizing_transform=tf.log)\n ==> [tf.log(1.1 + 1.1**2), tf.log(2.2 + 2.2**2)]\n ```\n\n \"\"\"\n\n return _interp_regular_1d_grid_impl(\n x,\n x_ref_min,\n x_ref_max,\n y_ref,\n axis=axis,\n batch_y_ref=True,\n fill_value=fill_value,\n fill_value_below=fill_value_below,\n fill_value_above=fill_value_above,\n grid_regularizing_transform=grid_regularizing_transform,\n name=name or 'batch_interp_regular_1d_grid')", "response": "This function computes a piecewise linear interpolation on a regular 1 - D grid."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef batch_interp_regular_nd_grid(x,\n x_ref_min,\n x_ref_max,\n y_ref,\n axis,\n fill_value='constant_extension',\n name=None):\n \"\"\"Multi-linear interpolation on a regular (constant spacing) grid.\n\n Given [a batch of] reference values, this function computes a multi-linear\n interpolant and evaluates it on [a batch of] of new `x` values.\n\n The interpolant is built from reference values indexed by `nd` dimensions\n of `y_ref`, starting at `axis`.\n\n For example, take the case of a `2-D` scalar valued function and no leading\n batch dimensions. In this case, `y_ref.shape = [C1, C2]` and `y_ref[i, j]`\n is the reference value corresponding to grid point\n\n ```\n [x_ref_min[0] + i * (x_ref_max[0] - x_ref_min[0]) / (C1 - 1),\n x_ref_min[1] + j * (x_ref_max[1] - x_ref_min[1]) / (C2 - 1)]\n ```\n\n In the general case, dimensions to the left of `axis` in `y_ref` are broadcast\n with leading dimensions in `x`, `x_ref_min`, `x_ref_max`.\n\n Args:\n x: Numeric `Tensor` The x-coordinates of the interpolated output values for\n each batch. Shape `[..., D, nd]`, designating [a batch of] `D`\n coordinates in `nd` space. `D` must be `>= 1` and is not a batch dim.\n x_ref_min: `Tensor` of same `dtype` as `x`. The minimum values of the\n (implicitly defined) reference `x_ref`. Shape `[..., nd]`.\n x_ref_max: `Tensor` of same `dtype` as `x`. The maximum values of the\n (implicitly defined) reference `x_ref`. Shape `[..., nd]`.\n y_ref: `Tensor` of same `dtype` as `x`. The reference output values. Shape\n `[..., C1, ..., Cnd, B1,...,BM]`, designating [a batch of] reference\n values indexed by `nd` dimensions, of a shape `[B1,...,BM]` valued\n function (for `M >= 0`).\n axis: Scalar integer `Tensor`. Dimensions `[axis, axis + nd)` of `y_ref`\n index the interpolation table. E.g. `3-D` interpolation of a scalar\n valued function requires `axis=-3` and a `3-D` matrix valued function\n requires `axis=-5`.\n fill_value: Determines what values output should take for `x` values that\n are below `x_ref_min` or above `x_ref_max`. Scalar `Tensor` or\n \"constant_extension\" ==> Extend as constant function.\n Default value: `\"constant_extension\"`\n name: A name to prepend to created ops.\n Default value: `\"batch_interp_regular_nd_grid\"`.\n\n Returns:\n y_interp: Interpolation between members of `y_ref`, at points `x`.\n `Tensor` of same `dtype` as `x`, and shape `[..., D, B1, ..., BM].`\n\n Raises:\n ValueError: If `rank(x) < 2` is determined statically.\n ValueError: If `axis` is not a scalar is determined statically.\n ValueError: If `axis + nd > rank(y_ref)` is determined statically.\n\n #### Examples\n\n Interpolate a function of one variable.\n\n ```python\n y_ref = tf.exp(tf.linspace(start=0., stop=10., 20))\n\n tfp.math.batch_interp_regular_nd_grid(\n # x.shape = [3, 1], x_ref_min/max.shape = [1]. Trailing `1` for `1-D`.\n x=[[6.0], [0.5], [3.3]], x_ref_min=[0.], x_ref_max=[1.], y_ref=y_ref)\n ==> approx [exp(6.0), exp(0.5), exp(3.3)]\n ```\n\n Interpolate a scalar function of two variables.\n\n ```python\n x_ref_min = [0., 2 * np.pi]\n x_ref_max = [0., 2 * np.pi]\n\n # Build y_ref.\n x0s, x1s = tf.meshgrid(\n tf.linspace(x_ref_min[0], x_ref_max[0], num=100),\n tf.linspace(x_ref_min[1], x_ref_max[1], num=100),\n indexing='ij')\n\n def func(x0, x1):\n return tf.sin(x0) * tf.cos(x1)\n\n y_ref = func(x0s, x1s)\n\n x = np.pi * tf.random_uniform(shape=(10, 2))\n\n tfp.math.batch_interp_regular_nd_grid(x, x_ref_min, x_ref_max, y_ref, axis=-2)\n ==> tf.sin(x[:, 0]) * tf.cos(x[:, 1])\n ```\n\n \"\"\"\n with tf.compat.v1.name_scope(\n name,\n default_name='interp_regular_nd_grid',\n values=[x, x_ref_min, x_ref_max, y_ref, fill_value]):\n dtype = dtype_util.common_dtype([x, x_ref_min, x_ref_max, y_ref],\n preferred_dtype=tf.float32)\n\n # Arg checking.\n if isinstance(fill_value, str):\n if fill_value != 'constant_extension':\n raise ValueError(\n 'A fill value ({}) was not an allowed string ({})'.format(\n fill_value, 'constant_extension'))\n else:\n fill_value = tf.convert_to_tensor(\n value=fill_value, name='fill_value', dtype=dtype)\n _assert_ndims_statically(fill_value, expect_ndims=0)\n\n # x.shape = [..., nd].\n x = tf.convert_to_tensor(value=x, name='x', dtype=dtype)\n _assert_ndims_statically(x, expect_ndims_at_least=2)\n\n # y_ref.shape = [..., C1,...,Cnd, B1,...,BM]\n y_ref = tf.convert_to_tensor(value=y_ref, name='y_ref', dtype=dtype)\n\n # x_ref_min.shape = [nd]\n x_ref_min = tf.convert_to_tensor(\n value=x_ref_min, name='x_ref_min', dtype=dtype)\n x_ref_max = tf.convert_to_tensor(\n value=x_ref_max, name='x_ref_max', dtype=dtype)\n _assert_ndims_statically(\n x_ref_min, expect_ndims_at_least=1, expect_static=True)\n _assert_ndims_statically(\n x_ref_max, expect_ndims_at_least=1, expect_static=True)\n\n # nd is the number of dimensions indexing the interpolation table, it's the\n # \"nd\" in the function name.\n nd = tf.compat.dimension_value(x_ref_min.shape[-1])\n if nd is None:\n raise ValueError('`x_ref_min.shape[-1]` must be known statically.')\n x_ref_max.shape[-1:].assert_is_compatible_with(x_ref_min.shape[-1:])\n\n # Convert axis and check it statically.\n axis = tf.convert_to_tensor(value=axis, dtype=tf.int32, name='axis')\n axis = distribution_util.make_non_negative_axis(axis, tf.rank(y_ref))\n axis.shape.assert_has_rank(0)\n axis_ = tf.get_static_value(axis)\n y_ref_rank_ = tf.get_static_value(tf.rank(y_ref))\n if axis_ is not None and y_ref_rank_ is not None:\n if axis_ + nd > y_ref_rank_:\n raise ValueError(\n 'Since dims `[axis, axis + nd)` index the interpolation table, we '\n 'must have `axis + nd <= rank(y_ref)`. Found: '\n '`axis`: {}, rank(y_ref): {}, and inferred `nd` from trailing '\n 'dimensions of `x_ref_min` to be {}.'.format(\n axis_, y_ref_rank_, nd))\n\n x_batch_shape = tf.shape(input=x)[:-2]\n x_ref_min_batch_shape = tf.shape(input=x_ref_min)[:-1]\n x_ref_max_batch_shape = tf.shape(input=x_ref_max)[:-1]\n y_ref_batch_shape = tf.shape(input=y_ref)[:axis]\n\n # Do a brute-force broadcast of batch dims (add zeros).\n batch_shape = y_ref_batch_shape\n for tensor in [x_batch_shape, x_ref_min_batch_shape, x_ref_max_batch_shape]:\n batch_shape = tf.broadcast_dynamic_shape(batch_shape, tensor)\n\n def _batch_of_zeros_with_rightmost_singletons(n_singletons):\n \"\"\"Return Tensor of zeros with some singletons on the rightmost dims.\"\"\"\n ones = tf.ones(shape=[n_singletons], dtype=tf.int32)\n return tf.zeros(shape=tf.concat([batch_shape, ones], axis=0), dtype=dtype)\n\n x += _batch_of_zeros_with_rightmost_singletons(n_singletons=2)\n x_ref_min += _batch_of_zeros_with_rightmost_singletons(n_singletons=1)\n x_ref_max += _batch_of_zeros_with_rightmost_singletons(n_singletons=1)\n y_ref += _batch_of_zeros_with_rightmost_singletons(\n n_singletons=tf.rank(y_ref) - axis)\n\n return _batch_interp_with_gather_nd(\n x=x,\n x_ref_min=x_ref_min,\n x_ref_max=x_ref_max,\n y_ref=y_ref,\n nd=nd,\n fill_value=fill_value,\n batch_dims=tf.get_static_value(tf.rank(x)) - 2)", "response": "This function computes a multi - linear interpolation on a regular ND grid."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _assert_ndims_statically(x,\n expect_ndims=None,\n expect_ndims_at_least=None,\n expect_static=False):\n \"\"\"Assert that Tensor x has expected number of dimensions.\"\"\"\n ndims = x.shape.ndims\n if ndims is None:\n if expect_static:\n raise ValueError('Expected static ndims. Found: {}'.format(x))\n return\n if expect_ndims is not None and ndims != expect_ndims:\n raise ValueError('ndims must be {}. Found: {}'.format(expect_ndims, ndims))\n if expect_ndims_at_least is not None and ndims < expect_ndims_at_least:\n raise ValueError('ndims must be at least {}. Found {}'.format(\n expect_ndims_at_least, ndims))", "response": "Assert that Tensor x has expected number of dimensions."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef _make_expand_x_fn_for_non_batch_interpolation(y_ref, axis):\n # This expansion is to help x broadcast with `y`, the output.\n # In the non-batch case, the output shape is going to be\n # y_ref.shape[:axis] + x.shape + y_ref.shape[axis+1:]\n\n # Recall we made axis non-negative\n y_ref_shape = tf.shape(input=y_ref)\n y_ref_shape_left = y_ref_shape[:axis]\n y_ref_shape_right = y_ref_shape[axis + 1:]\n\n def expand_ends(x, broadcast=False):\n \"\"\"Expand x so it can bcast w/ tensors of output shape.\"\"\"\n # Assume out_shape = A + x.shape + B, and rank(A) = axis.\n # Expand with singletons with same rank as A, B.\n expanded_shape = tf.pad(\n tensor=tf.shape(input=x),\n paddings=[[axis, tf.size(input=y_ref_shape_right)]],\n constant_values=1)\n x_expanded = tf.reshape(x, expanded_shape)\n\n if broadcast:\n out_shape = tf.concat((\n y_ref_shape_left,\n tf.shape(input=x),\n y_ref_shape_right,\n ),\n axis=0)\n if x.dtype.is_bool:\n x_expanded = x_expanded | tf.cast(tf.zeros(out_shape), tf.bool)\n else:\n x_expanded += tf.zeros(out_shape, dtype=x.dtype)\n return x_expanded\n\n return expand_ends", "response": "Make function to expand left and right dims of tensors shaped like x."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nmake function to expand left and right dimensions of tensors of x.", "response": "def _make_expand_x_fn_for_batch_interpolation(y_ref, axis):\n \"\"\"Make func to expand left/right (of axis) dims of tensors shaped like x.\"\"\"\n # This expansion is to help x broadcast with `y`, the output.\n # In the batch case, the output shape is going to be\n # Broadcast(y_ref.shape[:axis], x.shape[:-1]) +\n # x.shape[-1:] + y_ref.shape[axis+1:]\n\n # Recall we made axis non-negative\n y_ref_shape = tf.shape(input=y_ref)\n y_ref_shape_left = y_ref_shape[:axis]\n y_ref_shape_right = y_ref_shape[axis + 1:]\n\n def expand_right_dims(x, broadcast=False):\n \"\"\"Expand x so it can bcast w/ tensors of output shape.\"\"\"\n expanded_shape_left = tf.broadcast_dynamic_shape(\n tf.shape(input=x)[:-1],\n tf.ones([tf.size(input=y_ref_shape_left)], dtype=tf.int32))\n expanded_shape = tf.concat(\n (expanded_shape_left, tf.shape(input=x)[-1:],\n tf.ones([tf.size(input=y_ref_shape_right)], dtype=tf.int32)),\n axis=0)\n x_expanded = tf.reshape(x, expanded_shape)\n if broadcast:\n broadcast_shape_left = tf.broadcast_dynamic_shape(\n tf.shape(input=x)[:-1], y_ref_shape_left)\n broadcast_shape = tf.concat(\n (broadcast_shape_left, tf.shape(input=x)[-1:], y_ref_shape_right),\n axis=0)\n if x.dtype.is_bool:\n x_expanded = x_expanded | tf.cast(tf.zeros(broadcast_shape), tf.bool)\n else:\n x_expanded += tf.zeros(broadcast_shape, dtype=x.dtype)\n return x_expanded\n\n return expand_right_dims"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\nliking batch_gather but broadcasts to the left of axis.", "response": "def _batch_gather_with_broadcast(params, indices, axis):\n \"\"\"Like batch_gather, but broadcasts to the left of axis.\"\"\"\n # batch_gather assumes...\n # params.shape = [A1,...,AN, B1,...,BM]\n # indices.shape = [A1,...,AN, C]\n # which gives output of shape\n # [A1,...,AN, C, B1,...,BM]\n # Here we broadcast dims of each to the left of `axis` in params, and left of\n # the rightmost dim in indices, e.g. we can\n # have\n # params.shape = [A1,...,AN, B1,...,BM]\n # indices.shape = [a1,...,aN, C],\n # where ai broadcasts with Ai.\n\n # leading_bcast_shape is the broadcast of [A1,...,AN] and [a1,...,aN].\n leading_bcast_shape = tf.broadcast_dynamic_shape(\n tf.shape(input=params)[:axis],\n tf.shape(input=indices)[:-1])\n params += tf.zeros(\n tf.concat((leading_bcast_shape, tf.shape(input=params)[axis:]), axis=0),\n dtype=params.dtype)\n indices += tf.zeros(\n tf.concat((leading_bcast_shape, tf.shape(input=indices)[-1:]), axis=0),\n dtype=indices.dtype)\n return tf.compat.v1.batch_gather(params, indices)"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _broadcast_cat_event_and_params(event, params, base_dtype):\n if dtype_util.is_integer(event.dtype):\n pass\n elif dtype_util.is_floating(event.dtype):\n # When `validate_args=True` we've already ensured int/float casting\n # is closed.\n event = tf.cast(event, dtype=tf.int32)\n else:\n raise TypeError(\"`value` should have integer `dtype` or \"\n \"`self.dtype` ({})\".format(base_dtype))\n shape_known_statically = (\n tensorshape_util.rank(params.shape) is not None and\n tensorshape_util.is_fully_defined(params.shape[:-1]) and\n tensorshape_util.is_fully_defined(event.shape))\n if not shape_known_statically or params.shape[:-1] != event.shape:\n params *= tf.ones_like(event[..., tf.newaxis],\n dtype=params.dtype)\n params_shape = tf.shape(input=params)[:-1]\n event *= tf.ones(params_shape, dtype=event.dtype)\n if tensorshape_util.rank(params.shape) is not None:\n tensorshape_util.set_shape(event, params.shape[:-1])\n\n return event, params", "response": "Broadcasts the event or distribution parameters."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef expectation_importance_sampler_logspace(\n log_f,\n log_p,\n sampling_dist_q,\n z=None,\n n=None,\n seed=None,\n name='expectation_importance_sampler_logspace'):\n r\"\"\"Importance sampling with a positive function, in log-space.\n\n With \\\\(p(z) := exp^{log_p(z)}\\\\), and \\\\(f(z) = exp{log_f(z)}\\\\),\n this `Op` returns\n\n \\\\(Log[ n^{-1} sum_{i=1}^n [ f(z_i) p(z_i) / q(z_i) ] ], z_i ~ q,\\\\)\n \\\\(\\approx Log[ E_q[ f(Z) p(Z) / q(Z) ] ]\\\\)\n \\\\(= Log[E_p[f(Z)]]\\\\)\n\n This integral is done in log-space with max-subtraction to better handle the\n often extreme values that `f(z) p(z) / q(z)` can take on.\n\n In contrast to `expectation_importance_sampler`, this `Op` returns values in\n log-space.\n\n\n User supplies either `Tensor` of samples `z`, or number of samples to draw `n`\n\n Args:\n log_f: Callable mapping samples from `sampling_dist_q` to `Tensors` with\n shape broadcastable to `q.batch_shape`.\n For example, `log_f` works \"just like\" `sampling_dist_q.log_prob`.\n log_p: Callable mapping samples from `sampling_dist_q` to `Tensors` with\n shape broadcastable to `q.batch_shape`.\n For example, `log_p` works \"just like\" `q.log_prob`.\n sampling_dist_q: The sampling distribution.\n `tfp.distributions.Distribution`.\n `float64` `dtype` recommended.\n `log_p` and `q` should be supported on the same set.\n z: `Tensor` of samples from `q`, produced by `q.sample` for some `n`.\n n: Integer `Tensor`. Number of samples to generate if `z` is not provided.\n seed: Python integer to seed the random number generator.\n name: A name to give this `Op`.\n\n Returns:\n Logarithm of the importance sampling estimate. `Tensor` with `shape` equal\n to batch shape of `q`, and `dtype` = `q.dtype`.\n \"\"\"\n q = sampling_dist_q\n with tf.name_scope(name):\n z = _get_samples(q, z, n, seed)\n log_values = log_f(z) + log_p(z) - q.log_prob(z)\n return _logspace_mean(log_values)", "response": "r A function that returns a positive function in log - space and a positive function in log - space."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _logspace_mean(log_values):\n # center = Max[Log[values]], with stop-gradient\n # The center hopefully keep the exponentiated term small. It is canceled\n # from the final result, so putting stop gradient on it will not change the\n # final result. We put stop gradient on to eliminate unnecessary computation.\n center = tf.stop_gradient(_sample_max(log_values))\n\n # centered_values = exp{Log[values] - E[Log[values]]}\n centered_values = tf.math.exp(log_values - center)\n\n # log_mean_of_values = Log[ E[centered_values] ] + center\n # = Log[ E[exp{log_values - E[log_values]}] ] + center\n # = Log[E[values]] - E[log_values] + center\n # = Log[E[values]]\n log_mean_of_values = tf.math.log(_sample_mean(centered_values)) + center\n\n return log_mean_of_values", "response": "Evaluate log_values in a stable manner."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef _broadcast_event_and_samples(event, samples, event_ndims):\n # This is the shape of self.samples, without the samples axis, i.e. the shape\n # of the result of a call to dist.sample(). This way we can broadcast it with\n # event to get a properly-sized event, then add the singleton dim back at\n # -event_ndims - 1.\n samples_shape = tf.concat(\n [tf.shape(input=samples)[:-event_ndims - 1],\n tf.shape(input=samples)[tf.rank(samples) - event_ndims:]],\n axis=0)\n event *= tf.ones(samples_shape, dtype=event.dtype)\n event = tf.expand_dims(event, axis=-event_ndims - 1)\n samples *= tf.ones_like(event, dtype=samples.dtype)\n\n return event, samples", "response": "Broadcasts the event or samples."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef one_step(self, current_state, previous_kernel_results):\n with tf.compat.v1.name_scope(\n name=mcmc_util.make_name(self.name, 'mh', 'one_step'),\n values=[current_state, previous_kernel_results]):\n # Take one inner step.\n [\n proposed_state,\n proposed_results,\n ] = self.inner_kernel.one_step(\n current_state,\n previous_kernel_results.accepted_results)\n\n if (not has_target_log_prob(proposed_results) or\n not has_target_log_prob(previous_kernel_results.accepted_results)):\n raise ValueError('\"target_log_prob\" must be a member of '\n '`inner_kernel` results.')\n\n # Compute log(acceptance_ratio).\n to_sum = [proposed_results.target_log_prob,\n -previous_kernel_results.accepted_results.target_log_prob]\n try:\n if (not mcmc_util.is_list_like(\n proposed_results.log_acceptance_correction)\n or proposed_results.log_acceptance_correction):\n to_sum.append(proposed_results.log_acceptance_correction)\n except AttributeError:\n warnings.warn('Supplied inner `TransitionKernel` does not have a '\n '`log_acceptance_correction`. Assuming its value is `0.`')\n log_accept_ratio = mcmc_util.safe_sum(\n to_sum, name='compute_log_accept_ratio')\n\n # If proposed state reduces likelihood: randomly accept.\n # If proposed state increases likelihood: always accept.\n # I.e., u < min(1, accept_ratio), where u ~ Uniform[0,1)\n # ==> log(u) < log_accept_ratio\n log_uniform = tf.math.log(\n tf.random.uniform(\n shape=tf.shape(input=proposed_results.target_log_prob),\n dtype=proposed_results.target_log_prob.dtype.base_dtype,\n seed=self._seed_stream()))\n is_accepted = log_uniform < log_accept_ratio\n\n next_state = mcmc_util.choose(\n is_accepted,\n proposed_state,\n current_state,\n name='choose_next_state')\n\n kernel_results = MetropolisHastingsKernelResults(\n accepted_results=mcmc_util.choose(\n is_accepted,\n proposed_results,\n previous_kernel_results.accepted_results,\n name='choose_inner_results'),\n is_accepted=is_accepted,\n log_accept_ratio=log_accept_ratio,\n proposed_state=proposed_state,\n proposed_results=proposed_results,\n extra=[],\n )\n\n return next_state, kernel_results", "response": "Takes one step of the TransitionKernel."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef bootstrap_results(self, init_state):\n with tf.compat.v1.name_scope(\n name=mcmc_util.make_name(self.name, 'mh', 'bootstrap_results'),\n values=[init_state]):\n pkr = self.inner_kernel.bootstrap_results(init_state)\n if not has_target_log_prob(pkr):\n raise ValueError(\n '\"target_log_prob\" must be a member of `inner_kernel` results.')\n x = pkr.target_log_prob\n return MetropolisHastingsKernelResults(\n accepted_results=pkr,\n is_accepted=tf.ones_like(x, dtype=tf.bool),\n log_accept_ratio=tf.zeros_like(x),\n proposed_state=init_state,\n proposed_results=pkr,\n extra=[],\n )", "response": "Returns an object with the same type as returned by one_step."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\napply the BFGS minimization of a differentiable function using Jorge Nocedal and Stephen Wright.", "response": "def minimize(value_and_gradients_function,\n initial_position,\n tolerance=1e-8,\n x_tolerance=0,\n f_relative_tolerance=0,\n initial_inverse_hessian_estimate=None,\n max_iterations=50,\n parallel_iterations=1,\n stopping_condition=None,\n name=None):\n \"\"\"Applies the BFGS algorithm to minimize a differentiable function.\n\n Performs unconstrained minimization of a differentiable function using the\n BFGS scheme. For details of the algorithm, see [Nocedal and Wright(2006)][1].\n\n ### Usage:\n\n The following example demonstrates the BFGS optimizer attempting to find the\n minimum for a simple two dimensional quadratic objective function.\n\n ```python\n minimum = np.array([1.0, 1.0]) # The center of the quadratic bowl.\n scales = np.array([2.0, 3.0]) # The scales along the two axes.\n\n # The objective function and the gradient.\n def quadratic(x):\n value = tf.reduce_sum(scales * (x - minimum) ** 2)\n return value, tf.gradients(value, x)[0]\n\n start = tf.constant([0.6, 0.8]) # Starting point for the search.\n optim_results = tfp.optimizer.bfgs_minimize(\n quadratic, initial_position=start, tolerance=1e-8)\n\n with tf.Session() as session:\n results = session.run(optim_results)\n # Check that the search converged\n assert(results.converged)\n # Check that the argmin is close to the actual value.\n np.testing.assert_allclose(results.position, minimum)\n # Print out the total number of function evaluations it took. Should be 6.\n print (\"Function evaluations: %d\" % results.num_objective_evaluations)\n ```\n\n ### References:\n [1]: Jorge Nocedal, Stephen Wright. Numerical Optimization. Springer Series in\n Operations Research. pp 136-140. 2006\n http://pages.mtu.edu/~struther/Courses/OLD/Sp2013/5630/Jorge_Nocedal_Numerical_optimization_267490.pdf\n\n Args:\n value_and_gradients_function: A Python callable that accepts a point as a\n real `Tensor` and returns a tuple of `Tensor`s of real dtype containing\n the value of the function and its gradient at that point. The function\n to be minimized. The input should be of shape `[..., n]`, where `n` is\n the size of the domain of input points, and all others are batching\n dimensions. The first component of the return value should be a real\n `Tensor` of matching shape `[...]`. The second component (the gradient)\n should also be of shape `[..., n]` like the input value to the function.\n initial_position: real `Tensor` of shape `[..., n]`. The starting point, or\n points when using batching dimensions, of the search procedure. At these\n points the function value and the gradient norm should be finite.\n tolerance: Scalar `Tensor` of real dtype. Specifies the gradient tolerance\n for the procedure. If the supremum norm of the gradient vector is below\n this number, the algorithm is stopped.\n x_tolerance: Scalar `Tensor` of real dtype. If the absolute change in the\n position between one iteration and the next is smaller than this number,\n the algorithm is stopped.\n f_relative_tolerance: Scalar `Tensor` of real dtype. If the relative change\n in the objective value between one iteration and the next is smaller\n than this value, the algorithm is stopped.\n initial_inverse_hessian_estimate: Optional `Tensor` of the same dtype\n as the components of the output of the `value_and_gradients_function`.\n If specified, the shape should broadcastable to shape `[..., n, n]`; e.g.\n if a single `[n, n]` matrix is provided, it will be automatically\n broadcasted to all batches. Alternatively, one can also specify a\n different hessian estimate for each batch member.\n For the correctness of the algorithm, it is required that this parameter\n be symmetric and positive definite. Specifies the starting estimate for\n the inverse of the Hessian at the initial point. If not specified,\n the identity matrix is used as the starting estimate for the\n inverse Hessian.\n max_iterations: Scalar positive int32 `Tensor`. The maximum number of\n iterations for BFGS updates.\n parallel_iterations: Positive integer. The number of iterations allowed to\n run in parallel.\n stopping_condition: (Optional) A Python function that takes as input two\n Boolean tensors of shape `[...]`, and returns a Boolean scalar tensor.\n The input tensors are `converged` and `failed`, indicating the current\n status of each respective batch member; the return value states whether\n the algorithm should stop. The default is tfp.optimizer.converged_all\n which only stops when all batch members have either converged or failed.\n An alternative is tfp.optimizer.converged_any which stops as soon as one\n batch member has converged, or when all have failed.\n name: (Optional) Python str. The name prefixed to the ops created by this\n function. If not supplied, the default name 'minimize' is used.\n\n Returns:\n optimizer_results: A namedtuple containing the following items:\n converged: boolean tensor of shape `[...]` indicating for each batch\n member whether the minimum was found within tolerance.\n failed: boolean tensor of shape `[...]` indicating for each batch\n member whether a line search step failed to find a suitable step size\n satisfying Wolfe conditions. In the absence of any constraints on the\n number of objective evaluations permitted, this value will\n be the complement of `converged`. However, if there is\n a constraint and the search stopped due to available\n evaluations being exhausted, both `failed` and `converged`\n will be simultaneously False.\n num_objective_evaluations: The total number of objective\n evaluations performed.\n position: A tensor of shape `[..., n]` containing the last argument value\n found during the search from each starting point. If the search\n converged, then this value is the argmin of the objective function.\n objective_value: A tensor of shape `[...]` with the value of the\n objective function at the `position`. If the search converged, then\n this is the (local) minimum of the objective function.\n objective_gradient: A tensor of shape `[..., n]` containing the gradient\n of the objective function at the `position`. If the search converged\n the max-norm of this tensor should be below the tolerance.\n inverse_hessian_estimate: A tensor of shape `[..., n, n]` containing the\n inverse of the estimated Hessian.\n \"\"\"\n with tf.compat.v1.name_scope(\n name, 'minimize',\n [initial_position, tolerance, initial_inverse_hessian_estimate]):\n initial_position = tf.convert_to_tensor(\n value=initial_position, name='initial_position')\n dtype = initial_position.dtype.base_dtype\n tolerance = tf.convert_to_tensor(\n value=tolerance, dtype=dtype, name='grad_tolerance')\n f_relative_tolerance = tf.convert_to_tensor(\n value=f_relative_tolerance, dtype=dtype, name='f_relative_tolerance')\n x_tolerance = tf.convert_to_tensor(\n value=x_tolerance, dtype=dtype, name='x_tolerance')\n max_iterations = tf.convert_to_tensor(\n value=max_iterations, name='max_iterations')\n\n input_shape = distribution_util.prefer_static_shape(initial_position)\n batch_shape, domain_size = input_shape[:-1], input_shape[-1]\n\n if stopping_condition is None:\n stopping_condition = bfgs_utils.converged_all\n\n # Control inputs are an optional list of tensors to evaluate before\n # the start of the search procedure. These can be used to assert the\n # validity of inputs to the search procedure.\n control_inputs = None\n\n if initial_inverse_hessian_estimate is None:\n # Create a default initial inverse Hessian.\n initial_inv_hessian = tf.eye(domain_size,\n batch_shape=batch_shape,\n dtype=dtype,\n name='initial_inv_hessian')\n else:\n # If an initial inverse Hessian is supplied, compute some control inputs\n # to ensure that it is positive definite and symmetric.\n initial_inv_hessian = tf.convert_to_tensor(\n value=initial_inverse_hessian_estimate,\n dtype=dtype,\n name='initial_inv_hessian')\n control_inputs = _inv_hessian_control_inputs(initial_inv_hessian)\n hessian_shape = tf.concat([batch_shape, [domain_size, domain_size]], 0)\n initial_inv_hessian = tf.broadcast_to(initial_inv_hessian, hessian_shape)\n\n # The `state` here is a `BfgsOptimizerResults` tuple with values for the\n # current state of the algorithm computation.\n def _cond(state):\n \"\"\"Continue if iterations remain and stopping condition is not met.\"\"\"\n return ((state.num_iterations < max_iterations) &\n tf.logical_not(stopping_condition(state.converged, state.failed)))\n\n def _body(state):\n \"\"\"Main optimization loop.\"\"\"\n search_direction = _get_search_direction(state.inverse_hessian_estimate,\n state.objective_gradient)\n derivative_at_start_pt = tf.reduce_sum(\n input_tensor=state.objective_gradient * search_direction, axis=-1)\n\n # If the derivative at the start point is not negative, recompute the\n # search direction with the initial inverse Hessian.\n needs_reset = (~state.failed & ~state.converged &\n (derivative_at_start_pt >= 0))\n\n search_direction_reset = _get_search_direction(\n initial_inv_hessian, state.objective_gradient)\n\n actual_serch_direction = tf.where(\n needs_reset, search_direction_reset, search_direction)\n actual_inv_hessian = tf.where(\n needs_reset, initial_inv_hessian, state.inverse_hessian_estimate)\n\n # Replace the hessian estimate in the state, in case it had to be reset.\n current_state = bfgs_utils.update_fields(\n state, inverse_hessian_estimate=actual_inv_hessian)\n\n next_state = bfgs_utils.line_search_step(\n current_state,\n value_and_gradients_function, actual_serch_direction,\n tolerance, f_relative_tolerance, x_tolerance, stopping_condition)\n\n # Update the inverse Hessian if needed and continue.\n return [_update_inv_hessian(current_state, next_state)]\n\n kwargs = bfgs_utils.get_initial_state_args(\n value_and_gradients_function,\n initial_position,\n tolerance,\n control_inputs)\n kwargs['inverse_hessian_estimate'] = initial_inv_hessian\n initial_state = BfgsOptimizerResults(**kwargs)\n return tf.while_loop(\n cond=_cond,\n body=_body,\n loop_vars=[initial_state],\n parallel_iterations=parallel_iterations)[0]"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef _inv_hessian_control_inputs(inv_hessian):\n # The easiest way to validate if the inverse Hessian is positive definite is\n # to compute its Cholesky decomposition.\n is_positive_definite = tf.reduce_all(\n input_tensor=tf.math.is_finite(tf.linalg.cholesky(inv_hessian)),\n axis=[-1, -2])\n\n # Then check that the supplied inverse Hessian is symmetric.\n is_symmetric = tf.equal(bfgs_utils.norm(\n inv_hessian - _batch_transpose(inv_hessian), dims=2), 0)\n\n # Simply adding a control dependencies on these results is not enough to\n # trigger them, we need to add asserts on the results.\n return [tf.Assert(is_positive_definite,\n ['Initial inverse Hessian is not positive definite.',\n inv_hessian]),\n tf.Assert(is_symmetric,\n ['Initial inverse Hessian is not symmetric',\n inv_hessian])]", "response": "Computes control inputs to validate a provided inverse Hessian."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nupdate the inverse Hessian estimate of the BGFS state by computing the next inverse Hessian estimate.", "response": "def _update_inv_hessian(prev_state, next_state):\n \"\"\"Update the BGFS state by computing the next inverse hessian estimate.\"\"\"\n # Only update the inverse Hessian if not already failed or converged.\n should_update = ~next_state.converged & ~next_state.failed\n\n # Compute the normalization term (y^T . s), should not update if is singular.\n gradient_delta = next_state.objective_gradient - prev_state.objective_gradient\n position_delta = next_state.position - prev_state.position\n normalization_factor = tf.reduce_sum(\n input_tensor=gradient_delta * position_delta, axis=-1)\n should_update = should_update & ~tf.equal(normalization_factor, 0)\n\n def _do_update_inv_hessian():\n next_inv_hessian = _bfgs_inv_hessian_update(\n gradient_delta, position_delta, normalization_factor,\n prev_state.inverse_hessian_estimate)\n return bfgs_utils.update_fields(\n next_state,\n inverse_hessian_estimate=tf.where(should_update,\n next_inv_hessian,\n prev_state.inverse_hessian_estimate))\n\n return prefer_static.cond(\n tf.reduce_any(input_tensor=should_update),\n _do_update_inv_hessian,\n lambda: next_state)"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef _bfgs_inv_hessian_update(grad_delta, position_delta, normalization_factor,\n inv_hessian_estimate):\n \"\"\"Applies the BFGS update to the inverse Hessian estimate.\n\n The BFGS update rule is (note A^T denotes the transpose of a vector/matrix A).\n\n ```None\n rho = 1/(grad_delta^T * position_delta)\n U = (I - rho * position_delta * grad_delta^T)\n H_1 = U * H_0 * U^T + rho * position_delta * position_delta^T\n ```\n\n Here, `H_0` is the inverse Hessian estimate at the previous iteration and\n `H_1` is the next estimate. Note that `*` should be interpreted as the\n matrix multiplication (with the understanding that matrix multiplication for\n scalars is usual multiplication and for matrix with vector is the action of\n the matrix on the vector.).\n\n The implementation below utilizes an expanded version of the above formula\n to avoid the matrix multiplications that would be needed otherwise. By\n expansion it is easy to see that one only needs matrix-vector or\n vector-vector operations. The expanded version is:\n\n ```None\n f = 1 + rho * (grad_delta^T * H_0 * grad_delta)\n H_1 - H_0 = - rho * [position_delta * (H_0 * grad_delta)^T +\n (H_0 * grad_delta) * position_delta^T] +\n rho * f * [position_delta * position_delta^T]\n ```\n\n All the terms in square brackets are matrices and are constructed using\n vector outer products. All the other terms on the right hand side are scalars.\n Also worth noting that the first and second lines are both rank 1 updates\n applied to the current inverse Hessian estimate.\n\n Args:\n grad_delta: Real `Tensor` of shape `[..., n]`. The difference between the\n gradient at the new position and the old position.\n position_delta: Real `Tensor` of shape `[..., n]`. The change in position\n from the previous iteration to the current one.\n normalization_factor: Real `Tensor` of shape `[...]`. Should be equal to\n `grad_delta^T * position_delta`, i.e. `1/rho` as defined above.\n inv_hessian_estimate: Real `Tensor` of shape `[..., n, n]`. The previous\n estimate of the inverse Hessian. Should be positive definite and\n symmetric.\n\n Returns:\n A tuple containing the following fields\n is_valid: A Boolean `Tensor` of shape `[...]` indicating batch members\n where the update succeeded. The update can fail if the position change\n becomes orthogonal to the gradient change.\n next_inv_hessian_estimate: A `Tensor` of shape `[..., n, n]`. The next\n Hessian estimate updated using the BFGS update scheme. If the\n `inv_hessian_estimate` is symmetric and positive definite, the\n `next_inv_hessian_estimate` is guaranteed to satisfy the same\n conditions.\n \"\"\"\n # The quadratic form: y^T.H.y; where H is the inverse Hessian and y is the\n # gradient change.\n conditioned_grad_delta = _mul_right(inv_hessian_estimate, grad_delta)\n conditioned_grad_delta_norm = tf.reduce_sum(\n input_tensor=conditioned_grad_delta * grad_delta, axis=-1)\n\n # The first rank 1 update term requires the outer product: s.y^T.\n cross_term = _tensor_product(position_delta, conditioned_grad_delta)\n\n def _expand_scalar(s):\n # Expand dimensions of a batch of scalars to multiply or divide a matrix.\n return s[..., tf.newaxis, tf.newaxis]\n\n # Symmetrize\n cross_term += _tensor_product(conditioned_grad_delta, position_delta)\n position_term = _tensor_product(position_delta, position_delta)\n with tf.control_dependencies([position_term]):\n position_term *= _expand_scalar(\n 1 + conditioned_grad_delta_norm / normalization_factor)\n\n return (inv_hessian_estimate +\n (position_term - cross_term) / _expand_scalar(normalization_factor))", "response": "Applies the BFGS update to the inverse Hessian estimate."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\ncompute the product of a matrix with a vector on the right.", "response": "def _mul_right(mat, vec):\n \"\"\"Computes the product of a matrix with a vector on the right.\n\n Note this supports dynamic shapes and batched computation.\n\n Examples:\n\n M = tf.reshape(tf.range(6), shape=(3, 2))\n # => [[0, 1],\n # [2, 3],\n # [4, 5]]\n v = tf.constant([1, 2]) # Shape: (2,)\n _mul_right(M, v)\n # => [ 2, 8, 14] # Shape: (3,)\n\n M = tf.reshape(tf.range(30), shape=(2, 3, 5))\n # => [[[ 0, 1, 2, 3, 4],\n # [ 5, 6, 7, 8, 9],\n # [10, 11, 12, 13, 14]],\n #\n # [[15, 16, 17, 18, 19],\n # [20, 21, 22, 23, 24],\n # [25, 26, 27, 28, 29]]]\n v = tf.reshape(tf.range(10), shape=(2, 5))\n # => [[0, 1, 2, 3, 4],\n # [5, 6, 7, 8, 9]]\n _mul_right(M, v)\n # => [[ 30, 80, 130],\n # [605, 780, 955]] # Shape: (2, 3)\n\n Args:\n mat: A `tf.Tensor` of shape `[..., n, m]`.\n vec: A `tf.Tensor` of shape `[..., m]`.\n\n Returns:\n A tensor of shape `[..., n]` with matching batch dimensions.\n \"\"\"\n return tf.squeeze(tf.matmul(mat, tf.expand_dims(vec, axis=-1)), axis=-1)"} {"SOURCE": "codesearchnet", "instruction": "Can you implement a function in Python 3 that\ncomputes the outer product of two possibly batched vectors.", "response": "def _tensor_product(t1, t2):\n \"\"\"Computes the outer product of two possibly batched vectors.\n\n Args:\n t1: A `tf.Tensor` of shape `[..., n]`.\n t2: A `tf.Tensor` of shape `[..., m]`.\n\n Returns:\n A tensor of shape `[..., n, m]` with matching batch dimensions, let's call\n it `r`, whose components are:\n\n ```None\n r[..., i, j] = t1[..., i] * t2[..., j]\n ```\n \"\"\"\n return tf.matmul(tf.expand_dims(t1, axis=-1), tf.expand_dims(t2, axis=-2))"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _batch_transpose(mat):\n n = distribution_util.prefer_static_rank(mat)\n perm = tf.range(n)\n perm = tf.concat([perm[:-2], [perm[-1], perm[-2]]], axis=0)\n return tf.transpose(a=mat, perm=perm)", "response": "Transpose a possibly batched matrix."} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nreturns a Tensor with right - most ndims summed.", "response": "def sum_rightmost_ndims_preserving_shape(x, ndims):\n \"\"\"Return `Tensor` with right-most ndims summed.\n\n Args:\n x: the `Tensor` whose right-most `ndims` dimensions to sum\n ndims: number of right-most dimensions to sum.\n\n Returns:\n A `Tensor` resulting from calling `reduce_sum` on the `ndims` right-most\n dimensions. If the shape of `x` is statically known, the result will also\n have statically known shape. Otherwise, the resulting shape will only be\n known at runtime.\n \"\"\"\n x = tf.convert_to_tensor(value=x)\n if x.shape.ndims is not None:\n axes = tf.range(x.shape.ndims - ndims, x.shape.ndims)\n else:\n axes = tf.range(tf.rank(x) - ndims, tf.rank(x))\n return tf.reduce_sum(input_tensor=x, axis=axes)"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nreturn common dtype of items in arg_list or None.", "response": "def maybe_get_common_dtype(arg_list):\n \"\"\"Return common dtype of arg_list, or None.\n\n Args:\n arg_list: an iterable of items which are either `None` or have a `dtype`\n property.\n\n Returns:\n dtype: The common dtype of items in `arg_list`, or `None` if the list is\n empty or all items are `None`.\n \"\"\"\n # Note that `all` defaults to `True` if `arg_list` is empty.\n if all(a is None for a in arg_list):\n return None\n return dtype_util.common_dtype(arg_list, tf.float32)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\napply an unconstrained minimization of a differentiable function using the L - BFGS algorithm.", "response": "def minimize(value_and_gradients_function,\n initial_position,\n num_correction_pairs=10,\n tolerance=1e-8,\n x_tolerance=0,\n f_relative_tolerance=0,\n initial_inverse_hessian_estimate=None,\n max_iterations=50,\n parallel_iterations=1,\n stopping_condition=None,\n name=None):\n \"\"\"Applies the L-BFGS algorithm to minimize a differentiable function.\n\n Performs unconstrained minimization of a differentiable function using the\n L-BFGS scheme. See [Nocedal and Wright(2006)][1] for details of the algorithm.\n\n ### Usage:\n\n The following example demonstrates the L-BFGS optimizer attempting to find the\n minimum for a simple high-dimensional quadratic objective function.\n\n ```python\n # A high-dimensional quadratic bowl.\n ndims = 60\n minimum = np.ones([ndims], dtype='float64')\n scales = np.arange(ndims, dtype='float64') + 1.0\n\n # The objective function and the gradient.\n def quadratic(x):\n value = tf.reduce_sum(scales * (x - minimum) ** 2)\n return value, tf.gradients(value, x)[0]\n\n start = np.arange(ndims, 0, -1, dtype='float64')\n optim_results = tfp.optimizer.lbfgs_minimize(\n quadratic, initial_position=start, num_correction_pairs=10,\n tolerance=1e-8)\n\n with tf.Session() as session:\n results = session.run(optim_results)\n # Check that the search converged\n assert(results.converged)\n # Check that the argmin is close to the actual value.\n np.testing.assert_allclose(results.position, minimum)\n ```\n\n ### References:\n\n [1] Jorge Nocedal, Stephen Wright. Numerical Optimization. Springer Series\n in Operations Research. pp 176-180. 2006\n\n http://pages.mtu.edu/~struther/Courses/OLD/Sp2013/5630/Jorge_Nocedal_Numerical_optimization_267490.pdf\n\n Args:\n value_and_gradients_function: A Python callable that accepts a point as a\n real `Tensor` and returns a tuple of `Tensor`s of real dtype containing\n the value of the function and its gradient at that point. The function\n to be minimized. The input is of shape `[..., n]`, where `n` is the size\n of the domain of input points, and all others are batching dimensions.\n The first component of the return value is a real `Tensor` of matching\n shape `[...]`. The second component (the gradient) is also of shape\n `[..., n]` like the input value to the function.\n initial_position: Real `Tensor` of shape `[..., n]`. The starting point, or\n points when using batching dimensions, of the search procedure. At these\n points the function value and the gradient norm should be finite.\n num_correction_pairs: Positive integer. Specifies the maximum number of\n (position_delta, gradient_delta) correction pairs to keep as implicit\n approximation of the Hessian matrix.\n tolerance: Scalar `Tensor` of real dtype. Specifies the gradient tolerance\n for the procedure. If the supremum norm of the gradient vector is below\n this number, the algorithm is stopped.\n x_tolerance: Scalar `Tensor` of real dtype. If the absolute change in the\n position between one iteration and the next is smaller than this number,\n the algorithm is stopped.\n f_relative_tolerance: Scalar `Tensor` of real dtype. If the relative change\n in the objective value between one iteration and the next is smaller\n than this value, the algorithm is stopped.\n initial_inverse_hessian_estimate: None. Option currently not supported.\n max_iterations: Scalar positive int32 `Tensor`. The maximum number of\n iterations for L-BFGS updates.\n parallel_iterations: Positive integer. The number of iterations allowed to\n run in parallel.\n stopping_condition: (Optional) A Python function that takes as input two\n Boolean tensors of shape `[...]`, and returns a Boolean scalar tensor.\n The input tensors are `converged` and `failed`, indicating the current\n status of each respective batch member; the return value states whether\n the algorithm should stop. The default is tfp.optimizer.converged_all\n which only stops when all batch members have either converged or failed.\n An alternative is tfp.optimizer.converged_any which stops as soon as one\n batch member has converged, or when all have failed.\n name: (Optional) Python str. The name prefixed to the ops created by this\n function. If not supplied, the default name 'minimize' is used.\n\n Returns:\n optimizer_results: A namedtuple containing the following items:\n converged: Scalar boolean tensor indicating whether the minimum was\n found within tolerance.\n failed: Scalar boolean tensor indicating whether a line search\n step failed to find a suitable step size satisfying Wolfe\n conditions. In the absence of any constraints on the\n number of objective evaluations permitted, this value will\n be the complement of `converged`. However, if there is\n a constraint and the search stopped due to available\n evaluations being exhausted, both `failed` and `converged`\n will be simultaneously False.\n num_objective_evaluations: The total number of objective\n evaluations performed.\n position: A tensor containing the last argument value found\n during the search. If the search converged, then\n this value is the argmin of the objective function.\n objective_value: A tensor containing the value of the objective\n function at the `position`. If the search converged, then this is\n the (local) minimum of the objective function.\n objective_gradient: A tensor containing the gradient of the objective\n function at the `position`. If the search converged the\n max-norm of this tensor should be below the tolerance.\n position_deltas: A tensor encoding information about the latest\n changes in `position` during the algorithm execution.\n gradient_deltas: A tensor encoding information about the latest\n changes in `objective_gradient` during the algorithm execution.\n \"\"\"\n if initial_inverse_hessian_estimate is not None:\n raise NotImplementedError(\n 'Support of initial_inverse_hessian_estimate arg not yet implemented')\n\n if stopping_condition is None:\n stopping_condition = bfgs_utils.converged_all\n\n with tf.compat.v1.name_scope(name, 'minimize', [initial_position, tolerance]):\n initial_position = tf.convert_to_tensor(\n value=initial_position, name='initial_position')\n dtype = initial_position.dtype.base_dtype\n tolerance = tf.convert_to_tensor(\n value=tolerance, dtype=dtype, name='grad_tolerance')\n f_relative_tolerance = tf.convert_to_tensor(\n value=f_relative_tolerance, dtype=dtype, name='f_relative_tolerance')\n x_tolerance = tf.convert_to_tensor(\n value=x_tolerance, dtype=dtype, name='x_tolerance')\n max_iterations = tf.convert_to_tensor(\n value=max_iterations, name='max_iterations')\n\n # The `state` here is a `LBfgsOptimizerResults` tuple with values for the\n # current state of the algorithm computation.\n def _cond(state):\n \"\"\"Continue if iterations remain and stopping condition is not met.\"\"\"\n return ((state.num_iterations < max_iterations) &\n tf.logical_not(stopping_condition(state.converged, state.failed)))\n\n def _body(current_state):\n \"\"\"Main optimization loop.\"\"\"\n search_direction = _get_search_direction(current_state)\n\n # TODO(b/120134934): Check if the derivative at the start point is not\n # negative, if so then reset position/gradient deltas and recompute\n # search direction.\n\n next_state = bfgs_utils.line_search_step(\n current_state,\n value_and_gradients_function, search_direction,\n tolerance, f_relative_tolerance, x_tolerance, stopping_condition)\n\n # If not failed or converged, update the Hessian estimate.\n should_update = ~(next_state.converged | next_state.failed)\n state_after_inv_hessian_update = bfgs_utils.update_fields(\n next_state,\n position_deltas=_queue_push(\n current_state.position_deltas, should_update,\n next_state.position - current_state.position),\n gradient_deltas=_queue_push(\n current_state.gradient_deltas, should_update,\n next_state.objective_gradient - current_state.objective_gradient))\n return [state_after_inv_hessian_update]\n\n initial_state = _get_initial_state(value_and_gradients_function,\n initial_position,\n num_correction_pairs,\n tolerance)\n return tf.while_loop(\n cond=_cond,\n body=_body,\n loop_vars=[initial_state],\n parallel_iterations=parallel_iterations)[0]"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\ncreating LBfgsOptimizerResults with initial state of search procedure.", "response": "def _get_initial_state(value_and_gradients_function,\n initial_position,\n num_correction_pairs,\n tolerance):\n \"\"\"Create LBfgsOptimizerResults with initial state of search procedure.\"\"\"\n init_args = bfgs_utils.get_initial_state_args(\n value_and_gradients_function,\n initial_position,\n tolerance)\n empty_queue = _make_empty_queue_for(num_correction_pairs, initial_position)\n init_args.update(position_deltas=empty_queue, gradient_deltas=empty_queue)\n return LBfgsOptimizerResults(**init_args)"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef _get_search_direction(state):\n # The number of correction pairs that have been collected so far.\n num_elements = tf.minimum(\n state.num_iterations,\n distribution_util.prefer_static_shape(state.position_deltas)[0])\n\n def _two_loop_algorithm():\n \"\"\"L-BFGS two-loop algorithm.\"\"\"\n # Correction pairs are always appended to the end, so only the latest\n # `num_elements` vectors have valid position/gradient deltas.\n position_deltas = state.position_deltas[-num_elements:]\n gradient_deltas = state.gradient_deltas[-num_elements:]\n\n # Pre-compute all `inv_rho[i]`s.\n inv_rhos = tf.reduce_sum(\n input_tensor=gradient_deltas * position_deltas, axis=-1)\n\n def first_loop(acc, args):\n _, q_direction = acc\n position_delta, gradient_delta, inv_rho = args\n alpha = tf.reduce_sum(\n input_tensor=position_delta * q_direction, axis=-1) / inv_rho\n direction_delta = tf.expand_dims(alpha, axis=-1) * gradient_delta\n return (alpha, q_direction - direction_delta)\n\n # Run first loop body computing and collecting `alpha[i]`s, while also\n # computing the updated `q_direction` at each step.\n zero = tf.zeros_like(inv_rhos[0])\n alphas, q_directions = tf.scan(\n first_loop, [position_deltas, gradient_deltas, inv_rhos],\n initializer=(zero, state.objective_gradient), reverse=True)\n\n # We use `H^0_k = gamma_k * I` as an estimate for the initial inverse\n # hessian for the k-th iteration; then `r_direction = H^0_k * q_direction`.\n gamma_k = inv_rhos[-1] / tf.reduce_sum(\n input_tensor=gradient_deltas[-1] * gradient_deltas[-1], axis=-1)\n r_direction = tf.expand_dims(gamma_k, axis=-1) * q_directions[0]\n\n def second_loop(r_direction, args):\n alpha, position_delta, gradient_delta, inv_rho = args\n beta = tf.reduce_sum(\n input_tensor=gradient_delta * r_direction, axis=-1) / inv_rho\n direction_delta = tf.expand_dims(alpha - beta, axis=-1) * position_delta\n return r_direction + direction_delta\n\n # Finally, run second loop body computing the updated `r_direction` at each\n # step.\n r_directions = tf.scan(\n second_loop, [alphas, position_deltas, gradient_deltas, inv_rhos],\n initializer=r_direction)\n return -r_directions[-1]\n\n return prefer_static.cond(tf.equal(num_elements, 0),\n (lambda: -state.objective_gradient),\n _two_loop_algorithm)", "response": "Computes the search direction to follow at the current state."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _make_empty_queue_for(k, element):\n queue_shape = tf.concat(\n [[k], distribution_util.prefer_static_shape(element)], axis=0)\n return tf.zeros(queue_shape, dtype=element.dtype.base_dtype)", "response": "Creates a zero - filed queue suitable to hold k element - shaped tensors."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef _psd_mask(x):\n # Allegedly\n # https://scicomp.stackexchange.com/questions/12979/testing-if-a-matrix-is-positive-semi-definite\n # it is more efficient to test for positive semi-definiteness by\n # trying to compute the Cholesky decomposition -- the matrix is PSD\n # if you succeed and not PSD if you fail. However, TensorFlow's\n # Cholesky raises an exception if _any_ of the input matrices are\n # not PSD, from which I don't know how to extract _which ones_, so I\n # proceed by explicitly computing all the eigenvalues and checking\n # whether they are all positive or not.\n #\n # Also, as was discussed in the answer, it is somewhat dangerous to\n # treat SPD-ness as binary in floating-point arithmetic. Cholesky\n # factorization can complete and 'look' like everything is fine\n # (e.g., O(1) entries and a diagonal of all ones) but the matrix can\n # have an exponential condition number.\n eigenvalues, _ = tf.linalg.eigh(x)\n return tf.cast(\n tf.reduce_min(input_tensor=eigenvalues, axis=-1) >= 0, dtype=x.dtype)", "response": "Computes whether each square matrix in the input is positive semi - definite."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef _det_large_enough_mask(x, det_bounds):\n # For the curious: I wonder whether it is possible and desirable to\n # use a Cholesky decomposition-based algorithm for this, since the\n # only matrices whose determinant this code cares about will be PSD.\n # Didn't figure out how to code that in TensorFlow.\n #\n # Expert opinion is that it would be about twice as fast since\n # Cholesky is roughly half the cost of Gaussian Elimination with\n # Partial Pivoting. But this is less of an impact than the switch in\n # _psd_mask.\n return tf.cast(tf.linalg.det(x) > det_bounds, dtype=x.dtype)", "response": "Returns True if the input matches the given determinant limit."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nreturning a uniformly random tensor of correlation - like matrices.", "response": "def _uniform_correlation_like_matrix(num_rows, batch_shape, dtype, seed):\n \"\"\"Returns a uniformly random `Tensor` of \"correlation-like\" matrices.\n\n A \"correlation-like\" matrix is a symmetric square matrix with all entries\n between -1 and 1 (inclusive) and 1s on the main diagonal. Of these,\n the ones that are positive semi-definite are exactly the correlation\n matrices.\n\n Args:\n num_rows: Python `int` dimension of the correlation-like matrices.\n batch_shape: `Tensor` or Python `tuple` of `int` shape of the\n batch to return.\n dtype: `dtype` of the `Tensor` to return.\n seed: Random seed.\n\n Returns:\n matrices: A `Tensor` of shape `batch_shape + [num_rows, num_rows]`\n and dtype `dtype`. Each entry is in [-1, 1], and each matrix\n along the bottom two dimensions is symmetric and has 1s on the\n main diagonal.\n \"\"\"\n num_entries = num_rows * (num_rows + 1) / 2\n ones = tf.ones(shape=[num_entries], dtype=dtype)\n # It seems wasteful to generate random values for the diagonal since\n # I am going to throw them away, but `fill_triangular` fills the\n # diagonal, so I probably need them.\n # It's not impossible that it would be more efficient to just fill\n # the whole matrix with random values instead of messing with\n # `fill_triangular`. Then would need to filter almost half out with\n # `matrix_band_part`.\n unifs = uniform.Uniform(-ones, ones).sample(batch_shape, seed=seed)\n tril = util.fill_triangular(unifs)\n symmetric = tril + tf.linalg.matrix_transpose(tril)\n diagonal_ones = tf.ones(\n shape=util.pad(batch_shape, axis=0, back=True, value=num_rows),\n dtype=dtype)\n return tf.linalg.set_diag(symmetric, diagonal_ones)"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef correlation_matrix_volume_rejection_samples(\n det_bounds, dim, sample_shape, dtype, seed):\n \"\"\"Returns rejection samples from trying to get good correlation matrices.\n\n The proposal being rejected from is the uniform distribution on\n \"correlation-like\" matrices. We say a matrix is \"correlation-like\"\n if it is a symmetric square matrix with all entries between -1 and 1\n (inclusive) and 1s on the main diagonal. Of these, the ones that\n are positive semi-definite are exactly the correlation matrices.\n\n The rejection algorithm, then, is to sample a `Tensor` of\n `sample_shape` correlation-like matrices of dimensions `dim` by\n `dim`, and check each one for (i) being a correlation matrix (i.e.,\n PSD), and (ii) having determinant at least the corresponding entry\n of `det_bounds`.\n\n Args:\n det_bounds: A `Tensor` of lower bounds on the determinants of\n acceptable matrices. The shape must broadcast with `sample_shape`.\n dim: A Python `int` dimension of correlation matrices to sample.\n sample_shape: Python `tuple` of `int` shape of the samples to\n compute, excluding the two matrix dimensions.\n dtype: The `dtype` in which to do the computation.\n seed: Random seed.\n\n Returns:\n weights: A `Tensor` of shape `sample_shape`. Each entry is 0 if the\n corresponding matrix was not a correlation matrix, or had too\n small of a determinant. Otherwise, the entry is the\n multiplicative inverse of the density of proposing that matrix\n uniformly, i.e., the volume of the set of `dim` by `dim`\n correlation-like matrices.\n volume: The volume of the set of `dim` by `dim` correlation-like\n matrices.\n \"\"\"\n with tf.compat.v1.name_scope(\"rejection_sampler\"):\n rej_proposals = _uniform_correlation_like_matrix(\n dim, sample_shape, dtype, seed=seed)\n rej_proposal_volume = 2. ** (dim * (dim - 1) / 2.)\n # The density of proposing any given point is 1 / rej_proposal_volume;\n # The weight of that point should be scaled by\n # 1 / density = rej_proposal_volume.\n rej_weights = rej_proposal_volume * _psd_mask(\n rej_proposals) * _det_large_enough_mask(rej_proposals, det_bounds)\n return rej_weights, rej_proposal_volume", "response": "Returns the samples from trying to get good correlation matrices."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef _clopper_pearson_confidence_interval(samples, error_rate):\n # TODO(b/78025336) Migrate this confidence interval function\n # to statistical_testing.py. In order to do that\n # - Get the binomial CDF from the Binomial distribution\n # - Implement scalar root finding in TF. Batch bisection search\n # shouldn't be too hard, and is definitely good enough for this\n # problem. Batching the Brent algorithm (from scipy) that is used\n # here may be more involved, but may also not be necessary---it's\n # only used here because scipy made it convenient. In particular,\n # robustness is more important than speed here, which may make\n # bisection search actively better.\n # - The rest is just a matter of rewriting in the appropriate style.\n if optimize is None or stats is None:\n raise ValueError(\n \"Scipy is required for computing Clopper-Pearson confidence intervals\")\n if len(samples.shape) != 1:\n raise ValueError(\"Batch semantics not implemented\")\n n = len(samples)\n low = np.amin(samples)\n high = np.amax(samples)\n successes = np.count_nonzero(samples - low)\n failures = np.count_nonzero(samples - high)\n if successes + failures != n:\n uniques = np.unique(samples)\n msg = (\"Purportedly Bernoulli distribution had distinct samples\"\n \" {}, {}, and {}\".format(uniques[0], uniques[1], uniques[2]))\n raise ValueError(msg)\n def p_small_enough(p):\n prob = stats.binom.logcdf(successes, n, p)\n return prob - np.log(error_rate / 2.)\n def p_big_enough(p):\n prob = stats.binom.logsf(successes, n, p)\n return prob - np.log(error_rate / 2.)\n high_p = optimize.brentq(\n p_small_enough, float(successes) / n, 1., rtol=1e-9)\n low_p = optimize.brentq(\n p_big_enough, 0., float(successes) / n, rtol=1e-9)\n low_interval = low + (high - low) * low_p\n high_interval = low + (high - low) * high_p\n return (low_interval, high_interval)", "response": "This function calculates the confidence interval for the given samples."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\ncomputes the confidence intervals for the true volumes of the given determinant bounds.", "response": "def compute_true_volumes(\n det_bounds, dim, num_samples, error_rate=1e-6, seed=42):\n \"\"\"Returns confidence intervals for the desired correlation matrix volumes.\n\n The confidence intervals are computed by the [Clopper-Pearson method]\n (https://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval).\n\n Args:\n det_bounds: A rank-1 numpy array of lower bounds on the\n determinants of acceptable matrices. Entries must be unique.\n dim: A Python `int` dimension of correlation matrices to sample.\n num_samples: The number of samples to draw.\n error_rate: The statistical significance of the returned\n confidence intervals. The significance is broadcast: Each\n returned interval separately may be incorrect with probability\n (under the sample of correlation-like matrices drawn internally)\n at most `error_rate`.\n seed: Random seed.\n\n Returns:\n bounds: A Python `dict` mapping each determinant bound to the low, high\n tuple giving the confidence interval.\n \"\"\"\n bounds = {}\n with tf.compat.v1.Session() as sess:\n rej_weights, _ = correlation_matrix_volume_rejection_samples(\n det_bounds, dim, [num_samples, len(det_bounds)], np.float32, seed=seed)\n rej_weights = sess.run(rej_weights)\n for rw, det in zip(np.rollaxis(rej_weights, 1), det_bounds):\n template = (\"Estimating volume of {}x{} correlation \"\n \"matrices with determinant >= {}.\")\n print(template.format(dim, dim, det))\n sys.stdout.flush()\n bounds[det] = _clopper_pearson_confidence_interval(\n rw, error_rate=error_rate)\n return bounds"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\ncompute the cumulative density function (CDF) of von Mises distribution. Denote the density of vonMises(loc=0, concentration=concentration) by p(t). Note that p(t) is periodic, p(t) = p(t + 2 pi). The CDF at the point x is defined as int_{-pi}^x p(t) dt. Thus, when x in [-pi, pi], the CDF is in [0, 1]; when x is in [pi, 3pi], the CDF is in [1, 2], etc. The CDF is not available in closed form. Instead, we use the method [1] which uses either a series expansion or a Normal approximation, depending on the value of concentration. We also compute the derivative of the CDF w.r.t. both x and concentration. The derivative w.r.t. x is p(x), while the derivative w.r.t. concentration is computed using automatic differentiation. We use forward mode for the series case (which allows to save memory) and backward mode for the Normal approximation. Arguments: x: The point at which to evaluate the CDF. concentration: The concentration parameter of the von Mises distribution. Returns: The value of the CDF computed elementwise. References: [1] G. Hill \"Algorithm 518: Incomplete Bessel Function I_0. The Von Mises Distribution.\" ACM Transactions on Mathematical Software, 1977", "response": "def von_mises_cdf(x, concentration):\n \"\"\"Computes the cumulative density function (CDF) of von Mises distribution.\n\n Denote the density of vonMises(loc=0, concentration=concentration) by p(t).\n Note that p(t) is periodic, p(t) = p(t + 2 pi).\n The CDF at the point x is defined as int_{-pi}^x p(t) dt.\n Thus, when x in [-pi, pi], the CDF is in [0, 1]; when x is in [pi, 3pi], the\n CDF is in [1, 2], etc.\n\n The CDF is not available in closed form. Instead, we use the method [1]\n which uses either a series expansion or a Normal approximation, depending on\n the value of concentration.\n\n We also compute the derivative of the CDF w.r.t. both x and concentration.\n The derivative w.r.t. x is p(x), while the derivative w.r.t. concentration is\n computed\n using automatic differentiation. We use forward mode for the series case\n (which allows to save memory) and backward mode for the Normal approximation.\n\n Arguments:\n x: The point at which to evaluate the CDF.\n concentration: The concentration parameter of the von Mises distribution.\n\n Returns:\n The value of the CDF computed elementwise.\n\n References:\n [1] G. Hill \"Algorithm 518: Incomplete Bessel Function I_0. The Von Mises\n Distribution.\" ACM Transactions on Mathematical Software, 1977\n \"\"\"\n x = tf.convert_to_tensor(value=x)\n concentration = tf.convert_to_tensor(value=concentration)\n dtype = x.dtype\n\n # Map x to [-pi, pi].\n num_periods = tf.round(x / (2. * np.pi))\n x -= (2. * np.pi) * num_periods\n\n # We take the hyperparameters from Table I of [1], the row for D=8\n # decimal digits of accuracy. ck is the cut-off for concentration:\n # if concentration < ck, the series expansion is used;\n # otherwise, the Normal approximation is used.\n ck = 10.5\n # The number of terms in the series expansion. [1] chooses it as a function\n # of concentration, n(concentration). This is hard to implement in TF.\n # Instead, we upper bound it over concentrations:\n # num_terms = ceil ( max_{concentration <= ck} n(concentration) ).\n # The maximum is achieved for concentration = ck.\n num_terms = 20\n\n cdf_series, dcdf_dconcentration_series = _von_mises_cdf_series(\n x, concentration, num_terms, dtype)\n cdf_normal, dcdf_dconcentration_normal = _von_mises_cdf_normal(\n x, concentration, dtype)\n\n use_series = concentration < ck\n cdf = tf.where(use_series, cdf_series, cdf_normal)\n cdf += num_periods\n dcdf_dconcentration = tf.where(use_series, dcdf_dconcentration_series,\n dcdf_dconcentration_normal)\n\n def grad(dy):\n prob = tf.exp(concentration * (tf.cos(x) - 1.)) / (\n (2. * np.pi) * tf.math.bessel_i0e(concentration))\n return dy * prob, dy * dcdf_dconcentration\n\n return cdf, grad"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\ncompute the von Mises CDF and its derivative via series expansion.", "response": "def _von_mises_cdf_series(x, concentration, num_terms, dtype):\n \"\"\"Computes the von Mises CDF and its derivative via series expansion.\"\"\"\n # Keep the number of terms as a float. It should be a small integer, so\n # exactly representable as a float.\n num_terms = tf.cast(num_terms, dtype=dtype)\n\n def loop_body(n, rn, drn_dconcentration, vn, dvn_dconcentration):\n \"\"\"One iteration of the series loop.\"\"\"\n\n denominator = 2. * n / concentration + rn\n ddenominator_dk = -2. * n / concentration ** 2 + drn_dconcentration\n rn = 1. / denominator\n drn_dconcentration = -ddenominator_dk / denominator ** 2\n\n multiplier = tf.sin(n * x) / n + vn\n vn = rn * multiplier\n dvn_dconcentration = (drn_dconcentration * multiplier +\n rn * dvn_dconcentration)\n n -= 1.\n\n return n, rn, drn_dconcentration, vn, dvn_dconcentration\n\n (_, _, _, vn, dvn_dconcentration) = tf.while_loop(\n cond=lambda n, *_: n > 0.,\n body=loop_body,\n loop_vars=(\n num_terms, # n\n tf.zeros_like(x, name=\"rn\"),\n tf.zeros_like(x, name=\"drn_dconcentration\"),\n tf.zeros_like(x, name=\"vn\"),\n tf.zeros_like(x, name=\"dvn_dconcentration\"),\n ),\n )\n\n cdf = .5 + x / (2. * np.pi) + vn / np.pi\n dcdf_dconcentration = dvn_dconcentration / np.pi\n\n # Clip the result to [0, 1].\n cdf_clipped = tf.clip_by_value(cdf, 0., 1.)\n # The clipped values do not depend on concentration anymore, so set their\n # derivative to zero.\n dcdf_dconcentration *= tf.cast((cdf >= 0.) & (cdf <= 1.), dtype)\n\n return cdf_clipped, dcdf_dconcentration"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\ncomputes the von Mises CDF and its derivative via Normal approximation.", "response": "def _von_mises_cdf_normal(x, concentration, dtype):\n \"\"\"Computes the von Mises CDF and its derivative via Normal approximation.\"\"\"\n\n def cdf_func(concentration):\n \"\"\"A helper function that is passed to value_and_gradient.\"\"\"\n # z is an \"almost Normally distributed\" random variable.\n z = ((np.sqrt(2. / np.pi) / tf.math.bessel_i0e(concentration)) *\n tf.sin(.5 * x))\n\n # This is the correction described in [1] which reduces the error\n # of the Normal approximation.\n z2 = z ** 2\n z3 = z2 * z\n z4 = z2 ** 2\n c = 24. * concentration\n c1 = 56.\n\n xi = z - z3 / ((c - 2. * z2 - 16.) / 3. -\n (z4 + (7. / 4.) * z2 + 167. / 2.) / (c - c1 - z2 + 3.)) ** 2\n\n distrib = normal.Normal(tf.cast(0., dtype), tf.cast(1., dtype))\n\n return distrib.cdf(xi)\n\n return value_and_gradient(cdf_func, concentration)"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef random_von_mises(shape, concentration, dtype=tf.float32, seed=None):\n seed = SeedStream(seed, salt=\"von_mises\")\n concentration = tf.convert_to_tensor(\n value=concentration, dtype=dtype, name=\"concentration\")\n\n @tf.custom_gradient\n def rejection_sample_with_gradient(concentration):\n \"\"\"Performs rejection sampling for standardized von Mises.\n\n A nested function is required because @tf.custom_gradient does not handle\n non-tensor inputs such as dtype. Instead, they are captured by the outer\n scope.\n\n Arguments:\n concentration: The concentration parameter of the distribution.\n\n Returns:\n Differentiable samples of standardized von Mises.\n \"\"\"\n r = 1. + tf.sqrt(1. + 4. * concentration ** 2)\n rho = (r - tf.sqrt(2. * r)) / (2. * concentration)\n\n s_exact = (1. + rho ** 2) / (2. * rho)\n\n # For low concentration, s becomes numerically unstable.\n # To fix that, we use an approximation. Here is the derivation.\n # First-order Taylor expansion at conc = 0 gives\n # sqrt(1 + 4 concentration^2) ~= 1 + (2 concentration)^2 / 2.\n # Therefore, r ~= 2 + 2 concentration. By plugging this into rho, we have\n # rho ~= conc + 1 / conc - sqrt(1 + 1 / concentration^2).\n # Let's expand the last term at concentration=0 up to the linear term:\n # sqrt(1 + 1 / concentration^2) ~= 1 / concentration + concentration / 2\n # Thus, rho ~= concentration / 2. Finally,\n # s = 1 / (2 rho) + rho / 2 ~= 1 / concentration + concentration / 4.\n # Since concentration is small, we drop the second term and simply use\n # s ~= 1 / concentration.\n s_approximate = 1. / concentration\n\n # To compute the cutoff, we compute s_exact using mpmath with 30 decimal\n # digits precision and compare that to the s_exact and s_approximate\n # computed with dtype. Then, the cutoff is the largest concentration for\n # which abs(s_exact - s_exact_mpmath) > abs(s_approximate - s_exact_mpmath).\n s_concentration_cutoff_dict = {\n tf.float16: 1.8e-1,\n tf.float32: 2e-2,\n tf.float64: 1.2e-4,\n }\n s_concentration_cutoff = s_concentration_cutoff_dict[dtype]\n\n s = tf.where(concentration > s_concentration_cutoff, s_exact, s_approximate)\n\n def loop_body(done, u, w):\n \"\"\"Resample the non-accepted points.\"\"\"\n # We resample u each time completely. Only its sign is used outside the\n # loop, which is random.\n u = tf.random.uniform(\n shape, minval=-1., maxval=1., dtype=dtype, seed=seed())\n z = tf.cos(np.pi * u)\n # Update the non-accepted points.\n w = tf.where(done, w, (1. + s * z) / (s + z))\n y = concentration * (s - w)\n\n v = tf.random.uniform(\n shape, minval=0., maxval=1., dtype=dtype, seed=seed())\n accept = (y * (2. - y) >= v) | (tf.math.log(y / v) + 1. >= y)\n\n return done | accept, u, w\n\n _, u, w = tf.while_loop(\n cond=lambda done, *_: ~tf.reduce_all(input_tensor=done),\n body=loop_body,\n loop_vars=(\n tf.zeros(shape, dtype=tf.bool, name=\"done\"),\n tf.zeros(shape, dtype=dtype, name=\"u\"),\n tf.zeros(shape, dtype=dtype, name=\"w\"),\n ),\n # The expected number of iterations depends on concentration.\n # It monotonically increases from one iteration for concentration = 0 to\n # sqrt(2 pi / e) ~= 1.52 iterations for concentration = +inf [1].\n # We use a limit of 100 iterations to avoid infinite loops\n # for very large / nan concentration.\n maximum_iterations=100,\n parallel_iterations=1 if seed.original_seed is None else 10,\n )\n\n x = tf.sign(u) * tf.math.acos(w)\n\n def grad(dy):\n \"\"\"The gradient of the von Mises samples w.r.t. concentration.\"\"\"\n broadcast_concentration = concentration + tf.zeros_like(x)\n _, dcdf_dconcentration = value_and_gradient(\n lambda conc: von_mises_cdf(x, conc), broadcast_concentration)\n inv_prob = tf.exp(-broadcast_concentration * (tf.cos(x) - 1.)) * (\n (2. * np.pi) * tf.math.bessel_i0e(broadcast_concentration))\n # Compute the implicit reparameterization gradient [2],\n # dz/dconc = -(dF(z; conc) / dconc) / p(z; conc)\n ret = dy * (-inv_prob * dcdf_dconcentration)\n # Sum over the sample dimensions. Assume that they are always the first\n # ones.\n num_sample_dimensions = (tf.rank(broadcast_concentration) -\n tf.rank(concentration))\n return tf.reduce_sum(\n input_tensor=ret, axis=tf.range(num_sample_dimensions))\n\n return x, grad\n\n return rejection_sample_with_gradient(concentration)", "response": "Returns a random tensor from the standardized von Mises distribution."} {"SOURCE": "codesearchnet", "instruction": "How would you explain what the following Python 3 function does\ndef one_step(\n objective_function,\n population,\n population_values=None,\n differential_weight=0.5,\n crossover_prob=0.9,\n seed=None,\n name=None):\n \"\"\"Performs one step of the differential evolution algorithm.\n\n Args:\n objective_function: A Python callable that accepts a batch of possible\n solutions and returns the values of the objective function at those\n arguments as a rank 1 real `Tensor`. This specifies the function to be\n minimized. The input to this callable may be either a single `Tensor`\n or a Python `list` of `Tensor`s. The signature must match the format of\n the argument `population`. (i.e. objective_function(*population) must\n return the value of the function to be minimized).\n population: `Tensor` or Python `list` of `Tensor`s representing the\n current population vectors. Each `Tensor` must be of the same real dtype.\n The first dimension indexes individual population members while the\n rest of the dimensions are consumed by the value function. For example,\n if the population is a single `Tensor` of shape [n, m1, m2], then `n` is\n the population size and the output of `objective_function` applied to the\n population is a `Tensor` of shape [n]. If the population is a python\n list of `Tensor`s then each `Tensor` in the list should have the first\n axis of a common size, say `n` and `objective_function(*population)`\n should return a `Tensor of shape [n]. The population must have at least\n 4 members for the algorithm to work correctly.\n population_values: A `Tensor` of rank 1 and real dtype. The result of\n applying `objective_function` to the `population`. If not supplied it is\n computed using the `objective_function`.\n Default value: None.\n differential_weight: Real scalar `Tensor`. Must be positive and less than\n 2.0. The parameter controlling the strength of mutation.\n Default value: 0.5\n crossover_prob: Real scalar `Tensor`. Must be between 0 and 1. The\n probability of recombination per site.\n Default value: 0.9\n seed: `int` or None. The random seed for this `Op`. If `None`, no seed is\n applied.\n Default value: None.\n name: (Optional) Python str. The name prefixed to the ops created by this\n function. If not supplied, the default name 'one_step' is\n used.\n Default value: None\n\n Returns:\n A sequence containing the following elements (in order):\n next_population: A `Tensor` or Python `list` of `Tensor`s of the same\n structure as the input population. The population at the next generation.\n next_population_values: A `Tensor` of same shape and dtype as input\n `population_values`. The function values for the `next_population`.\n \"\"\"\n with tf.compat.v1.name_scope(\n name, 'one_step',\n [population, population_values, differential_weight, crossover_prob]):\n population, _ = _ensure_list(population)\n if population_values is None:\n population_values = objective_function(*population)\n population_size = tf.shape(input=population[0])[0]\n seed_stream = distributions.SeedStream(seed, salt='one_step')\n mixing_indices = _get_mixing_indices(population_size, seed=seed_stream())\n # Construct the mutated solution vectors. There is one for each member of\n # the population.\n mutants = _get_mutants(population,\n population_size,\n mixing_indices,\n differential_weight)\n # Perform recombination between the parents and the mutants.\n candidates = _binary_crossover(population,\n population_size,\n mutants,\n crossover_prob,\n seed=seed_stream())\n candidate_values = objective_function(*candidates)\n if population_values is None:\n population_values = objective_function(*population)\n\n infinity = tf.zeros_like(population_values) + np.inf\n\n population_values = tf.where(\n tf.math.is_nan(population_values), x=infinity, y=population_values)\n\n to_replace = candidate_values < population_values\n next_population = [\n tf.where(to_replace, x=candidates_part, y=population_part)\n for candidates_part, population_part in zip(candidates, population)\n ]\n next_values = tf.where(to_replace, x=candidate_values, y=population_values)\n\n return next_population, next_values", "response": "A function that performs one step of differential evolution algorithm."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef minimize(objective_function,\n initial_population=None,\n initial_position=None,\n population_size=50,\n population_stddev=1.,\n max_iterations=100,\n func_tolerance=0,\n position_tolerance=1e-8,\n differential_weight=0.5,\n crossover_prob=0.9,\n seed=None,\n name=None):\n \"\"\"Applies the Differential evolution algorithm to minimize a function.\n\n Differential Evolution is an evolutionary optimization algorithm which works\n on a set of candidate solutions called the population. It iteratively\n improves the population by applying genetic operators of mutation and\n recombination. The objective function `f` supplies the fitness of each\n candidate. A candidate `s_1` is considered better than `s_2` if\n `f(s_1) < f(s_2)`.\n\n This method allows the user to either specify an initial population or a\n single candidate solution. If a single solution is specified, a population\n of the specified size is initialized by adding independent normal noise\n to the candidate solution.\n\n The implementation also supports a multi-part specification of the state. For\n example, consider the objective function:\n\n ```python\n # x is a tensor of shape [n, m] while y is of shape [n].\n def objective(x, y):\n return tf.math.reduce_sum(x ** 2, axis=-1) + y ** 2\n ```\n The state in this case is specified by two input tensors `x` and `y`. To\n apply the algorithm to this objective function, one would need to specify\n either an initial population as a list of two tensors of shapes\n `[population_size, k]` and `[population_size]`. The following code shows the\n complete example:\n\n ```python\n population_size = 40\n # With an initial population and a multi-part state.\n initial_population = (tf.random.normal([population_size]),\n tf.random.normal([population_size]))\n def easom_fn(x, y):\n return -(tf.math.cos(x) * tf.math.cos(y) *\n tf.math.exp(-(x-np.pi)**2 - (y-np.pi)**2))\n\n optim_results = tfp.optimizers.differential_evolution_minimize(\n easom_fn,\n initial_population=initial_population,\n seed=43210)\n\n print (optim_results.converged)\n print (optim_results.position) # Should be (close to) [pi, pi].\n print (optim_results.objective_value) # Should be -1.\n\n\n # With a single starting point\n initial_position = (tf.constant(1.0), tf.constant(1.0))\n\n optim_results = tfp.optimizers.differential_evolution_minimize(\n easom_fn,\n initial_position=initial_position,\n population_size=40,\n population_stddev=2.0,\n seed=43210)\n ```\n\n Args:\n objective_function: A Python callable that accepts a batch of possible\n solutions and returns the values of the objective function at those\n arguments as a rank 1 real `Tensor`. This specifies the function to be\n minimized. The input to this callable may be either a single `Tensor`\n or a Python `list` of `Tensor`s. The signature must match the format of\n the argument `population`. (i.e. objective_function(*population) must\n return the value of the function to be minimized).\n initial_population: A real `Tensor` or Python list of `Tensor`s.\n If a list, each `Tensor` must be of rank at least 1 and with a common\n first dimension. The first dimension indexes into the candidate solutions\n while the rest of the dimensions (if any) index into an individual\n solution. The size of the population must be at least 4. This is a\n requirement of the DE algorithm.\n initial_position: A real `Tensor` of any shape. The seed solution used\n to initialize the population of solutions. If this parameter is specified\n then `initial_population` must not be specified.\n population_size: A positive scalar int32 `Tensor` greater than 4. The\n size of the population to evolve. This parameter is ignored if\n `initial_population` is specified.\n Default value: 50.\n population_stddev: A positive scalar real `Tensor` of the same dtype\n as `initial_position`. This parameter is ignored if `initial_population`\n is specified. Used to generate the population from the `initial_position`\n by adding random normal noise with zero mean and the specified standard\n deviation.\n Default value: 1.0\n max_iterations: Positive scalar int32 `Tensor`. The maximum number of\n generations to evolve the population for.\n Default value: 100\n func_tolerance: Scalar `Tensor` of the same dtype as the output of the\n `objective_function`. The algorithm stops if the absolute difference\n between the largest and the smallest objective function value in the\n population is below this number.\n Default value: 0\n position_tolerance: Scalar `Tensor` of the same real dtype as\n `initial_position` or `initial_population`. The algorithm terminates if\n the largest absolute difference between the coordinates of the population\n members is below this threshold.\n Default value: 1e-8\n differential_weight: Real scalar `Tensor`. Must be positive and less than\n 2.0. The parameter controlling the strength of mutation in the algorithm.\n Default value: 0.5\n crossover_prob: Real scalar `Tensor`. Must be between 0 and 1. The\n probability of recombination per site.\n Default value: 0.9\n seed: `int` or None. The random seed for this `Op`. If `None`, no seed is\n applied.\n Default value: None.\n name: (Optional) Python str. The name prefixed to the ops created by this\n function. If not supplied, the default name\n 'differential_evolution_minimize' is used.\n Default value: None\n\n Returns:\n optimizer_results: An object containing the following attributes:\n converged: Scalar boolean `Tensor` indicating whether the minimum was\n found within the specified tolerances.\n num_objective_evaluations: The total number of objective\n evaluations performed.\n position: A `Tensor` containing the best point found during the search.\n If the search converged, then this value is the argmin of the\n objective function within the specified tolerances.\n objective_value: A `Tensor` containing the value of the objective\n function at the `position`. If the search\n converged, then this is the (local) minimum of\n the objective function.\n final_population: The final state of the population.\n final_objective_values: The objective function evaluated at the\n final population.\n initial_population: The starting population.\n initial_objective_values: The objective function evaluated at the\n initial population.\n num_iterations: The number of iterations of the main algorithm body.\n\n Raises:\n ValueError: If neither the initial population, nor the initial position\n are specified or if both are specified.\n \"\"\"\n\n if initial_population is None and initial_position is None:\n raise ValueError('Either the initial population or the initial position '\n 'must be specified.')\n if initial_population is not None and initial_position is not None:\n raise ValueError('Only one of initial population or initial position '\n 'should be specified')\n\n with tf.compat.v1.name_scope(\n name,\n default_name='minimize',\n values=[\n initial_population, initial_position, population_size,\n population_stddev, max_iterations, func_tolerance, position_tolerance,\n differential_weight, crossover_prob\n ]):\n (\n was_iterable,\n population,\n population_values,\n max_iterations,\n func_tolerance,\n position_tolerance,\n differential_weight,\n crossover_prob\n ) = _get_initial_args(objective_function,\n initial_population,\n initial_position,\n population_size,\n population_stddev,\n max_iterations,\n func_tolerance,\n position_tolerance,\n differential_weight,\n crossover_prob,\n seed)\n\n def evolve_body(loop_vars):\n \"\"\"Performs one step of the evolution.\"\"\"\n next_population, next_population_values = one_step(\n objective_function,\n loop_vars.population,\n population_values=loop_vars.population_values,\n differential_weight=differential_weight,\n crossover_prob=crossover_prob,\n seed=seed)\n converged = _check_convergence(next_population,\n next_population_values,\n func_tolerance,\n position_tolerance)\n\n failed = _check_failure(next_population_values)\n\n return [_MinimizeLoopVars(\n converged=converged,\n failed=failed,\n num_iterations=loop_vars.num_iterations+1,\n population=next_population,\n population_values=next_population_values)]\n\n def evolve_cond(loop_vars):\n should_stop = (\n loop_vars.failed |\n loop_vars.converged |\n (max_iterations is not None and\n loop_vars.num_iterations >= max_iterations))\n return ~should_stop\n\n initial_vars = _MinimizeLoopVars(\n converged=tf.convert_to_tensor(value=False),\n failed=tf.convert_to_tensor(value=False),\n num_iterations=tf.convert_to_tensor(value=0),\n population=population,\n population_values=population_values)\n final_state = tf.while_loop(\n cond=evolve_cond, body=evolve_body, loop_vars=(initial_vars,))[0]\n best_position, best_values = _find_best_in_population(\n final_state.population,\n final_state.population_values)\n # Ensure we return a similar structure to what the user supplied.\n final_population = final_state.population\n if not was_iterable:\n final_population = final_population[0]\n best_position = best_position[0]\n return DifferentialEvolutionOptimizerResults(\n converged=final_state.converged,\n failed=final_state.failed,\n position=best_position,\n objective_value=best_values,\n final_population=final_population,\n final_objective_values=final_state.population_values,\n initial_population=population,\n initial_objective_values=population_values,\n num_iterations=final_state.num_iterations)", "response": "Applies the objective function to minimize a function."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nchecks if all the population values are NaN or infinite.", "response": "def _check_failure(population_values):\n \"\"\"Checks if all the population values are NaN/infinite.\"\"\"\n return tf.math.reduce_all(input_tensor=tf.math.is_inf(population_values))"} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\nfinds the population member with the lowest value.", "response": "def _find_best_in_population(population, values):\n \"\"\"Finds the population member with the lowest value.\"\"\"\n best_value = tf.math.reduce_min(input_tensor=values)\n best_index = tf.where(tf.math.equal(values, best_value))[0, 0]\n\n return ([population_part[best_index] for population_part in population],\n best_value)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate a brief explanation for the following Python 3 code\ndef _check_convergence(population,\n population_values,\n func_tolerance,\n position_tolerance):\n \"\"\"Checks whether the convergence criteria have been met.\"\"\"\n # Check func tolerance\n value_range = tf.math.abs(\n tf.math.reduce_max(input_tensor=population_values) -\n tf.math.reduce_min(input_tensor=population_values))\n value_converged = value_range <= func_tolerance\n # Ideally, we would compute the position convergence by computing the\n # pairwise distance between every member of the population and checking if\n # the maximum of those is less than the supplied tolerance. However, this is\n # completely infeasible in terms of performance. We adopt a more conservative\n # approach which checks the distance between the first population member\n # with the rest of the population. If the largest such distance is less than\n # half the supplied tolerance, we stop. The reason why this is sufficient is\n # as follows. For any pair of distinct points (a, b) in the population, we\n # have the relation: |a - b| <= |x0 - a| + |x0 - b|, where x0 is any\n # other point. In particular, let x0 be the first element of the population\n # and suppose that the largest distance between this point and any other\n # member is epsilon. Then, for any pair of points (a, b),\n # |a - b| <= 2 * epsilon and hence, the maximum distance between any pair of\n # points in the population is bounded above by twice the distance between\n # the first point and other points.\n half_tol = position_tolerance / 2\n def part_converged(part):\n return tf.math.reduce_max(input_tensor=tf.math.abs(part -\n part[0])) <= half_tol\n\n x_converged = tf.math.reduce_all(\n input_tensor=[part_converged(part) for part in population])\n return value_converged | x_converged", "response": "Checks whether the convergence criteria have been met."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _get_starting_population(initial_population,\n initial_position,\n population_size,\n population_stddev,\n seed):\n \"\"\"Constructs the initial population.\n\n If an initial population is not already provided, this function constructs\n a population by adding random normal noise to the initial position.\n\n Args:\n initial_population: None or a list of `Tensor`s. The initial population.\n initial_position: None or a list of `Tensor`s. The initial position.\n If initial_population is None, this argument must not be None.\n population_size: Scalar integer `Tensor`. The number of members in the\n population. If the initial population is not None, this parameter is\n ignored.\n population_stddev: A positive scalar real `Tensor` of the same dtype\n as `initial_position` or `initial_population` (whichever is not None).\n This parameter is ignored if `initial_population`\n is specified. Used to generate the population from the\n `initial_position` by adding random normal noise with zero mean and\n the specified standard deviation.\n seed: Seed for random number generation.\n\n Returns:\n A list of `Tensor`s. The initial population.\n \"\"\"\n if initial_population is not None:\n return [tf.convert_to_tensor(value=part) for part in initial_population]\n # Constructs the population by adding normal noise to the initial position.\n seed_stream = distributions.SeedStream(seed, salt='get_starting_population')\n population = []\n for part in initial_position:\n part = tf.convert_to_tensor(value=part)\n part_event_shape = tf.shape(input=part)\n # We only draw population_size-1 random vectors because we want to ensure\n # that the supplied position is part of the population. The first member\n # is set to be the initial_position.\n population_part_shape = tf.concat([[population_size-1],\n part_event_shape], axis=0)\n population_part = tf.random.normal(population_part_shape,\n stddev=population_stddev,\n dtype=part.dtype.base_dtype,\n seed=seed_stream())\n population_part += part\n population_part = tf.concat([[part], population_part], axis=0)\n population.append(population_part)\n return population", "response": "Constructs the starting population."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nperforming binary crossover for the current population.", "response": "def _binary_crossover(population,\n population_size,\n mutants,\n crossover_prob,\n seed):\n \"\"\"Performs recombination by binary crossover for the current population.\n\n Let v_i denote the i'th component of the member v and m_i the corresponding\n component of the mutant vector corresponding to v. Then the crossed over\n vector w_i is determined by setting w_i =\n (m_i with probability=crossover_prob else v_i). In addition, DE requires that\n at least one of the components is crossed over (otherwise we end\n up with no change). This is done by choosing on index say k randomly where\n a force crossover is performed (i.e. w_k = m_k). This is the scheme\n implemented in this function.\n\n Args:\n population: A Python list of `Tensor`s where each `Tensor` in the list\n must be of rank at least 1 and all the elements must have a common\n first dimension. The base population to cross over.\n population_size: A scalar integer `Tensor`. The number of elements in the\n population (i.e. size of the first dimension of any member of\n `population`).\n mutants: A Python list of `Tensor`s with the same structure as `population`.\n The mutated population.\n crossover_prob: A postive real scalar `Tensor` bounded above by 1.0. The\n probability of a crossover being performed for each axis.\n seed: `int` or None. The random seed for this `Op`. If `None`, no seed is\n applied.\n\n Returns:\n A list of `Tensor`s of the same structure, dtype and shape as `population`.\n The recombined population.\n \"\"\"\n sizes = [tf.cast(tf.size(input=x), dtype=tf.float64) for x in population]\n seed_stream = distributions.SeedStream(seed, salt='binary_crossover')\n force_crossover_group = distributions.Categorical(sizes).sample(\n [population_size, 1], seed=seed_stream())\n recombinants = []\n for i, population_part in enumerate(population):\n pop_part_flat = tf.reshape(population_part, [population_size, -1])\n mutant_part_flat = tf.reshape(mutants[i], [population_size, -1])\n part_size = tf.size(input=population_part) // population_size\n force_crossovers = tf.one_hot(\n tf.random.uniform([population_size],\n minval=0,\n maxval=part_size,\n dtype=tf.int32,\n seed=seed_stream()),\n part_size,\n on_value=True,\n off_value=False,\n dtype=tf.bool) # Tensor of shape [population_size, size]\n group_mask = tf.math.equal(force_crossover_group, i)\n force_crossovers &= group_mask\n do_binary_crossover = tf.random.uniform(\n [population_size, part_size],\n dtype=crossover_prob.dtype.base_dtype,\n seed=seed_stream()) < crossover_prob\n do_binary_crossover |= force_crossovers\n recombinant_flat = tf.where(\n do_binary_crossover,\n x=mutant_part_flat,\n y=pop_part_flat)\n recombinant = tf.reshape(recombinant_flat, tf.shape(input=population_part))\n recombinants.append(recombinant)\n return recombinants"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\ncompute the mutatated vectors for each population member.", "response": "def _get_mutants(population,\n population_size,\n mixing_indices,\n differential_weight):\n \"\"\"Computes the mutatated vectors for each population member.\n\n Args:\n population: Python `list` of `Tensor`s representing the\n current population vectors. Each `Tensor` must be of the same real dtype.\n The first dimension of each `Tensor` indexes individual\n population members. For example, if the population is a list with a\n single `Tensor` of shape [n, m1, m2], then `n` is the population size and\n the shape of an individual solution is [m1, m2].\n If there is more than one element in the population, then each `Tensor`\n in the list should have the first axis of the same size.\n population_size: Scalar integer `Tensor`. The size of the population.\n mixing_indices: `Tensor` of integral dtype and shape [n, 3] where `n` is the\n number of members in the population. Each element of the `Tensor` must be\n a valid index into the first dimension of the population (i.e range\n between `0` and `n-1` inclusive).\n differential_weight: Real scalar `Tensor`. Must be positive and less than\n 2.0. The parameter controlling the strength of mutation.\n\n Returns:\n mutants: `Tensor` or Python `list` of `Tensor`s of the same shape and dtype\n as the input population. The mutated vectors.\n \"\"\"\n mixing_indices = tf.reshape(mixing_indices, [-1])\n weights = tf.stack([1.0, differential_weight, -differential_weight])\n def _mutant_part(population_part):\n donors = tf.gather(population_part, mixing_indices)\n donors = tf.transpose(\n a=tf.reshape(donors, [population_size, 3, -1]), perm=[0, 2, 1])\n return tf.math.reduce_sum(input_tensor=donors * weights, axis=-1)\n\n return [_mutant_part(population_part) for population_part in population]"} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _get_mixing_indices(size, seed=None, name=None):\n with tf.compat.v1.name_scope(\n name, default_name='get_mixing_indices', values=[size]):\n size = tf.convert_to_tensor(value=size)\n dtype = size.dtype\n seed_stream = distributions.SeedStream(seed, salt='get_mixing_indices')\n first = tf.random.uniform([size],\n maxval=size-1,\n dtype=dtype,\n seed=seed_stream())\n second = tf.random.uniform([size],\n maxval=size-2,\n dtype=dtype,\n seed=seed_stream())\n third = tf.random.uniform([size],\n maxval=size-3,\n dtype=dtype,\n seed=seed_stream())\n\n # Shift second if it is on top of or to the right of first\n second = tf.where(first < second, x=second, y=second + 1)\n smaller = tf.math.minimum(first, second)\n larger = tf.math.maximum(first, second)\n # Shift the third one so it does not coincide with either the first or the\n # second number. Assuming first < second, shift by 1 if the number is in\n # [first, second) and by 2 if the number is greater than or equal to the\n # second.\n third = tf.where(third < smaller, x=third, y=third + 1)\n third = tf.where(third < larger, x=third, y=third + 1)\n sample = tf.stack([first, second, third], axis=1)\n to_avoid = tf.expand_dims(tf.range(size), axis=-1)\n sample = tf.where(sample < to_avoid, x=sample, y=sample + 1)\n return sample", "response": "Generates an array of indices suitable for a differential evolution mutation operation."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _ensure_list(tensor_or_list):\n if isinstance(tensor_or_list, (list, tuple)):\n return list(tensor_or_list), True\n return [tensor_or_list], False", "response": "Converts the input arg to a list if it is not a list already."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef _get_tol(tol, dtype, validate_args):\n if tol is None:\n return tf.convert_to_tensor(value=0, dtype=dtype)\n\n tol = tf.convert_to_tensor(value=tol, dtype=dtype)\n if validate_args:\n tol = distribution_util.with_dependencies([\n assert_util.assert_non_negative(\n tol, message=\"Argument 'tol' must be non-negative\")\n ], tol)\n return tol", "response": "Gets a Tensor of type dtype 0 if tol is None validation optional."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function for\ncalculating the batched KL divergence with a Deterministic distribution.", "response": "def _kl_deterministic_distribution(a, b, name=None):\n \"\"\"Calculate the batched KL divergence `KL(a || b)` with `a` Deterministic.\n\n Args:\n a: instance of a Deterministic distribution object.\n b: instance of a Distribution distribution object.\n name: (optional) Name to use for created operations. Default is\n \"kl_deterministic_distribution\".\n\n Returns:\n Batchwise `KL(a || b)`.\n \"\"\"\n with tf.name_scope(name or \"kl_deterministic_distribution\"):\n return -b.log_prob(a.loc)"} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nclipping values to a specified min and max while leaving gradient unaltered.", "response": "def clip_by_value_preserve_gradient(t, clip_value_min, clip_value_max,\n name=None):\n \"\"\"Clips values to a specified min and max while leaving gradient unaltered.\n\n Like `tf.clip_by_value`, this function returns a tensor of the same type and\n shape as input `t` but with values clamped to be no smaller than to\n `clip_value_min` and no larger than `clip_value_max`. Unlike\n `tf.clip_by_value`, the gradient is unaffected by this op, i.e.,\n\n ```python\n tf.gradients(tfp.math.clip_by_value_preserve_gradient(x), x)[0]\n # ==> ones_like(x)\n ```\n\n Note: `clip_value_min` needs to be smaller or equal to `clip_value_max` for\n correct results.\n\n Args:\n t: A `Tensor`.\n clip_value_min: A scalar `Tensor`, or a `Tensor` with the same shape\n as `t`. The minimum value to clip by.\n clip_value_max: A scalar `Tensor`, or a `Tensor` with the same shape\n as `t`. The maximum value to clip by.\n name: A name for the operation (optional).\n Default value: `'clip_by_value_preserve_gradient'`.\n\n Returns:\n clipped_t: A clipped `Tensor`.\n \"\"\"\n with tf.compat.v1.name_scope(name, 'clip_by_value_preserve_gradient',\n [t, clip_value_min, clip_value_max]):\n t = tf.convert_to_tensor(value=t, name='t')\n clip_t = tf.clip_by_value(t, clip_value_min, clip_value_max)\n return t + tf.stop_gradient(clip_t - t)"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef build_input_pipeline(train_images, batch_size):\n\n training_dataset = tf.data.Dataset.from_tensor_slices(train_images)\n training_batches = training_dataset.shuffle(\n 50000, reshuffle_each_iteration=True).repeat().batch(batch_size)\n training_iterator = tf.compat.v1.data.make_one_shot_iterator(training_batches)\n images = training_iterator.get_next()\n return images", "response": "Build an iterator over training batches."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nsave a synthetic image as a PNG file.", "response": "def plot_generated_images(images, fname):\n \"\"\"Save a synthetic image as a PNG file.\n\n Args:\n images: samples of synthetic images generated by the generative network.\n fname: Python `str`, filename to save the plot to.\n \"\"\"\n fig = plt.figure(figsize=(4, 4))\n canvas = backend_agg.FigureCanvasAgg(fig)\n\n for i, image in enumerate(images):\n ax = fig.add_subplot(4, 4, i + 1)\n plt.axis('off')\n ax.set_xticklabels([])\n ax.set_yticklabels([])\n ax.imshow(image.reshape(IMAGE_SHAPE[:-1]), cmap='Greys_r')\n\n fig.tight_layout()\n plt.subplots_adjust(wspace=0.05, hspace=0.05)\n canvas.print_figure(fname, format='png')"} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function for\nconverting a sequence of productions into a string of terminal symbols.", "response": "def convert_to_string(self, productions):\n \"\"\"Converts a sequence of productions into a string of terminal symbols.\n\n Args:\n productions: Tensor of shape [1, num_productions, num_production_rules].\n Slices along the `num_productions` dimension represent one-hot vectors.\n\n Returns:\n str that concatenates all terminal symbols from `productions`.\n\n Raises:\n ValueError: If the first production rule does not begin with\n `self.start_symbol`.\n \"\"\"\n symbols = []\n for production in tf.unstack(productions, axis=1):\n lhs, rhs = self.production_rules[tf.argmax(input=production, axis=-1)]\n if not symbols: # first iteration\n if lhs != self.start_symbol:\n raise ValueError(\"`productions` must begin with `self.start_symbol`.\")\n symbols = rhs\n else:\n # Greedily unroll the nonterminal symbols based on the first occurrence\n # in a linear sequence.\n index = symbols.index(lhs)\n symbols = symbols[:index] + rhs + symbols[index + 1:]\n string = \"\".join(symbols)\n return string"} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nproduce a masking tensor for valid production rules.", "response": "def mask(self, symbol, on_value, off_value):\n \"\"\"Produces a masking tensor for (in)valid production rules.\n\n Args:\n symbol: str, a symbol in the grammar.\n on_value: Value to use for a valid production rule.\n off_value: Value to use for an invalid production rule.\n\n Returns:\n Tensor of shape [1, num_production_rules]. An element is `on_value`\n if its corresponding production rule has `symbol` on its left-hand-side;\n the element is `off_value` otherwise.\n \"\"\"\n mask_values = [on_value if lhs == symbol else off_value\n for lhs, _ in self.production_rules]\n mask_values = tf.reshape(mask_values, [1, len(self.production_rules)])\n return mask_values"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef call(self, inputs):\n del inputs # unused\n latent_code = ed.MultivariateNormalDiag(loc=tf.zeros(self.latent_size),\n sample_shape=1,\n name=\"latent_code\")\n state = self.lstm.zero_state(1, dtype=tf.float32)\n t = 0\n productions = []\n stack = [self.grammar.start_symbol]\n while stack:\n symbol = stack.pop()\n net, state = self.lstm(latent_code, state)\n logits = (self.output_layer(net) +\n self.grammar.mask(symbol, on_value=0., off_value=-1e9))\n production = ed.OneHotCategorical(logits=logits,\n name=\"production_\" + str(t))\n _, rhs = self.grammar.production_rules[tf.argmax(\n input=production, axis=-1)]\n for symbol in rhs:\n if symbol in self.grammar.nonterminal_symbols:\n stack.append(symbol)\n productions.append(production)\n t += 1\n return tf.stack(productions, axis=1)", "response": "Runs the model forward to generate a sequence of productions."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef call(self, inputs):\n net = self.encoder_net(tf.cast(inputs, tf.float32))\n return ed.MultivariateNormalDiag(\n loc=net[..., :self.latent_size],\n scale_diag=tf.nn.softplus(net[..., self.latent_size:]),\n name=\"latent_code_posterior\")", "response": "Runs the model forward to return a stochastic encoding."} {"SOURCE": "codesearchnet", "instruction": "Explain what the following Python 3 code does\ndef _hat_integral(self, x):\n x = tf.cast(x, self.power.dtype)\n t = self.power - 1.\n return tf.exp((-t) * tf.math.log1p(x) - tf.math.log(t))", "response": "Integral of the hat function used for sampling."} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _hat_integral_inverse(self, x):\n x = tf.cast(x, self.power.dtype)\n t = self.power - 1.\n return tf.math.expm1(-(tf.math.log(t) + tf.math.log(x)) / t)", "response": "Inverse function of _hat_integral."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef matrix_rank(a, tol=None, validate_args=False, name=None):\n with tf.compat.v1.name_scope(name, 'matrix_rank', [a, tol]):\n a = tf.convert_to_tensor(value=a, dtype_hint=tf.float32, name='a')\n assertions = _maybe_validate_matrix(a, validate_args)\n if assertions:\n with tf.control_dependencies(assertions):\n a = tf.identity(a)\n s = tf.linalg.svd(a, compute_uv=False)\n if tol is None:\n if a.shape[-2:].is_fully_defined():\n m = np.max(a.shape[-2:].as_list())\n else:\n m = tf.reduce_max(input_tensor=tf.shape(input=a)[-2:])\n eps = np.finfo(a.dtype.as_numpy_dtype).eps\n tol = (eps * tf.cast(m, a.dtype) *\n tf.reduce_max(input_tensor=s, axis=-1, keepdims=True))\n return tf.reduce_sum(input_tensor=tf.cast(s > tol, tf.int32), axis=-1)", "response": "Compute the matrix rank ; the number of non - zero SVD singular values."} {"SOURCE": "codesearchnet", "instruction": "How would you code a function in Python 3 to\nconcatenate cholesky decomposition of chol.", "response": "def cholesky_concat(chol, cols, name=None):\n \"\"\"Concatenates `chol @ chol.T` with additional rows and columns.\n\n This operation is conceptually identical to:\n ```python\n def cholesky_concat_slow(chol, cols): # cols shaped (n + m) x m = z x m\n mat = tf.matmul(chol, chol, adjoint_b=True) # batch of n x n\n # Concat columns.\n mat = tf.concat([mat, cols[..., :tf.shape(mat)[-2], :]], axis=-1) # n x z\n # Concat rows.\n mat = tf.concat([mat, tf.linalg.matrix_transpose(cols)], axis=-2) # z x z\n return tf.linalg.cholesky(mat)\n ```\n but whereas `cholesky_concat_slow` would cost `O(z**3)` work,\n `cholesky_concat` only costs `O(z**2 + m**3)` work.\n\n The resulting (implicit) matrix must be symmetric and positive definite.\n Thus, the bottom right `m x m` must be self-adjoint, and we do not require a\n separate `rows` argument (which can be inferred from `conj(cols.T)`).\n\n Args:\n chol: Cholesky decomposition of `mat = chol @ chol.T`.\n cols: The new columns whose first `n` rows we would like concatenated to the\n right of `mat = chol @ chol.T`, and whose conjugate transpose we would\n like concatenated to the bottom of `concat(mat, cols[:n,:])`. A `Tensor`\n with final dims `(n+m, m)`. The first `n` rows are the top right rectangle\n (their conjugate transpose forms the bottom left), and the bottom `m x m`\n is self-adjoint.\n name: Optional name for this op.\n\n Returns:\n chol_concat: The Cholesky decomposition of:\n ```\n [ [ mat cols[:n, :] ]\n [ conj(cols.T) ] ]\n ```\n \"\"\"\n with tf.compat.v2.name_scope(name or 'cholesky_extend'):\n dtype = dtype_util.common_dtype([chol, cols], preferred_dtype=tf.float32)\n chol = tf.convert_to_tensor(value=chol, name='chol', dtype=dtype)\n cols = tf.convert_to_tensor(value=cols, name='cols', dtype=dtype)\n n = prefer_static.shape(chol)[-1]\n mat_nm, mat_mm = cols[..., :n, :], cols[..., n:, :]\n solved_nm = linear_operator_util.matrix_triangular_solve_with_broadcast(\n chol, mat_nm)\n lower_right_mm = tf.linalg.cholesky(\n mat_mm - tf.matmul(solved_nm, solved_nm, adjoint_a=True))\n lower_left_mn = tf.math.conj(tf.linalg.matrix_transpose(solved_nm))\n out_batch = prefer_static.shape(solved_nm)[:-2]\n chol = tf.broadcast_to(\n chol,\n tf.concat([out_batch, prefer_static.shape(chol)[-2:]], axis=0))\n top_right_zeros_nm = tf.zeros_like(solved_nm)\n return tf.concat([tf.concat([chol, top_right_zeros_nm], axis=-1),\n tf.concat([lower_left_mn, lower_right_mm], axis=-1)],\n axis=-2)"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script for\nswapping `m` and `i` on axis -1. (Helper for pivoted_cholesky.) Given a batch of int64 vectors `vecs`, scalar index `m`, and compatibly shaped per-vector indices `i`, this function swaps elements `m` and `i` in each vector. For the use-case below, these are permutation vectors. Args: vecs: Vectors on which we perform the swap, int64 `Tensor`. m: Scalar int64 `Tensor`, the index into which the `i`th element is going. i: Batch int64 `Tensor`, shaped like vecs.shape[:-1] + [1]; the index into which the `m`th element is going. Returns: vecs: The updated vectors.", "response": "def _swap_m_with_i(vecs, m, i):\n \"\"\"Swaps `m` and `i` on axis -1. (Helper for pivoted_cholesky.)\n\n Given a batch of int64 vectors `vecs`, scalar index `m`, and compatibly shaped\n per-vector indices `i`, this function swaps elements `m` and `i` in each\n vector. For the use-case below, these are permutation vectors.\n\n Args:\n vecs: Vectors on which we perform the swap, int64 `Tensor`.\n m: Scalar int64 `Tensor`, the index into which the `i`th element is going.\n i: Batch int64 `Tensor`, shaped like vecs.shape[:-1] + [1]; the index into\n which the `m`th element is going.\n\n Returns:\n vecs: The updated vectors.\n \"\"\"\n vecs = tf.convert_to_tensor(value=vecs, dtype=tf.int64, name='vecs')\n m = tf.convert_to_tensor(value=m, dtype=tf.int64, name='m')\n i = tf.convert_to_tensor(value=i, dtype=tf.int64, name='i')\n trailing_elts = tf.broadcast_to(\n tf.range(m + 1,\n prefer_static.shape(vecs, out_type=tf.int64)[-1]),\n prefer_static.shape(vecs[..., m + 1:]))\n shp = prefer_static.shape(trailing_elts)\n trailing_elts = tf.where(\n tf.equal(trailing_elts, tf.broadcast_to(i, shp)),\n tf.broadcast_to(tf.gather(vecs, [m], axis=-1), shp),\n tf.broadcast_to(vecs[..., m + 1:], shp))\n # TODO(bjp): Could we use tensor_scatter_nd_update?\n vecs_shape = vecs.shape\n vecs = tf.concat([\n vecs[..., :m],\n tf.gather(vecs, i, batch_dims=prefer_static.rank(vecs) - 1), trailing_elts\n ], axis=-1)\n tensorshape_util.set_shape(vecs, vecs_shape)\n return vecs"} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef pivoted_cholesky(matrix, max_rank, diag_rtol=1e-3, name=None):\n with tf.compat.v2.name_scope(name or 'pivoted_cholesky'):\n dtype = dtype_util.common_dtype([matrix, diag_rtol],\n preferred_dtype=tf.float32)\n matrix = tf.convert_to_tensor(value=matrix, name='matrix', dtype=dtype)\n if tensorshape_util.rank(matrix.shape) is None:\n raise NotImplementedError('Rank of `matrix` must be known statically')\n\n max_rank = tf.convert_to_tensor(\n value=max_rank, name='max_rank', dtype=tf.int64)\n max_rank = tf.minimum(max_rank,\n prefer_static.shape(matrix, out_type=tf.int64)[-1])\n diag_rtol = tf.convert_to_tensor(\n value=diag_rtol, dtype=dtype, name='diag_rtol')\n matrix_diag = tf.linalg.diag_part(matrix)\n # matrix is P.D., therefore all matrix_diag > 0, so we don't need abs.\n orig_error = tf.reduce_max(input_tensor=matrix_diag, axis=-1)\n\n def cond(m, pchol, perm, matrix_diag):\n \"\"\"Condition for `tf.while_loop` continuation.\"\"\"\n del pchol\n del perm\n error = tf.linalg.norm(tensor=matrix_diag, ord=1, axis=-1)\n max_err = tf.reduce_max(input_tensor=error / orig_error)\n return (m < max_rank) & (tf.equal(m, 0) | (max_err > diag_rtol))\n\n batch_dims = tensorshape_util.rank(matrix.shape) - 2\n def batch_gather(params, indices, axis=-1):\n return tf.gather(params, indices, axis=axis, batch_dims=batch_dims)\n\n def body(m, pchol, perm, matrix_diag):\n \"\"\"Body of a single `tf.while_loop` iteration.\"\"\"\n # Here is roughly a numpy, non-batched version of what's going to happen.\n # (See also Algorithm 1 of Harbrecht et al.)\n # 1: maxi = np.argmax(matrix_diag[perm[m:]]) + m\n # 2: maxval = matrix_diag[perm][maxi]\n # 3: perm[m], perm[maxi] = perm[maxi], perm[m]\n # 4: row = matrix[perm[m]][perm[m + 1:]]\n # 5: row -= np.sum(pchol[:m][perm[m + 1:]] * pchol[:m][perm[m]]], axis=-2)\n # 6: pivot = np.sqrt(maxval); row /= pivot\n # 7: row = np.concatenate([[[pivot]], row], -1)\n # 8: matrix_diag[perm[m:]] -= row**2\n # 9: pchol[m, perm[m:]] = row\n\n # Find the maximal position of the (remaining) permuted diagonal.\n # Steps 1, 2 above.\n permuted_diag = batch_gather(matrix_diag, perm[..., m:])\n maxi = tf.argmax(\n input=permuted_diag, axis=-1, output_type=tf.int64)[..., tf.newaxis]\n maxval = batch_gather(permuted_diag, maxi)\n maxi = maxi + m\n maxval = maxval[..., 0]\n # Update perm: Swap perm[...,m] with perm[...,maxi]. Step 3 above.\n perm = _swap_m_with_i(perm, m, maxi)\n # Step 4.\n row = batch_gather(matrix, perm[..., m:m + 1], axis=-2)\n row = batch_gather(row, perm[..., m + 1:])\n # Step 5.\n prev_rows = pchol[..., :m, :]\n prev_rows_perm_m_onward = batch_gather(prev_rows, perm[..., m + 1:])\n prev_rows_pivot_col = batch_gather(prev_rows, perm[..., m:m + 1])\n row -= tf.reduce_sum(\n input_tensor=prev_rows_perm_m_onward * prev_rows_pivot_col,\n axis=-2)[..., tf.newaxis, :]\n # Step 6.\n pivot = tf.sqrt(maxval)[..., tf.newaxis, tf.newaxis]\n # Step 7.\n row = tf.concat([pivot, row / pivot], axis=-1)\n # TODO(b/130899118): Pad grad fails with int64 paddings.\n # Step 8.\n paddings = tf.concat([\n tf.zeros([prefer_static.rank(pchol) - 1, 2], dtype=tf.int32),\n [[tf.cast(m, tf.int32), 0]]], axis=0)\n diag_update = tf.pad(tensor=row**2, paddings=paddings)[..., 0, :]\n reverse_perm = _invert_permutation(perm)\n matrix_diag -= batch_gather(diag_update, reverse_perm)\n # Step 9.\n row = tf.pad(tensor=row, paddings=paddings)\n # TODO(bjp): Defer the reverse permutation all-at-once at the end?\n row = batch_gather(row, reverse_perm)\n pchol_shape = pchol.shape\n pchol = tf.concat([pchol[..., :m, :], row, pchol[..., m + 1:, :]],\n axis=-2)\n tensorshape_util.set_shape(pchol, pchol_shape)\n return m + 1, pchol, perm, matrix_diag\n\n m = np.int64(0)\n pchol = tf.zeros_like(matrix[..., :max_rank, :])\n matrix_shape = prefer_static.shape(matrix, out_type=tf.int64)\n perm = tf.broadcast_to(\n prefer_static.range(matrix_shape[-1]), matrix_shape[:-1])\n _, pchol, _, _ = tf.while_loop(\n cond=cond, body=body, loop_vars=(m, pchol, perm, matrix_diag))\n pchol = tf.linalg.matrix_transpose(pchol)\n tensorshape_util.set_shape(\n pchol, tensorshape_util.concatenate(matrix_diag.shape, [None]))\n return pchol", "response": "Computes the ( partial ) pivoted Cholesky decomposition of matrix."} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef pinv(a, rcond=None, validate_args=False, name=None):\n with tf.compat.v1.name_scope(name, 'pinv', [a, rcond]):\n a = tf.convert_to_tensor(value=a, name='a')\n\n assertions = _maybe_validate_matrix(a, validate_args)\n if assertions:\n with tf.control_dependencies(assertions):\n a = tf.identity(a)\n\n dtype = a.dtype.as_numpy_dtype\n\n if rcond is None:\n def get_dim_size(dim):\n if tf.compat.dimension_value(a.shape[dim]) is not None:\n return tf.compat.dimension_value(a.shape[dim])\n return tf.shape(input=a)[dim]\n\n num_rows = get_dim_size(-2)\n num_cols = get_dim_size(-1)\n if isinstance(num_rows, int) and isinstance(num_cols, int):\n max_rows_cols = float(max(num_rows, num_cols))\n else:\n max_rows_cols = tf.cast(tf.maximum(num_rows, num_cols), dtype)\n rcond = 10. * max_rows_cols * np.finfo(dtype).eps\n\n rcond = tf.convert_to_tensor(value=rcond, dtype=dtype, name='rcond')\n\n # Calculate pseudo inverse via SVD.\n # Note: if a is symmetric then u == v. (We might observe additional\n # performance by explicitly setting `v = u` in such cases.)\n [\n singular_values, # Sigma\n left_singular_vectors, # U\n right_singular_vectors, # V\n ] = tf.linalg.svd(a, full_matrices=False, compute_uv=True)\n\n # Saturate small singular values to inf. This has the effect of make\n # `1. / s = 0.` while not resulting in `NaN` gradients.\n cutoff = rcond * tf.reduce_max(input_tensor=singular_values, axis=-1)\n singular_values = tf.where(\n singular_values > cutoff[..., tf.newaxis], singular_values,\n tf.fill(tf.shape(input=singular_values), np.array(np.inf, dtype)))\n\n # Although `a == tf.matmul(u, s * v, transpose_b=True)` we swap\n # `u` and `v` here so that `tf.matmul(pinv(A), A) = tf.eye()`, i.e.,\n # a matrix inverse has \"transposed\" semantics.\n a_pinv = tf.matmul(\n right_singular_vectors / singular_values[..., tf.newaxis, :],\n left_singular_vectors,\n adjoint_b=True)\n\n if a.shape.ndims is not None:\n a_pinv.set_shape(a.shape[:-2].concatenate([a.shape[-1], a.shape[-2]]))\n\n return a_pinv", "response": "Compute the Moore - Penrose pseudo - inverse of a matrix."} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef lu_solve(lower_upper, perm, rhs,\n validate_args=False,\n name=None):\n \"\"\"Solves systems of linear eqns `A X = RHS`, given LU factorizations.\n\n Note: this function does not verify the implied matrix is actually invertible\n nor is this condition checked even when `validate_args=True`.\n\n Args:\n lower_upper: `lu` as returned by `tf.linalg.lu`, i.e., if\n `matmul(P, matmul(L, U)) = X` then `lower_upper = L + U - eye`.\n perm: `p` as returned by `tf.linag.lu`, i.e., if\n `matmul(P, matmul(L, U)) = X` then `perm = argmax(P)`.\n rhs: Matrix-shaped float `Tensor` representing targets for which to solve;\n `A X = RHS`. To handle vector cases, use:\n `lu_solve(..., rhs[..., tf.newaxis])[..., 0]`.\n validate_args: Python `bool` indicating whether arguments should be checked\n for correctness. Note: this function does not verify the implied matrix is\n actually invertible, even when `validate_args=True`.\n Default value: `False` (i.e., don't validate arguments).\n name: Python `str` name given to ops managed by this object.\n Default value: `None` (i.e., \"lu_solve\").\n\n Returns:\n x: The `X` in `A @ X = RHS`.\n\n #### Examples\n\n ```python\n import numpy as np\n import tensorflow as tf\n import tensorflow_probability as tfp\n\n x = [[[1., 2],\n [3, 4]],\n [[7, 8],\n [3, 4]]]\n inv_x = tfp.math.lu_solve(*tf.linalg.lu(x), rhs=tf.eye(2))\n tf.assert_near(tf.matrix_inverse(x), inv_x)\n # ==> True\n ```\n\n \"\"\"\n\n with tf.compat.v1.name_scope(name, 'lu_solve', [lower_upper, perm, rhs]):\n lower_upper = tf.convert_to_tensor(\n value=lower_upper, dtype_hint=tf.float32, name='lower_upper')\n perm = tf.convert_to_tensor(value=perm, dtype_hint=tf.int32, name='perm')\n rhs = tf.convert_to_tensor(\n value=rhs, dtype_hint=lower_upper.dtype, name='rhs')\n\n assertions = _lu_solve_assertions(lower_upper, perm, rhs, validate_args)\n if assertions:\n with tf.control_dependencies(assertions):\n lower_upper = tf.identity(lower_upper)\n perm = tf.identity(perm)\n rhs = tf.identity(rhs)\n\n if rhs.shape.ndims == 2 and perm.shape.ndims == 1:\n # Both rhs and perm have scalar batch_shape.\n permuted_rhs = tf.gather(rhs, perm, axis=-2)\n else:\n # Either rhs or perm have non-scalar batch_shape or we can't determine\n # this information statically.\n rhs_shape = tf.shape(input=rhs)\n broadcast_batch_shape = tf.broadcast_dynamic_shape(\n rhs_shape[:-2],\n tf.shape(input=perm)[:-1])\n d, m = rhs_shape[-2], rhs_shape[-1]\n rhs_broadcast_shape = tf.concat([broadcast_batch_shape, [d, m]], axis=0)\n\n # Tile out rhs.\n broadcast_rhs = tf.broadcast_to(rhs, rhs_broadcast_shape)\n broadcast_rhs = tf.reshape(broadcast_rhs, [-1, d, m])\n\n # Tile out perm and add batch indices.\n broadcast_perm = tf.broadcast_to(perm, rhs_broadcast_shape[:-1])\n broadcast_perm = tf.reshape(broadcast_perm, [-1, d])\n broadcast_batch_size = tf.reduce_prod(input_tensor=broadcast_batch_shape)\n broadcast_batch_indices = tf.broadcast_to(\n tf.range(broadcast_batch_size)[:, tf.newaxis],\n [broadcast_batch_size, d])\n broadcast_perm = tf.stack([broadcast_batch_indices, broadcast_perm],\n axis=-1)\n\n permuted_rhs = tf.gather_nd(broadcast_rhs, broadcast_perm)\n permuted_rhs = tf.reshape(permuted_rhs, rhs_broadcast_shape)\n\n lower = tf.linalg.set_diag(\n tf.linalg.band_part(lower_upper, num_lower=-1, num_upper=0),\n tf.ones(tf.shape(input=lower_upper)[:-1], dtype=lower_upper.dtype))\n return linear_operator_util.matrix_triangular_solve_with_broadcast(\n lower_upper, # Only upper is accessed.\n linear_operator_util.matrix_triangular_solve_with_broadcast(\n lower, permuted_rhs),\n lower=False)", "response": "Solves system of linear eqns A X = RHS given LU factorizations."} {"SOURCE": "codesearchnet", "instruction": "Create a Python 3 function to\ncompute the matrix inverse given the matrix s LU decomposition.", "response": "def lu_matrix_inverse(lower_upper, perm, validate_args=False, name=None):\n \"\"\"Computes a matrix inverse given the matrix's LU decomposition.\n\n This op is conceptually identical to,\n\n ````python\n inv_X = tf.lu_matrix_inverse(*tf.linalg.lu(X))\n tf.assert_near(tf.matrix_inverse(X), inv_X)\n # ==> True\n ```\n\n Note: this function does not verify the implied matrix is actually invertible\n nor is this condition checked even when `validate_args=True`.\n\n Args:\n lower_upper: `lu` as returned by `tf.linalg.lu`, i.e., if\n `matmul(P, matmul(L, U)) = X` then `lower_upper = L + U - eye`.\n perm: `p` as returned by `tf.linag.lu`, i.e., if\n `matmul(P, matmul(L, U)) = X` then `perm = argmax(P)`.\n validate_args: Python `bool` indicating whether arguments should be checked\n for correctness. Note: this function does not verify the implied matrix is\n actually invertible, even when `validate_args=True`.\n Default value: `False` (i.e., don't validate arguments).\n name: Python `str` name given to ops managed by this object.\n Default value: `None` (i.e., \"lu_matrix_inverse\").\n\n Returns:\n inv_x: The matrix_inv, i.e.,\n `tf.matrix_inverse(tfp.math.lu_reconstruct(lu, perm))`.\n\n #### Examples\n\n ```python\n import numpy as np\n import tensorflow as tf\n import tensorflow_probability as tfp\n\n x = [[[3., 4], [1, 2]],\n [[7., 8], [3, 4]]]\n inv_x = tfp.math.lu_matrix_inverse(*tf.linalg.lu(x))\n tf.assert_near(tf.matrix_inverse(x), inv_x)\n # ==> True\n ```\n\n \"\"\"\n\n with tf.compat.v1.name_scope(name, 'lu_matrix_inverse', [lower_upper, perm]):\n lower_upper = tf.convert_to_tensor(\n value=lower_upper, dtype_hint=tf.float32, name='lower_upper')\n perm = tf.convert_to_tensor(value=perm, dtype_hint=tf.int32, name='perm')\n assertions = _lu_reconstruct_assertions(lower_upper, perm, validate_args)\n if assertions:\n with tf.control_dependencies(assertions):\n lower_upper = tf.identity(lower_upper)\n perm = tf.identity(perm)\n shape = tf.shape(input=lower_upper)\n return lu_solve(\n lower_upper, perm,\n rhs=tf.eye(shape[-1], batch_shape=shape[:-2], dtype=lower_upper.dtype),\n validate_args=False)"} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef lu_reconstruct(lower_upper, perm, validate_args=False, name=None):\n with tf.compat.v1.name_scope(name, 'lu_reconstruct', [lower_upper, perm]):\n lower_upper = tf.convert_to_tensor(\n value=lower_upper, dtype_hint=tf.float32, name='lower_upper')\n perm = tf.convert_to_tensor(value=perm, dtype_hint=tf.int32, name='perm')\n\n assertions = _lu_reconstruct_assertions(lower_upper, perm, validate_args)\n if assertions:\n with tf.control_dependencies(assertions):\n lower_upper = tf.identity(lower_upper)\n perm = tf.identity(perm)\n\n shape = tf.shape(input=lower_upper)\n\n lower = tf.linalg.set_diag(\n tf.linalg.band_part(lower_upper, num_lower=-1, num_upper=0),\n tf.ones(shape[:-1], dtype=lower_upper.dtype))\n upper = tf.linalg.band_part(lower_upper, num_lower=0, num_upper=-1)\n x = tf.matmul(lower, upper)\n\n if lower_upper.shape.ndims is None or lower_upper.shape.ndims != 2:\n # We either don't know the batch rank or there are >0 batch dims.\n batch_size = tf.reduce_prod(input_tensor=shape[:-2])\n d = shape[-1]\n x = tf.reshape(x, [batch_size, d, d])\n perm = tf.reshape(perm, [batch_size, d])\n perm = tf.map_fn(tf.math.invert_permutation, perm)\n batch_indices = tf.broadcast_to(\n tf.range(batch_size)[:, tf.newaxis],\n [batch_size, d])\n x = tf.gather_nd(x, tf.stack([batch_indices, perm], axis=-1))\n x = tf.reshape(x, shape)\n else:\n x = tf.gather(x, tf.math.invert_permutation(perm))\n\n x.set_shape(lower_upper.shape)\n return x", "response": "The inverse LU decomposition of X."} {"SOURCE": "codesearchnet", "instruction": "Can you write a function in Python 3 where it\nreturns a list of assertions related to lu_reconstruct assumptions.", "response": "def _lu_reconstruct_assertions(lower_upper, perm, validate_args):\n \"\"\"Returns list of assertions related to `lu_reconstruct` assumptions.\"\"\"\n assertions = []\n\n message = 'Input `lower_upper` must have at least 2 dimensions.'\n if lower_upper.shape.ndims is not None:\n if lower_upper.shape.ndims < 2:\n raise ValueError(message)\n elif validate_args:\n assertions.append(\n tf.compat.v1.assert_rank_at_least(lower_upper, rank=2, message=message))\n\n message = '`rank(lower_upper)` must equal `rank(perm) + 1`'\n if lower_upper.shape.ndims is not None and perm.shape.ndims is not None:\n if lower_upper.shape.ndims != perm.shape.ndims + 1:\n raise ValueError(message)\n elif validate_args:\n assertions.append(\n tf.compat.v1.assert_rank(\n lower_upper, rank=tf.rank(perm) + 1, message=message))\n\n message = '`lower_upper` must be square.'\n if lower_upper.shape[:-2].is_fully_defined():\n if lower_upper.shape[-2] != lower_upper.shape[-1]:\n raise ValueError(message)\n elif validate_args:\n m, n = tf.split(tf.shape(input=lower_upper)[-2:], num_or_size_splits=2)\n assertions.append(tf.compat.v1.assert_equal(m, n, message=message))\n\n return assertions"} {"SOURCE": "codesearchnet", "instruction": "Here you have a function in Python 3, explain what it does\ndef _lu_solve_assertions(lower_upper, perm, rhs, validate_args):\n assertions = _lu_reconstruct_assertions(lower_upper, perm, validate_args)\n\n message = 'Input `rhs` must have at least 2 dimensions.'\n if rhs.shape.ndims is not None:\n if rhs.shape.ndims < 2:\n raise ValueError(message)\n elif validate_args:\n assertions.append(\n tf.compat.v1.assert_rank_at_least(rhs, rank=2, message=message))\n\n message = '`lower_upper.shape[-1]` must equal `rhs.shape[-1]`.'\n if (tf.compat.dimension_value(lower_upper.shape[-1]) is not None and\n tf.compat.dimension_value(rhs.shape[-2]) is not None):\n if lower_upper.shape[-1] != rhs.shape[-2]:\n raise ValueError(message)\n elif validate_args:\n assertions.append(\n tf.compat.v1.assert_equal(\n tf.shape(input=lower_upper)[-1],\n tf.shape(input=rhs)[-2],\n message=message))\n\n return assertions", "response": "Returns list of assertions related to lu_solve assumptions."} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 script to\nreturn a Tensor or SparseTensor with a Tensor.", "response": "def sparse_or_dense_matmul(sparse_or_dense_a,\n dense_b,\n validate_args=False,\n name=None,\n **kwargs):\n \"\"\"Returns (batched) matmul of a SparseTensor (or Tensor) with a Tensor.\n\n Args:\n sparse_or_dense_a: `SparseTensor` or `Tensor` representing a (batch of)\n matrices.\n dense_b: `Tensor` representing a (batch of) matrices, with the same batch\n shape as `sparse_or_dense_a`. The shape must be compatible with the shape\n of `sparse_or_dense_a` and kwargs.\n validate_args: When `True`, additional assertions might be embedded in the\n graph.\n Default value: `False` (i.e., no graph assertions are added).\n name: Python `str` prefixed to ops created by this function.\n Default value: \"sparse_or_dense_matmul\".\n **kwargs: Keyword arguments to `tf.sparse_tensor_dense_matmul` or\n `tf.matmul`.\n\n Returns:\n product: A dense (batch of) matrix-shaped Tensor of the same batch shape and\n dtype as `sparse_or_dense_a` and `dense_b`. If `sparse_or_dense_a` or\n `dense_b` is adjointed through `kwargs` then the shape is adjusted\n accordingly.\n \"\"\"\n with tf.compat.v1.name_scope(name, 'sparse_or_dense_matmul',\n [sparse_or_dense_a, dense_b]):\n dense_b = tf.convert_to_tensor(\n value=dense_b, dtype_hint=tf.float32, name='dense_b')\n\n if validate_args:\n assert_a_rank_at_least_2 = tf.compat.v1.assert_rank_at_least(\n sparse_or_dense_a,\n rank=2,\n message='Input `sparse_or_dense_a` must have at least 2 dimensions.')\n assert_b_rank_at_least_2 = tf.compat.v1.assert_rank_at_least(\n dense_b,\n rank=2,\n message='Input `dense_b` must have at least 2 dimensions.')\n with tf.control_dependencies(\n [assert_a_rank_at_least_2, assert_b_rank_at_least_2]):\n sparse_or_dense_a = tf.identity(sparse_or_dense_a)\n dense_b = tf.identity(dense_b)\n\n if isinstance(sparse_or_dense_a,\n (tf.SparseTensor, tf.compat.v1.SparseTensorValue)):\n return _sparse_tensor_dense_matmul(sparse_or_dense_a, dense_b, **kwargs)\n else:\n return tf.matmul(sparse_or_dense_a, dense_b, **kwargs)"} {"SOURCE": "codesearchnet", "instruction": "How would you implement a function in Python 3 that\nreturns a Tensor of the same shape as sparse_or_dense_matrix and dense_vector.", "response": "def sparse_or_dense_matvecmul(sparse_or_dense_matrix,\n dense_vector,\n validate_args=False,\n name=None,\n **kwargs):\n \"\"\"Returns (batched) matmul of a (sparse) matrix with a column vector.\n\n Args:\n sparse_or_dense_matrix: `SparseTensor` or `Tensor` representing a (batch of)\n matrices.\n dense_vector: `Tensor` representing a (batch of) vectors, with the same\n batch shape as `sparse_or_dense_matrix`. The shape must be compatible with\n the shape of `sparse_or_dense_matrix` and kwargs.\n validate_args: When `True`, additional assertions might be embedded in the\n graph.\n Default value: `False` (i.e., no graph assertions are added).\n name: Python `str` prefixed to ops created by this function.\n Default value: \"sparse_or_dense_matvecmul\".\n **kwargs: Keyword arguments to `tf.sparse_tensor_dense_matmul` or\n `tf.matmul`.\n\n Returns:\n product: A dense (batch of) vector-shaped Tensor of the same batch shape and\n dtype as `sparse_or_dense_matrix` and `dense_vector`.\n \"\"\"\n with tf.compat.v1.name_scope(name, 'sparse_or_dense_matvecmul',\n [sparse_or_dense_matrix, dense_vector]):\n dense_vector = tf.convert_to_tensor(\n value=dense_vector, dtype_hint=tf.float32, name='dense_vector')\n return tf.squeeze(\n sparse_or_dense_matmul(\n sparse_or_dense_matrix,\n dense_vector[..., tf.newaxis],\n validate_args=validate_args,\n **kwargs),\n axis=[-1])"} {"SOURCE": "codesearchnet", "instruction": "Write a Python 3 function that can\nreturn a dense matrix - shaped Tensor with a Tensor.", "response": "def _sparse_tensor_dense_matmul(sp_a, b, **kwargs):\n \"\"\"Returns (batched) matmul of a SparseTensor with a Tensor.\n\n Args:\n sp_a: `SparseTensor` representing a (batch of) matrices.\n b: `Tensor` representing a (batch of) matrices, with the same batch shape of\n `sp_a`. The shape must be compatible with the shape of `sp_a` and kwargs.\n **kwargs: Keyword arguments to `tf.sparse_tensor_dense_matmul`.\n\n Returns:\n product: A dense (batch of) matrix-shaped Tensor of the same batch shape and\n dtype as `sp_a` and `b`. If `sp_a` or `b` is adjointed through `kwargs` then\n the shape is adjusted accordingly.\n \"\"\"\n batch_shape = _get_shape(sp_a)[:-2]\n\n # Reshape the SparseTensor into a rank 3 SparseTensors, with the\n # batch shape flattened to a single dimension. If the batch rank is 0, then\n # we add a batch dimension of rank 1.\n sp_a = tf.sparse.reshape(sp_a, tf.concat([[-1], _get_shape(sp_a)[-2:]],\n axis=0))\n # Reshape b to stack the batch dimension along the rows.\n b = tf.reshape(b, tf.concat([[-1], _get_shape(b)[-1:]], axis=0))\n\n # Convert the SparseTensor to a matrix in block diagonal form with blocks of\n # matrices [M, N]. This allow us to use tf.sparse_tensor_dense_matmul which\n # only accepts rank 2 (Sparse)Tensors.\n out = tf.sparse.sparse_dense_matmul(_sparse_block_diag(sp_a), b, **kwargs)\n\n # Finally retrieve the original batch shape from the resulting rank 2 Tensor.\n # Note that we avoid inferring the final shape from `sp_a` or `b` because we\n # might have transposed one or both of them.\n return tf.reshape(\n out,\n tf.concat([batch_shape, [-1], _get_shape(out)[-1:]], axis=0))"} {"SOURCE": "codesearchnet", "instruction": "Implement a function in Python 3 to\nreturn a SparseTensor from a batch of SparseTensors.", "response": "def _sparse_block_diag(sp_a):\n \"\"\"Returns a block diagonal rank 2 SparseTensor from a batch of SparseTensors.\n\n Args:\n sp_a: A rank 3 `SparseTensor` representing a batch of matrices.\n\n Returns:\n sp_block_diag_a: matrix-shaped, `float` `SparseTensor` with the same dtype\n as `sparse_or_matrix`, of shape [B * M, B * N] where `sp_a` has shape\n [B, M, N]. Each [M, N] batch of `sp_a` is lined up along the diagonal.\n \"\"\"\n # Construct the matrix [[M, N], [1, 0], [0, 1]] which would map the index\n # (b, i, j) to (Mb + i, Nb + j). This effectively creates a block-diagonal\n # matrix of dense shape [B * M, B * N].\n # Note that this transformation doesn't increase the number of non-zero\n # entries in the SparseTensor.\n sp_a_shape = tf.convert_to_tensor(value=_get_shape(sp_a, tf.int64))\n ind_mat = tf.concat([[sp_a_shape[-2:]], tf.eye(2, dtype=tf.int64)], axis=0)\n indices = tf.matmul(sp_a.indices, ind_mat)\n dense_shape = sp_a_shape[0] * sp_a_shape[1:]\n return tf.SparseTensor(\n indices=indices, values=sp_a.values, dense_shape=dense_shape)"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef _maybe_validate_matrix(a, validate_args):\n assertions = []\n if not a.dtype.is_floating:\n raise TypeError('Input `a` must have `float`-like `dtype` '\n '(saw {}).'.format(a.dtype.name))\n if a.shape.ndims is not None:\n if a.shape.ndims < 2:\n raise ValueError('Input `a` must have at least 2 dimensions '\n '(saw: {}).'.format(a.shape.ndims))\n elif validate_args:\n assertions.append(tf.compat.v1.assert_rank_at_least(\n a, rank=2, message='Input `a` must have at least 2 dimensions.'))\n return assertions", "response": "Checks that input is a float matrix."} {"SOURCE": "codesearchnet", "instruction": "Can you generate the documentation for the following Python 3 function\ndef _grad_neg_log_likelihood_and_fim(model_matrix, linear_response, response,\n model):\n \"\"\"Computes the neg-log-likelihood gradient and Fisher information for a GLM.\n\n Note that Fisher information is related to the Hessian of the log-likelihood\n by the equation\n\n ```none\n FisherInfo = E[Hessian with respect to model_coefficients of -LogLikelihood(\n Y | model_matrix, model_coefficients)]\n ```\n\n where `LogLikelihood` is the log-likelihood of a generalized linear model\n parameterized by `model_matrix` and `model_coefficients`, and the expectation\n is taken over Y, distributed according to the same GLM with the same parameter\n values.\n\n Args:\n model_matrix: (Batch of) matrix-shaped, `float` `Tensor` or `SparseTensor`\n where each row represents a sample's features. Has shape `[N, n]` where\n `N` is the number of data samples and `n` is the number of features per\n sample.\n linear_response: (Batch of) vector-shaped `Tensor` with the same dtype as\n `model_matrix`, equal to `model_matix @ model_coefficients` where\n `model_coefficients` are the coefficients of the linear component of the\n GLM.\n response: (Batch of) vector-shaped `Tensor` with the same dtype as\n `model_matrix` where each element represents a sample's observed response\n (to the corresponding row of features).\n model: `tfp.glm.ExponentialFamily`-like instance, which specifies the link\n function and distribution of the GLM, and thus characterizes the negative\n log-likelihood. Must have sufficient statistic equal to the response, that\n is, `T(y) = y`.\n\n Returns:\n grad_neg_log_likelihood: (Batch of) vector-shaped `Tensor` with the same\n shape and dtype as a single row of `model_matrix`, representing the\n gradient of the negative log likelihood of `response` given linear\n response `linear_response`.\n fim_middle: (Batch of) vector-shaped `Tensor` with the same shape and dtype\n as a single column of `model_matrix`, satisfying the equation\n `Fisher information =\n Transpose(model_matrix)\n @ diag(fim_middle)\n @ model_matrix`.\n \"\"\"\n # TODO(b/111926503): Determine whether there are some practical cases where it\n # is computationally favorable to compute the full FIM.\n mean, variance, grad_mean = model(linear_response)\n\n is_valid = (\n tf.math.is_finite(grad_mean) & tf.not_equal(grad_mean, 0.)\n & tf.math.is_finite(variance) & (variance > 0.))\n\n def _mask_if_invalid(x, mask):\n mask = tf.fill(\n tf.shape(input=x), value=np.array(mask, x.dtype.as_numpy_dtype))\n return tf.where(is_valid, x, mask)\n\n # TODO(b/111923449): Link to derivation once it's available.\n v = (response - mean) * _mask_if_invalid(grad_mean, 1) / _mask_if_invalid(\n variance, np.inf)\n grad_log_likelihood = sparse_or_dense_matvecmul(\n model_matrix, v, adjoint_a=True)\n fim_middle = _mask_if_invalid(grad_mean, 0.)**2 / _mask_if_invalid(\n variance, np.inf)\n return -grad_log_likelihood, fim_middle", "response": "Returns the gradient of the neg - log - likelihood of a generalized linear model and the Fisher information for a GLM."} {"SOURCE": "codesearchnet", "instruction": "Can you create a Python 3 function that\ngenerates the slices for building an autoregressive mask.", "response": "def _gen_slices(num_blocks, n_in, n_out, mask_type=MASK_EXCLUSIVE):\n \"\"\"Generate the slices for building an autoregressive mask.\"\"\"\n # TODO(b/67594795): Better support of dynamic shape.\n slices = []\n col = 0\n d_in = n_in // num_blocks\n d_out = n_out // num_blocks\n row = d_out if mask_type == MASK_EXCLUSIVE else 0\n for _ in range(num_blocks):\n row_slice = slice(row, None)\n col_slice = slice(col, col + d_in)\n slices.append([row_slice, col_slice])\n col += d_in\n row += d_out\n return slices"} {"SOURCE": "codesearchnet", "instruction": "Given the following Python 3 function, write the documentation\ndef _gen_mask(num_blocks,\n n_in,\n n_out,\n mask_type=MASK_EXCLUSIVE,\n dtype=tf.float32):\n \"\"\"Generate the mask for building an autoregressive dense layer.\"\"\"\n # TODO(b/67594795): Better support of dynamic shape.\n mask = np.zeros([n_out, n_in], dtype=dtype.as_numpy_dtype())\n slices = _gen_slices(num_blocks, n_in, n_out, mask_type=mask_type)\n for [row_slice, col_slice] in slices:\n mask[row_slice, col_slice] = 1\n return mask", "response": "Generate the mask for building an autoregressive dense layer."} {"SOURCE": "codesearchnet", "instruction": "Implement a Python 3 function for\nbuilding the Masked Autoregressive Density Estimator (Germain et al., 2015). This will be wrapped in a make_template to ensure the variables are only created once. It takes the input and returns the `loc` (\"mu\" in [Germain et al. (2015)][1]) and `log_scale` (\"alpha\" in [Germain et al. (2015)][1]) from the MADE network. Warning: This function uses `masked_dense` to create randomly initialized `tf.Variables`. It is presumed that these will be fit, just as you would any other neural architecture which uses `tf.layers.dense`. #### About Hidden Layers Each element of `hidden_layers` should be greater than the `input_depth` (i.e., `input_depth = tf.shape(input)[-1]` where `input` is the input to the neural network). This is necessary to ensure the autoregressivity property. #### About Clipping This function also optionally clips the `log_scale` (but possibly not its gradient). This is useful because if `log_scale` is too small/large it might underflow/overflow making it impossible for the `MaskedAutoregressiveFlow` bijector to implement a bijection. Additionally, the `log_scale_clip_gradient` `bool` indicates whether the gradient should also be clipped. The default does not clip the gradient; this is useful because it still provides gradient information (for fitting) yet solves the numerical stability problem. I.e., `log_scale_clip_gradient = False` means `grad[exp(clip(x))] = grad[x] exp(clip(x))` rather than the usual `grad[clip(x)] exp(clip(x))`. Args: hidden_layers: Python `list`-like of non-negative integer, scalars indicating the number of units in each hidden layer. Default: `[512, 512]. shift_only: Python `bool` indicating if only the `shift` term shall be computed. Default: `False`. activation: Activation function (callable). Explicitly setting to `None` implies a linear activation. log_scale_min_clip: `float`-like scalar `Tensor`, or a `Tensor` with the same shape as `log_scale`. The minimum value to clip by. Default: -5. log_scale_max_clip: `float`-like scalar `Tensor`, or a `Tensor` with the same shape as `log_scale`. The maximum value to clip by. Default: 3. log_scale_clip_gradient: Python `bool` indicating that the gradient of `tf.clip_by_value` should be preserved. Default: `False`. name: A name for ops managed by this function. Default: \"masked_autoregressive_default_template\". *args: `tf.layers.dense` arguments. **kwargs: `tf.layers.dense` keyword arguments. Returns: shift: `Float`-like `Tensor` of shift terms (the \"mu\" in [Germain et al. (2015)][1]). log_scale: `Float`-like `Tensor` of log(scale) terms (the \"alpha\" in [Germain et al. (2015)][1]). Raises: NotImplementedError: if rightmost dimension of `inputs` is unknown prior to graph execution. #### References [1]: Mathieu Germain, Karol Gregor, Iain Murray, and Hugo Larochelle. MADE: Masked Autoencoder for Distribution Estimation. In _International Conference on Machine Learning_, 2015. https://arxiv.org/abs/1502.03509", "response": "def masked_autoregressive_default_template(hidden_layers,\n shift_only=False,\n activation=tf.nn.relu,\n log_scale_min_clip=-5.,\n log_scale_max_clip=3.,\n log_scale_clip_gradient=False,\n name=None,\n *args, # pylint: disable=keyword-arg-before-vararg\n **kwargs):\n \"\"\"Build the Masked Autoregressive Density Estimator (Germain et al., 2015).\n\n This will be wrapped in a make_template to ensure the variables are only\n created once. It takes the input and returns the `loc` (\"mu\" in [Germain et\n al. (2015)][1]) and `log_scale` (\"alpha\" in [Germain et al. (2015)][1]) from\n the MADE network.\n\n Warning: This function uses `masked_dense` to create randomly initialized\n `tf.Variables`. It is presumed that these will be fit, just as you would any\n other neural architecture which uses `tf.layers.dense`.\n\n #### About Hidden Layers\n\n Each element of `hidden_layers` should be greater than the `input_depth`\n (i.e., `input_depth = tf.shape(input)[-1]` where `input` is the input to the\n neural network). This is necessary to ensure the autoregressivity property.\n\n #### About Clipping\n\n This function also optionally clips the `log_scale` (but possibly not its\n gradient). This is useful because if `log_scale` is too small/large it might\n underflow/overflow making it impossible for the `MaskedAutoregressiveFlow`\n bijector to implement a bijection. Additionally, the `log_scale_clip_gradient`\n `bool` indicates whether the gradient should also be clipped. The default does\n not clip the gradient; this is useful because it still provides gradient\n information (for fitting) yet solves the numerical stability problem. I.e.,\n `log_scale_clip_gradient = False` means\n `grad[exp(clip(x))] = grad[x] exp(clip(x))` rather than the usual\n `grad[clip(x)] exp(clip(x))`.\n\n Args:\n hidden_layers: Python `list`-like of non-negative integer, scalars\n indicating the number of units in each hidden layer. Default: `[512, 512].\n shift_only: Python `bool` indicating if only the `shift` term shall be\n computed. Default: `False`.\n activation: Activation function (callable). Explicitly setting to `None`\n implies a linear activation.\n log_scale_min_clip: `float`-like scalar `Tensor`, or a `Tensor` with the\n same shape as `log_scale`. The minimum value to clip by. Default: -5.\n log_scale_max_clip: `float`-like scalar `Tensor`, or a `Tensor` with the\n same shape as `log_scale`. The maximum value to clip by. Default: 3.\n log_scale_clip_gradient: Python `bool` indicating that the gradient of\n `tf.clip_by_value` should be preserved. Default: `False`.\n name: A name for ops managed by this function. Default:\n \"masked_autoregressive_default_template\".\n *args: `tf.layers.dense` arguments.\n **kwargs: `tf.layers.dense` keyword arguments.\n\n Returns:\n shift: `Float`-like `Tensor` of shift terms (the \"mu\" in\n [Germain et al. (2015)][1]).\n log_scale: `Float`-like `Tensor` of log(scale) terms (the \"alpha\" in\n [Germain et al. (2015)][1]).\n\n Raises:\n NotImplementedError: if rightmost dimension of `inputs` is unknown prior to\n graph execution.\n\n #### References\n\n [1]: Mathieu Germain, Karol Gregor, Iain Murray, and Hugo Larochelle. MADE:\n Masked Autoencoder for Distribution Estimation. In _International\n Conference on Machine Learning_, 2015. https://arxiv.org/abs/1502.03509\n \"\"\"\n name = name or \"masked_autoregressive_default_template\"\n with tf.compat.v2.name_scope(name):\n\n def _fn(x):\n \"\"\"MADE parameterized via `masked_autoregressive_default_template`.\"\"\"\n # TODO(b/67594795): Better support of dynamic shape.\n input_depth = tf.compat.dimension_value(\n tensorshape_util.with_rank_at_least(x.shape, 1)[-1])\n if input_depth is None:\n raise NotImplementedError(\n \"Rightmost dimension must be known prior to graph execution.\")\n input_shape = (\n np.int32(tensorshape_util.as_list(x.shape))\n if tensorshape_util.is_fully_defined(x.shape) else tf.shape(input=x))\n if tensorshape_util.rank(x.shape) == 1:\n x = x[tf.newaxis, ...]\n for i, units in enumerate(hidden_layers):\n x = masked_dense(\n inputs=x,\n units=units,\n num_blocks=input_depth,\n exclusive=True if i == 0 else False,\n activation=activation,\n *args, # pylint: disable=keyword-arg-before-vararg\n **kwargs)\n x = masked_dense(\n inputs=x,\n units=(1 if shift_only else 2) * input_depth,\n num_blocks=input_depth,\n activation=None,\n *args, # pylint: disable=keyword-arg-before-vararg\n **kwargs)\n if shift_only:\n x = tf.reshape(x, shape=input_shape)\n return x, None\n x = tf.reshape(x, shape=tf.concat([input_shape, [2]], axis=0))\n shift, log_scale = tf.unstack(x, num=2, axis=-1)\n which_clip = (\n tf.clip_by_value\n if log_scale_clip_gradient else clip_by_value_preserve_gradient)\n log_scale = which_clip(log_scale, log_scale_min_clip, log_scale_max_clip)\n return shift, log_scale\n\n return tf.compat.v1.make_template(name, _fn)"} {"SOURCE": "codesearchnet", "instruction": "Can you tell what is the following Python 3 function doing\ndef _create_input_order(input_size, input_order=\"left-to-right\"):\n if isinstance(input_order, six.string_types):\n if input_order == \"left-to-right\":\n return np.arange(start=1, stop=input_size + 1)\n elif input_order == \"right-to-left\":\n return np.arange(start=input_size, stop=0, step=-1)\n elif input_order == \"random\":\n ret = np.arange(start=1, stop=input_size + 1)\n np.random.shuffle(ret)\n return ret\n elif np.all(np.sort(input_order) == np.arange(1, input_size + 1)):\n return np.array(input_order)\n\n raise ValueError(\"Invalid input order: '{}'.\".format(input_order))", "response": "Creates a degree vectors for the input."} {"SOURCE": "codesearchnet", "instruction": "Make a summary of the following Python 3 code\ndef _create_degrees(input_size,\n hidden_units=None,\n input_order=\"left-to-right\",\n hidden_degrees=\"equal\"):\n \"\"\"Returns a list of degree vectors, one for each input and hidden layer.\n\n A unit with degree d can only receive input from units with degree < d. Output\n units always have the same degree as their associated input unit.\n\n Args:\n input_size: Number of inputs.\n hidden_units: list with the number of hidden units per layer. It does not\n include the output layer. Each hidden unit size must be at least the size\n of length (otherwise autoregressivity is not possible).\n input_order: Order of degrees to the input units: 'random', 'left-to-right',\n 'right-to-left', or an array of an explicit order. For example,\n 'left-to-right' builds an autoregressive model\n p(x) = p(x1) p(x2 | x1) ... p(xD | x