Upload 25 files

LICENSE

@@ -0,0 +1,674 @@
+                    GNU GENERAL PUBLIC LICENSE
+                       Version 3, 29 June 2007
+
+ Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+                            Preamble
+
+  The GNU General Public License is a free, copyleft license for
+software and other kinds of works.
+
+  The licenses for most software and other practical works are designed
+to take away your freedom to share and change the works.  By contrast,
+the GNU General Public License is intended to guarantee your freedom to
+share and change all versions of a program--to make sure it remains free
+software for all its users.  We, the Free Software Foundation, use the
+GNU General Public License for most of our software; it applies also to
+any other work released this way by its authors.  You can apply it to
+your programs, too.
+
+  When we speak of free software, we are referring to freedom, not
+price.  Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+them if you wish), that you receive source code or can get it if you
+want it, that you can change the software or use pieces of it in new
+free programs, and that you know you can do these things.
+
+  To protect your rights, we need to prevent others from denying you
+these rights or asking you to surrender the rights.  Therefore, you have
+certain responsibilities if you distribute copies of the software, or if
+you modify it: responsibilities to respect the freedom of others.
+
+  For example, if you distribute copies of such a program, whether
+gratis or for a fee, you must pass on to the recipients the same
+freedoms that you received.  You must make sure that they, too, receive
+or can get the source code.  And you must show them these terms so they
+know their rights.
+
+  Developers that use the GNU GPL protect your rights with two steps:
+(1) assert copyright on the software, and (2) offer you this License
+giving you legal permission to copy, distribute and/or modify it.
+
+  For the developers' and authors' protection, the GPL clearly explains
+that there is no warranty for this free software.  For both users' and
+authors' sake, the GPL requires that modified versions be marked as
+changed, so that their problems will not be attributed erroneously to
+authors of previous versions.
+
+  Some devices are designed to deny users access to install or run
+modified versions of the software inside them, although the manufacturer
+can do so.  This is fundamentally incompatible with the aim of
+protecting users' freedom to change the software.  The systematic
+pattern of such abuse occurs in the area of products for individuals to
+use, which is precisely where it is most unacceptable.  Therefore, we
+have designed this version of the GPL to prohibit the practice for those
+products.  If such problems arise substantially in other domains, we
+stand ready to extend this provision to those domains in future versions
+of the GPL, as needed to protect the freedom of users.
+
+  Finally, every program is threatened constantly by software patents.
+States should not allow patents to restrict development and use of
+software on general-purpose computers, but in those that do, we wish to
+avoid the special danger that patents applied to a free program could
+make it effectively proprietary.  To prevent this, the GPL assures that
+patents cannot be used to render the program non-free.
+
+  The precise terms and conditions for copying, distribution and
+modification follow.
+
+                       TERMS AND CONDITIONS
+
+  0. Definitions.
+
+  "This License" refers to version 3 of the GNU General Public License.
+
+  "Copyright" also means copyright-like laws that apply to other kinds of
+works, such as semiconductor masks.
+
+  "The Program" refers to any copyrightable work licensed under this
+License.  Each licensee is addressed as "you".  "Licensees" and
+"recipients" may be individuals or organizations.
+
+  To "modify" a work means to copy from or adapt all or part of the work
+in a fashion requiring copyright permission, other than the making of an
+exact copy.  The resulting work is called a "modified version" of the
+earlier work or a work "based on" the earlier work.
+
+  A "covered work" means either the unmodified Program or a work based
+on the Program.
+
+  To "propagate" a work means to do anything with it that, without
+permission, would make you directly or secondarily liable for
+infringement under applicable copyright law, except executing it on a
+computer or modifying a private copy.  Propagation includes copying,
+distribution (with or without modification), making available to the
+public, and in some countries other activities as well.
+
+  To "convey" a work means any kind of propagation that enables other
+parties to make or receive copies.  Mere interaction with a user through
+a computer network, with no transfer of a copy, is not conveying.
+
+  An interactive user interface displays "Appropriate Legal Notices"
+to the extent that it includes a convenient and prominently visible
+feature that (1) displays an appropriate copyright notice, and (2)
+tells the user that there is no warranty for the work (except to the
+extent that warranties are provided), that licensees may convey the
+work under this License, and how to view a copy of this License.  If
+the interface presents a list of user commands or options, such as a
+menu, a prominent item in the list meets this criterion.
+
+  1. Source Code.
+
+  The "source code" for a work means the preferred form of the work
+for making modifications to it.  "Object code" means any non-source
+form of a work.
+
+  A "Standard Interface" means an interface that either is an official
+standard defined by a recognized standards body, or, in the case of
+interfaces specified for a particular programming language, one that
+is widely used among developers working in that language.
+
+  The "System Libraries" of an executable work include anything, other
+than the work as a whole, that (a) is included in the normal form of
+packaging a Major Component, but which is not part of that Major
+Component, and (b) serves only to enable use of the work with that
+Major Component, or to implement a Standard Interface for which an
+implementation is available to the public in source code form.  A
+"Major Component", in this context, means a major essential component
+(kernel, window system, and so on) of the specific operating system
+(if any) on which the executable work runs, or a compiler used to
+produce the work, or an object code interpreter used to run it.
+
+  The "Corresponding Source" for a work in object code form means all
+the source code needed to generate, install, and (for an executable
+work) run the object code and to modify the work, including scripts to
+control those activities.  However, it does not include the work's
+System Libraries, or general-purpose tools or generally available free
+programs which are used unmodified in performing those activities but
+which are not part of the work.  For example, Corresponding Source
+includes interface definition files associated with source files for
+the work, and the source code for shared libraries and dynamically
+linked subprograms that the work is specifically designed to require,
+such as by intimate data communication or control flow between those
+subprograms and other parts of the work.
+
+  The Corresponding Source need not include anything that users
+can regenerate automatically from other parts of the Corresponding
+Source.
+
+  The Corresponding Source for a work in source code form is that
+same work.
+
+  2. Basic Permissions.
+
+  All rights granted under this License are granted for the term of
+copyright on the Program, and are irrevocable provided the stated
+conditions are met.  This License explicitly affirms your unlimited
+permission to run the unmodified Program.  The output from running a
+covered work is covered by this License only if the output, given its
+content, constitutes a covered work.  This License acknowledges your
+rights of fair use or other equivalent, as provided by copyright law.
+
+  You may make, run and propagate covered works that you do not
+convey, without conditions so long as your license otherwise remains
+in force.  You may convey covered works to others for the sole purpose
+of having them make modifications exclusively for you, or provide you
+with facilities for running those works, provided that you comply with
+the terms of this License in conveying all material for which you do
+not control copyright.  Those thus making or running the covered works
+for you must do so exclusively on your behalf, under your direction
+and control, on terms that prohibit them from making any copies of
+your copyrighted material outside their relationship with you.
+
+  Conveying under any other circumstances is permitted solely under
+the conditions stated below.  Sublicensing is not allowed; section 10
+makes it unnecessary.
+
+  3. Protecting Users' Legal Rights From Anti-Circumvention Law.
+
+  No covered work shall be deemed part of an effective technological
+measure under any applicable law fulfilling obligations under article
+11 of the WIPO copyright treaty adopted on 20 December 1996, or
+similar laws prohibiting or restricting circumvention of such
+measures.
+
+  When you convey a covered work, you waive any legal power to forbid
+circumvention of technological measures to the extent such circumvention
+is effected by exercising rights under this License with respect to
+the covered work, and you disclaim any intention to limit operation or
+modification of the work as a means of enforcing, against the work's
+users, your or third parties' legal rights to forbid circumvention of
+technological measures.
+
+  4. Conveying Verbatim Copies.
+
+  You may convey verbatim copies of the Program's source code as you
+receive it, in any medium, provided that you conspicuously and
+appropriately publish on each copy an appropriate copyright notice;
+keep intact all notices stating that this License and any
+non-permissive terms added in accord with section 7 apply to the code;
+keep intact all notices of the absence of any warranty; and give all
+recipients a copy of this License along with the Program.
+
+  You may charge any price or no price for each copy that you convey,
+and you may offer support or warranty protection for a fee.
+
+  5. Conveying Modified Source Versions.
+
+  You may convey a work based on the Program, or the modifications to
+produce it from the Program, in the form of source code under the
+terms of section 4, provided that you also meet all of these conditions:
+
+    a) The work must carry prominent notices stating that you modified
+    it, and giving a relevant date.
+
+    b) The work must carry prominent notices stating that it is
+    released under this License and any conditions added under section
+    7.  This requirement modifies the requirement in section 4 to
+    "keep intact all notices".
+
+    c) You must license the entire work, as a whole, under this
+    License to anyone who comes into possession of a copy.  This
+    License will therefore apply, along with any applicable section 7
+    additional terms, to the whole of the work, and all its parts,
+    regardless of how they are packaged.  This License gives no
+    permission to license the work in any other way, but it does not
+    invalidate such permission if you have separately received it.
+
+    d) If the work has interactive user interfaces, each must display
+    Appropriate Legal Notices; however, if the Program has interactive
+    interfaces that do not display Appropriate Legal Notices, your
+    work need not make them do so.
+
+  A compilation of a covered work with other separate and independent
+works, which are not by their nature extensions of the covered work,
+and which are not combined with it such as to form a larger program,
+in or on a volume of a storage or distribution medium, is called an
+"aggregate" if the compilation and its resulting copyright are not
+used to limit the access or legal rights of the compilation's users
+beyond what the individual works permit.  Inclusion of a covered work
+in an aggregate does not cause this License to apply to the other
+parts of the aggregate.
+
+  6. Conveying Non-Source Forms.
+
+  You may convey a covered work in object code form under the terms
+of sections 4 and 5, provided that you also convey the
+machine-readable Corresponding Source under the terms of this License,
+in one of these ways:
+
+    a) Convey the object code in, or embodied in, a physical product
+    (including a physical distribution medium), accompanied by the
+    Corresponding Source fixed on a durable physical medium
+    customarily used for software interchange.
+
+    b) Convey the object code in, or embodied in, a physical product
+    (including a physical distribution medium), accompanied by a
+    written offer, valid for at least three years and valid for as
+    long as you offer spare parts or customer support for that product
+    model, to give anyone who possesses the object code either (1) a
+    copy of the Corresponding Source for all the software in the
+    product that is covered by this License, on a durable physical
+    medium customarily used for software interchange, for a price no
+    more than your reasonable cost of physically performing this
+    conveying of source, or (2) access to copy the
+    Corresponding Source from a network server at no charge.
+
+    c) Convey individual copies of the object code with a copy of the
+    written offer to provide the Corresponding Source.  This
+    alternative is allowed only occasionally and noncommercially, and
+    only if you received the object code with such an offer, in accord
+    with subsection 6b.
+
+    d) Convey the object code by offering access from a designated
+    place (gratis or for a charge), and offer equivalent access to the
+    Corresponding Source in the same way through the same place at no
+    further charge.  You need not require recipients to copy the
+    Corresponding Source along with the object code.  If the place to
+    copy the object code is a network server, the Corresponding Source
+    may be on a different server (operated by you or a third party)
+    that supports equivalent copying facilities, provided you maintain
+    clear directions next to the object code saying where to find the
+    Corresponding Source.  Regardless of what server hosts the
+    Corresponding Source, you remain obligated to ensure that it is
+    available for as long as needed to satisfy these requirements.
+
+    e) Convey the object code using peer-to-peer transmission, provided
+    you inform other peers where the object code and Corresponding
+    Source of the work are being offered to the general public at no
+    charge under subsection 6d.
+
+  A separable portion of the object code, whose source code is excluded
+from the Corresponding Source as a System Library, need not be
+included in conveying the object code work.
+
+  A "User Product" is either (1) a "consumer product", which means any
+tangible personal property which is normally used for personal, family,
+or household purposes, or (2) anything designed or sold for incorporation
+into a dwelling.  In determining whether a product is a consumer product,
+doubtful cases shall be resolved in favor of coverage.  For a particular
+product received by a particular user, "normally used" refers to a
+typical or common use of that class of product, regardless of the status
+of the particular user or of the way in which the particular user
+actually uses, or expects or is expected to use, the product.  A product
+is a consumer product regardless of whether the product has substantial
+commercial, industrial or non-consumer uses, unless such uses represent
+the only significant mode of use of the product.
+
+  "Installation Information" for a User Product means any methods,
+procedures, authorization keys, or other information required to install
+and execute modified versions of a covered work in that User Product from
+a modified version of its Corresponding Source.  The information must
+suffice to ensure that the continued functioning of the modified object
+code is in no case prevented or interfered with solely because
+modification has been made.
+
+  If you convey an object code work under this section in, or with, or
+specifically for use in, a User Product, and the conveying occurs as
+part of a transaction in which the right of possession and use of the
+User Product is transferred to the recipient in perpetuity or for a
+fixed term (regardless of how the transaction is characterized), the
+Corresponding Source conveyed under this section must be accompanied
+by the Installation Information.  But this requirement does not apply
+if neither you nor any third party retains the ability to install
+modified object code on the User Product (for example, the work has
+been installed in ROM).
+
+  The requirement to provide Installation Information does not include a
+requirement to continue to provide support service, warranty, or updates
+for a work that has been modified or installed by the recipient, or for
+the User Product in which it has been modified or installed.  Access to a
+network may be denied when the modification itself materially and
+adversely affects the operation of the network or violates the rules and
+protocols for communication across the network.
+
+  Corresponding Source conveyed, and Installation Information provided,
+in accord with this section must be in a format that is publicly
+documented (and with an implementation available to the public in
+source code form), and must require no special password or key for
+unpacking, reading or copying.
+
+  7. Additional Terms.
+
+  "Additional permissions" are terms that supplement the terms of this
+License by making exceptions from one or more of its conditions.
+Additional permissions that are applicable to the entire Program shall
+be treated as though they were included in this License, to the extent
+that they are valid under applicable law.  If additional permissions
+apply only to part of the Program, that part may be used separately
+under those permissions, but the entire Program remains governed by
+this License without regard to the additional permissions.
+
+  When you convey a copy of a covered work, you may at your option
+remove any additional permissions from that copy, or from any part of
+it.  (Additional permissions may be written to require their own
+removal in certain cases when you modify the work.)  You may place
+additional permissions on material, added by you to a covered work,
+for which you have or can give appropriate copyright permission.
+
+  Notwithstanding any other provision of this License, for material you
+add to a covered work, you may (if authorized by the copyright holders of
+that material) supplement the terms of this License with terms:
+
+    a) Disclaiming warranty or limiting liability differently from the
+    terms of sections 15 and 16 of this License; or
+
+    b) Requiring preservation of specified reasonable legal notices or
+    author attributions in that material or in the Appropriate Legal
+    Notices displayed by works containing it; or
+
+    c) Prohibiting misrepresentation of the origin of that material, or
+    requiring that modified versions of such material be marked in
+    reasonable ways as different from the original version; or
+
+    d) Limiting the use for publicity purposes of names of licensors or
+    authors of the material; or
+
+    e) Declining to grant rights under trademark law for use of some
+    trade names, trademarks, or service marks; or
+
+    f) Requiring indemnification of licensors and authors of that
+    material by anyone who conveys the material (or modified versions of
+    it) with contractual assumptions of liability to the recipient, for
+    any liability that these contractual assumptions directly impose on
+    those licensors and authors.
+
+  All other non-permissive additional terms are considered "further
+restrictions" within the meaning of section 10.  If the Program as you
+received it, or any part of it, contains a notice stating that it is
+governed by this License along with a term that is a further
+restriction, you may remove that term.  If a license document contains
+a further restriction but permits relicensing or conveying under this
+License, you may add to a covered work material governed by the terms
+of that license document, provided that the further restriction does
+not survive such relicensing or conveying.
+
+  If you add terms to a covered work in accord with this section, you
+must place, in the relevant source files, a statement of the
+additional terms that apply to those files, or a notice indicating
+where to find the applicable terms.
+
+  Additional terms, permissive or non-permissive, may be stated in the
+form of a separately written license, or stated as exceptions;
+the above requirements apply either way.
+
+  8. Termination.
+
+  You may not propagate or modify a covered work except as expressly
+provided under this License.  Any attempt otherwise to propagate or
+modify it is void, and will automatically terminate your rights under
+this License (including any patent licenses granted under the third
+paragraph of section 11).
+
+  However, if you cease all violation of this License, then your
+license from a particular copyright holder is reinstated (a)
+provisionally, unless and until the copyright holder explicitly and
+finally terminates your license, and (b) permanently, if the copyright
+holder fails to notify you of the violation by some reasonable means
+prior to 60 days after the cessation.
+
+  Moreover, your license from a particular copyright holder is
+reinstated permanently if the copyright holder notifies you of the
+violation by some reasonable means, this is the first time you have
+received notice of violation of this License (for any work) from that
+copyright holder, and you cure the violation prior to 30 days after
+your receipt of the notice.
+
+  Termination of your rights under this section does not terminate the
+licenses of parties who have received copies or rights from you under
+this License.  If your rights have been terminated and not permanently
+reinstated, you do not qualify to receive new licenses for the same
+material under section 10.
+
+  9. Acceptance Not Required for Having Copies.
+
+  You are not required to accept this License in order to receive or
+run a copy of the Program.  Ancillary propagation of a covered work
+occurring solely as a consequence of using peer-to-peer transmission
+to receive a copy likewise does not require acceptance.  However,
+nothing other than this License grants you permission to propagate or
+modify any covered work.  These actions infringe copyright if you do
+not accept this License.  Therefore, by modifying or propagating a
+covered work, you indicate your acceptance of this License to do so.
+
+  10. Automatic Licensing of Downstream Recipients.
+
+  Each time you convey a covered work, the recipient automatically
+receives a license from the original licensors, to run, modify and
+propagate that work, subject to this License.  You are not responsible
+for enforcing compliance by third parties with this License.
+
+  An "entity transaction" is a transaction transferring control of an
+organization, or substantially all assets of one, or subdividing an
+organization, or merging organizations.  If propagation of a covered
+work results from an entity transaction, each party to that
+transaction who receives a copy of the work also receives whatever
+licenses to the work the party's predecessor in interest had or could
+give under the previous paragraph, plus a right to possession of the
+Corresponding Source of the work from the predecessor in interest, if
+the predecessor has it or can get it with reasonable efforts.
+
+  You may not impose any further restrictions on the exercise of the
+rights granted or affirmed under this License.  For example, you may
+not impose a license fee, royalty, or other charge for exercise of
+rights granted under this License, and you may not initiate litigation
+(including a cross-claim or counterclaim in a lawsuit) alleging that
+any patent claim is infringed by making, using, selling, offering for
+sale, or importing the Program or any portion of it.
+
+  11. Patents.
+
+  A "contributor" is a copyright holder who authorizes use under this
+License of the Program or a work on which the Program is based.  The
+work thus licensed is called the contributor's "contributor version".
+
+  A contributor's "essential patent claims" are all patent claims
+owned or controlled by the contributor, whether already acquired or
+hereafter acquired, that would be infringed by some manner, permitted
+by this License, of making, using, or selling its contributor version,
+but do not include claims that would be infringed only as a
+consequence of further modification of the contributor version.  For
+purposes of this definition, "control" includes the right to grant
+patent sublicenses in a manner consistent with the requirements of
+this License.
+
+  Each contributor grants you a non-exclusive, worldwide, royalty-free
+patent license under the contributor's essential patent claims, to
+make, use, sell, offer for sale, import and otherwise run, modify and
+propagate the contents of its contributor version.
+
+  In the following three paragraphs, a "patent license" is any express
+agreement or commitment, however denominated, not to enforce a patent
+(such as an express permission to practice a patent or covenant not to
+sue for patent infringement).  To "grant" such a patent license to a
+party means to make such an agreement or commitment not to enforce a
+patent against the party.
+
+  If you convey a covered work, knowingly relying on a patent license,
+and the Corresponding Source of the work is not available for anyone
+to copy, free of charge and under the terms of this License, through a
+publicly available network server or other readily accessible means,
+then you must either (1) cause the Corresponding Source to be so
+available, or (2) arrange to deprive yourself of the benefit of the
+patent license for this particular work, or (3) arrange, in a manner
+consistent with the requirements of this License, to extend the patent
+license to downstream recipients.  "Knowingly relying" means you have
+actual knowledge that, but for the patent license, your conveying the
+covered work in a country, or your recipient's use of the covered work
+in a country, would infringe one or more identifiable patents in that
+country that you have reason to believe are valid.
+
+  If, pursuant to or in connection with a single transaction or
+arrangement, you convey, or propagate by procuring conveyance of, a
+covered work, and grant a patent license to some of the parties
+receiving the covered work authorizing them to use, propagate, modify
+or convey a specific copy of the covered work, then the patent license
+you grant is automatically extended to all recipients of the covered
+work and works based on it.
+
+  A patent license is "discriminatory" if it does not include within
+the scope of its coverage, prohibits the exercise of, or is
+conditioned on the non-exercise of one or more of the rights that are
+specifically granted under this License.  You may not convey a covered
+work if you are a party to an arrangement with a third party that is
+in the business of distributing software, under which you make payment
+to the third party based on the extent of your activity of conveying
+the work, and under which the third party grants, to any of the
+parties who would receive the covered work from you, a discriminatory
+patent license (a) in connection with copies of the covered work
+conveyed by you (or copies made from those copies), or (b) primarily
+for and in connection with specific products or compilations that
+contain the covered work, unless you entered into that arrangement,
+or that patent license was granted, prior to 28 March 2007.
+
+  Nothing in this License shall be construed as excluding or limiting
+any implied license or other defenses to infringement that may
+otherwise be available to you under applicable patent law.
+
+  12. No Surrender of Others' Freedom.
+
+  If conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License.  If you cannot convey a
+covered work so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you may
+not convey it at all.  For example, if you agree to terms that obligate you
+to collect a royalty for further conveying from those to whom you convey
+the Program, the only way you could satisfy both those terms and this
+License would be to refrain entirely from conveying the Program.
+
+  13. Use with the GNU Affero General Public License.
+
+  Notwithstanding any other provision of this License, you have
+permission to link or combine any covered work with a work licensed
+under version 3 of the GNU Affero General Public License into a single
+combined work, and to convey the resulting work.  The terms of this
+License will continue to apply to the part which is the covered work,
+but the special requirements of the GNU Affero General Public License,
+section 13, concerning interaction through a network will apply to the
+combination as such.
+
+  14. Revised Versions of this License.
+
+  The Free Software Foundation may publish revised and/or new versions of
+the GNU General Public License from time to time.  Such new versions will
+be similar in spirit to the present version, but may differ in detail to
+address new problems or concerns.
+
+  Each version is given a distinguishing version number.  If the
+Program specifies that a certain numbered version of the GNU General
+Public License "or any later version" applies to it, you have the
+option of following the terms and conditions either of that numbered
+version or of any later version published by the Free Software
+Foundation.  If the Program does not specify a version number of the
+GNU General Public License, you may choose any version ever published
+by the Free Software Foundation.
+
+  If the Program specifies that a proxy can decide which future
+versions of the GNU General Public License can be used, that proxy's
+public statement of acceptance of a version permanently authorizes you
+to choose that version for the Program.
+
+  Later license versions may give you additional or different
+permissions.  However, no additional obligations are imposed on any
+author or copyright holder as a result of your choosing to follow a
+later version.
+
+  15. Disclaimer of Warranty.
+
+  THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
+APPLICABLE LAW.  EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
+HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
+OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
+THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+PURPOSE.  THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
+IS WITH YOU.  SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
+ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
+
+  16. Limitation of Liability.
+
+  IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
+THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
+GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
+USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
+DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
+PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
+EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
+SUCH DAMAGES.
+
+  17. Interpretation of Sections 15 and 16.
+
+  If the disclaimer of warranty and limitation of liability provided
+above cannot be given local legal effect according to their terms,
+reviewing courts shall apply local law that most closely approximates
+an absolute waiver of all civil liability in connection with the
+Program, unless a warranty or assumption of liability accompanies a
+copy of the Program in return for a fee.
+
+                     END OF TERMS AND CONDITIONS
+
+            How to Apply These Terms to Your New Programs
+
+  If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these terms.
+
+  To do so, attach the following notices to the program.  It is safest
+to attach them to the start of each source file to most effectively
+state the exclusion of warranty; and each file should have at least
+the "copyright" line and a pointer to where the full notice is found.
+
+    <one line to give the program's name and a brief idea of what it does.>
+    Copyright (C) <year>  <name of author>
+
+    This program is free software: you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License
+    along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+Also add information on how to contact you by electronic and paper mail.
+
+  If the program does terminal interaction, make it output a short
+notice like this when it starts in an interactive mode:
+
+    <program>  Copyright (C) <year>  <name of author>
+    This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
+    This is free software, and you are welcome to redistribute it
+    under certain conditions; type `show c' for details.
+
+The hypothetical commands `show w' and `show c' should show the appropriate
+parts of the General Public License.  Of course, your program's commands
+might be different; for a GUI interface, you would use an "about box".
+
+  You should also get your employer (if you work as a programmer) or school,
+if any, to sign a "copyright disclaimer" for the program, if necessary.
+For more information on this, and how to apply and follow the GNU GPL, see
+<https://www.gnu.org/licenses/>.
+
+  The GNU General Public License does not permit incorporating your program
+into proprietary programs.  If your program is a subroutine library, you
+may consider it more useful to permit linking proprietary applications with
+the library.  If this is what you want to do, use the GNU Lesser General
+Public License instead of this License.  But first, please read
+<https://www.gnu.org/licenses/why-not-lgpl.html>.

README.md

@@ -1,12 +1,153 @@
+# Deep Floor Plan Recognition using a Multi-task Network with Room-boundary-Guided Attention
+By Zhiliang ZENG, Xianzhi LI, Ying Kin Yu, and Chi-Wing Fu
+
+[2021/07/26: updated download link]
+
+[2019/08/28: updated train/test/score code & dataset]
+
+[2019/07/29: updated demo code & pretrained model]
+
+## Introduction
+
+This repository contains the code & annotation data for our ICCV 2019 paper: ['Deep Floor Plan Recognition Using a Multi-Task Network with Room-Boundary-Guided Attention'](https://arxiv.org/abs/1908.11025). In this paper, we present a new method for recognizing floor plan elements by exploring the spatial relationship between floor plan elements, model a hierarchy of floor plan elements, and design a multi-task network to learn to recognize room-boundary and room-type elements in floor plans.
+
+## Requirements
+
+- Python 3.7+
+- See `requirements.txt` for all dependencies (TensorFlow 1.x, Pillow, imageio, gradio, etc.)
+
+Our code has been tested by using tensorflow-gpu==1.10.1 & OpenCV==3.1.0. We used Nvidia Titan Xp GPU with CUDA 9.0 installed.
+
+## Python packages
+
+- [numpy]
+- [scipy]
+- [Pillow]
+- [matplotlib]
+
+## Data
+
+We share all our annotations and train-test split file [here](https://mycuhk-my.sharepoint.com/:f:/g/personal/1155052510_link_cuhk_edu_hk/EseSIeHQgPxArPlNpGdVp38BIjUg70jMiAO-w4f3s8B_dg?e=UXKbYO). Or download the annotation using the link in file "dataset/download_links.txt". The additional round plan is included in the annotations.
+
+Our annotations are saved as png format. The name with suffixes "\_wall.png", "\_close.png" and "\_room.png" are denoted "wall", "door & window" and "room types" label, respectively. We used these labels to train our multi-task network.
+
+The name with suffixes "\_close_wall.png" is the combination of "wall", "door & window" label. We don't use this label in our paper, but maybe useful for other tasks.
+
+The name with suffixes "\_multi.png" is the combination of all the labels. We used this kind of label to retrain the general segmentation network.
+
+We also provide our training data on R3D dataset in "tfrecord" format, which can improve the loading speed during training.
+
+To create the "tfrecord" training set, please refer to the example code in "utils/create_tfrecord.py"
+
+All the raw floor plan image please refer to the following two links:
+
+- R2V: <https://github.com/art-programmer/FloorplanTransformation.git>
+- R3D: <http://www.cs.toronto.edu/~fidler/projects/rent3D.html>
+
+## Usage
+
+To use our demo code, please first download the pretrained model, find the link in "pretrained/download_links.txt" file, unzip and put it into "pretrained" folder, then run
+
+```bash
+python demo.py --im_path=./demo/45719584.jpg 
+```
+
+To train the network, simply run
+
+```bash
+python main.py --pharse=Train
+```
+
+Run the following command to generate network outputs, all results are saved as png format.
+
+```bash
+python main.py --pharse=Test
+```
+
+To compute the evaluation metrics, please first inference the results, then simply run
+
+```bash
+python scores.py --dataset=R3D
+```
+
+To use our post-processing method, please first inference the results, then simply run
+
+```bash
+python postprocess.py
+```
+
+or 
+
+```bash
+python postprocess.py --result_dir=./[result_folder_path]
+```
+
+## Citation
+
+If you find our work useful in your research, please consider citing:
+
+---
+
+	@InProceedings{zlzeng2019deepfloor,
+		author = {Zhiliang ZENG, Xianzhi LI, Ying Kin Yu, and Chi-Wing Fu},
+		title = {Deep Floor Plan Recognition using a Multi-task Network with Room-boundary-Guided Attention},
+		booktitle = {IEEE International Conference on Computer Vision (ICCV)},
+		year = {2019}
+	}
+
+---
+
+---
+
+## Quick Start with Gradio (Python 3)
+
+1. Install dependencies:
+
+```bash
+pip install -r requirements.txt
+```
+
+2. Download the pretrained model (see `pretrained/download_links.txt`) and place it in the `pretrained` folder.
+
+3. Run the Gradio app:
+
+```bash
+python app.py
+```
+
+This will launch a web interface where you can upload a floorplan image and view the predicted segmentation.
+
 ---
-title: DeepFloorPlan
-emoji: 📈
-colorFrom: purple
-colorTo: green
-sdk: gradio
-sdk_version: 5.36.2
-app_file: app.py
-pinned: false
+
+## Deploy on Hugging Face Spaces
+
+- Upload the repository (with `app.py`, `deepfloorplan_inference.py`, and `requirements.txt`) to your Hugging Face Space.
+- Make sure the `pretrained` model weights are included or downloaded in the Space.
+- The Gradio app will be served automatically.
+
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+## API Usage with Hugging Face Inference Endpoints
+
+- Deploy the repository to a Hugging Face Inference Endpoint.
+- The endpoint will use `api_inference.py` and expose a `predict` function.
+- Example usage (Python):
+
+```python
+import requests
+from PIL import Image
+import io
+
+# Replace with your endpoint URL and token
+API_URL = 'https://api-inference.huggingface.co/models/your-username/your-model'
+headers = {"Authorization": f"Bearer YOUR_HF_TOKEN"}
+
+image = Image.open('your_image.jpg')
+buffer = io.BytesIO()
+image.save(buffer, format='PNG')
+response = requests.post(API_URL, headers=headers, files={"image": buffer.getvalue()})
+result = Image.open(io.BytesIO(response.content))
+result.show()
+```
+
+---

api_inference.py

@@ -0,0 +1,20 @@
+from PIL import Image
+import numpy as np
+from deepfloorplan_inference import DeepFloorPlanModel
+
+class EndpointModel:
+    def __init__(self):
+        self.model = DeepFloorPlanModel(model_dir='pretrained')
+
+    def __call__(self, image):
+        # image: PIL Image or numpy array
+        if isinstance(image, np.ndarray):
+            image = Image.fromarray(image)
+        result = self.model.predict(image)
+        return Image.fromarray(result.astype(np.uint8))
+
+# For Hugging Face Inference Endpoints
+model = EndpointModel()
+
+def predict(image):
+    return model(image) 

app.py

@@ -0,0 +1,25 @@
+import gradio as gr
+import numpy as np
+from PIL import Image
+from deepfloorplan_inference import DeepFloorPlanModel
+
+# Load model once at startup
+model = DeepFloorPlanModel(model_dir='pretrained')
+
+def predict_floorplan(image):
+    # image: PIL Image from Gradio
+    result = model.predict(image)
+    # Convert numpy array to PIL Image for Gradio output
+    return Image.fromarray(result.astype(np.uint8))
+
+iface = gr.Interface(
+    fn=predict_floorplan,
+    inputs=gr.Image(type="pil", label="Upload Floorplan Image"),
+    outputs=gr.Image(type="pil", label="Predicted Segmentation"),
+    title="Deep Floor Plan Segmentation",
+    description="Upload a floorplan image to get the predicted segmentation using the Deep Floor Plan model.",
+    allow_flagging="never"
+)
+
+if __name__ == "__main__":
+    iface.launch() 

dataset/download_links.txt

@@ -0,0 +1 @@
+https://mycuhk-my.sharepoint.com/:f:/g/personal/1155052510_link_cuhk_edu_hk/EseSIeHQgPxArPlNpGdVp38BIjUg70jMiAO-w4f3s8B_dg?e=UXKbYO

dataset/newyork

@@ -0,0 +1 @@
+/home/zlzeng/floorplan_v2/dataset/newyork

dataset/r2v_test.txt

@@ -0,0 +1,100 @@
+../dataset/jp/test/100_input.jpg	../dataset/jp/test/100_wall.png	../dataset/jp/test/100_close.png	../dataset/jp/test/100_rooms.png	../dataset/jp/test/100_close_wall.png
+../dataset/jp/test/10_input.jpg	../dataset/jp/test/10_wall.png	../dataset/jp/test/10_close.png	../dataset/jp/test/10_rooms.png	../dataset/jp/test/10_close_wall.png
+../dataset/jp/test/11_input.jpg	../dataset/jp/test/11_wall.png	../dataset/jp/test/11_close.png	../dataset/jp/test/11_rooms.png	../dataset/jp/test/11_close_wall.png
+../dataset/jp/test/12_input.jpg	../dataset/jp/test/12_wall.png	../dataset/jp/test/12_close.png	../dataset/jp/test/12_rooms.png	../dataset/jp/test/12_close_wall.png
+../dataset/jp/test/13_input.jpg	../dataset/jp/test/13_wall.png	../dataset/jp/test/13_close.png	../dataset/jp/test/13_rooms.png	../dataset/jp/test/13_close_wall.png
+../dataset/jp/test/14_input.jpg	../dataset/jp/test/14_wall.png	../dataset/jp/test/14_close.png	../dataset/jp/test/14_rooms.png	../dataset/jp/test/14_close_wall.png
+../dataset/jp/test/15_input.jpg	../dataset/jp/test/15_wall.png	../dataset/jp/test/15_close.png	../dataset/jp/test/15_rooms.png	../dataset/jp/test/15_close_wall.png
+../dataset/jp/test/16_input.jpg	../dataset/jp/test/16_wall.png	../dataset/jp/test/16_close.png	../dataset/jp/test/16_rooms.png	../dataset/jp/test/16_close_wall.png
+../dataset/jp/test/17_input.jpg	../dataset/jp/test/17_wall.png	../dataset/jp/test/17_close.png	../dataset/jp/test/17_rooms.png	../dataset/jp/test/17_close_wall.png
+../dataset/jp/test/18_input.jpg	../dataset/jp/test/18_wall.png	../dataset/jp/test/18_close.png	../dataset/jp/test/18_rooms.png	../dataset/jp/test/18_close_wall.png
+../dataset/jp/test/19_input.jpg	../dataset/jp/test/19_wall.png	../dataset/jp/test/19_close.png	../dataset/jp/test/19_rooms.png	../dataset/jp/test/19_close_wall.png
+../dataset/jp/test/1_input.jpg	../dataset/jp/test/1_wall.png	../dataset/jp/test/1_close.png	../dataset/jp/test/1_rooms.png	../dataset/jp/test/1_close_wall.png
+../dataset/jp/test/20_input.jpg	../dataset/jp/test/20_wall.png	../dataset/jp/test/20_close.png	../dataset/jp/test/20_rooms.png	../dataset/jp/test/20_close_wall.png
+../dataset/jp/test/21_input.jpg	../dataset/jp/test/21_wall.png	../dataset/jp/test/21_close.png	../dataset/jp/test/21_rooms.png	../dataset/jp/test/21_close_wall.png
+../dataset/jp/test/22_input.jpg	../dataset/jp/test/22_wall.png	../dataset/jp/test/22_close.png	../dataset/jp/test/22_rooms.png	../dataset/jp/test/22_close_wall.png
+../dataset/jp/test/23_input.jpg	../dataset/jp/test/23_wall.png	../dataset/jp/test/23_close.png	../dataset/jp/test/23_rooms.png	../dataset/jp/test/23_close_wall.png
+../dataset/jp/test/24_input.jpg	../dataset/jp/test/24_wall.png	../dataset/jp/test/24_close.png	../dataset/jp/test/24_rooms.png	../dataset/jp/test/24_close_wall.png
+../dataset/jp/test/25_input.jpg	../dataset/jp/test/25_wall.png	../dataset/jp/test/25_close.png	../dataset/jp/test/25_rooms.png	../dataset/jp/test/25_close_wall.png
+../dataset/jp/test/26_input.jpg	../dataset/jp/test/26_wall.png	../dataset/jp/test/26_close.png	../dataset/jp/test/26_rooms.png	../dataset/jp/test/26_close_wall.png
+../dataset/jp/test/27_input.jpg	../dataset/jp/test/27_wall.png	../dataset/jp/test/27_close.png	../dataset/jp/test/27_rooms.png	../dataset/jp/test/27_close_wall.png
+../dataset/jp/test/28_input.jpg	../dataset/jp/test/28_wall.png	../dataset/jp/test/28_close.png	../dataset/jp/test/28_rooms.png	../dataset/jp/test/28_close_wall.png
+../dataset/jp/test/29_input.jpg	../dataset/jp/test/29_wall.png	../dataset/jp/test/29_close.png	../dataset/jp/test/29_rooms.png	../dataset/jp/test/29_close_wall.png
+../dataset/jp/test/2_input.jpg	../dataset/jp/test/2_wall.png	../dataset/jp/test/2_close.png	../dataset/jp/test/2_rooms.png	../dataset/jp/test/2_close_wall.png
+../dataset/jp/test/30_input.jpg	../dataset/jp/test/30_wall.png	../dataset/jp/test/30_close.png	../dataset/jp/test/30_rooms.png	../dataset/jp/test/30_close_wall.png
+../dataset/jp/test/31_input.jpg	../dataset/jp/test/31_wall.png	../dataset/jp/test/31_close.png	../dataset/jp/test/31_rooms.png	../dataset/jp/test/31_close_wall.png
+../dataset/jp/test/32_input.jpg	../dataset/jp/test/32_wall.png	../dataset/jp/test/32_close.png	../dataset/jp/test/32_rooms.png	../dataset/jp/test/32_close_wall.png
+../dataset/jp/test/33_input.jpg	../dataset/jp/test/33_wall.png	../dataset/jp/test/33_close.png	../dataset/jp/test/33_rooms.png	../dataset/jp/test/33_close_wall.png
+../dataset/jp/test/34_input.jpg	../dataset/jp/test/34_wall.png	../dataset/jp/test/34_close.png	../dataset/jp/test/34_rooms.png	../dataset/jp/test/34_close_wall.png
+../dataset/jp/test/35_input.jpg	../dataset/jp/test/35_wall.png	../dataset/jp/test/35_close.png	../dataset/jp/test/35_rooms.png	../dataset/jp/test/35_close_wall.png
+../dataset/jp/test/36_input.jpg	../dataset/jp/test/36_wall.png	../dataset/jp/test/36_close.png	../dataset/jp/test/36_rooms.png	../dataset/jp/test/36_close_wall.png
+../dataset/jp/test/37_input.jpg	../dataset/jp/test/37_wall.png	../dataset/jp/test/37_close.png	../dataset/jp/test/37_rooms.png	../dataset/jp/test/37_close_wall.png
+../dataset/jp/test/38_input.jpg	../dataset/jp/test/38_wall.png	../dataset/jp/test/38_close.png	../dataset/jp/test/38_rooms.png	../dataset/jp/test/38_close_wall.png
+../dataset/jp/test/39_input.jpg	../dataset/jp/test/39_wall.png	../dataset/jp/test/39_close.png	../dataset/jp/test/39_rooms.png	../dataset/jp/test/39_close_wall.png
+../dataset/jp/test/3_input.jpg	../dataset/jp/test/3_wall.png	../dataset/jp/test/3_close.png	../dataset/jp/test/3_rooms.png	../dataset/jp/test/3_close_wall.png
+../dataset/jp/test/40_input.jpg	../dataset/jp/test/40_wall.png	../dataset/jp/test/40_close.png	../dataset/jp/test/40_rooms.png	../dataset/jp/test/40_close_wall.png
+../dataset/jp/test/41_input.jpg	../dataset/jp/test/41_wall.png	../dataset/jp/test/41_close.png	../dataset/jp/test/41_rooms.png	../dataset/jp/test/41_close_wall.png
+../dataset/jp/test/42_input.jpg	../dataset/jp/test/42_wall.png	../dataset/jp/test/42_close.png	../dataset/jp/test/42_rooms.png	../dataset/jp/test/42_close_wall.png
+../dataset/jp/test/43_input.jpg	../dataset/jp/test/43_wall.png	../dataset/jp/test/43_close.png	../dataset/jp/test/43_rooms.png	../dataset/jp/test/43_close_wall.png
+../dataset/jp/test/44_input.jpg	../dataset/jp/test/44_wall.png	../dataset/jp/test/44_close.png	../dataset/jp/test/44_rooms.png	../dataset/jp/test/44_close_wall.png
+../dataset/jp/test/45_input.jpg	../dataset/jp/test/45_wall.png	../dataset/jp/test/45_close.png	../dataset/jp/test/45_rooms.png	../dataset/jp/test/45_close_wall.png
+../dataset/jp/test/46_input.jpg	../dataset/jp/test/46_wall.png	../dataset/jp/test/46_close.png	../dataset/jp/test/46_rooms.png	../dataset/jp/test/46_close_wall.png
+../dataset/jp/test/47_input.jpg	../dataset/jp/test/47_wall.png	../dataset/jp/test/47_close.png	../dataset/jp/test/47_rooms.png	../dataset/jp/test/47_close_wall.png
+../dataset/jp/test/48_input.jpg	../dataset/jp/test/48_wall.png	../dataset/jp/test/48_close.png	../dataset/jp/test/48_rooms.png	../dataset/jp/test/48_close_wall.png
+../dataset/jp/test/49_input.jpg	../dataset/jp/test/49_wall.png	../dataset/jp/test/49_close.png	../dataset/jp/test/49_rooms.png	../dataset/jp/test/49_close_wall.png
+../dataset/jp/test/4_input.jpg	../dataset/jp/test/4_wall.png	../dataset/jp/test/4_close.png	../dataset/jp/test/4_rooms.png	../dataset/jp/test/4_close_wall.png
+../dataset/jp/test/50_input.jpg	../dataset/jp/test/50_wall.png	../dataset/jp/test/50_close.png	../dataset/jp/test/50_rooms.png	../dataset/jp/test/50_close_wall.png
+../dataset/jp/test/51_input.jpg	../dataset/jp/test/51_wall.png	../dataset/jp/test/51_close.png	../dataset/jp/test/51_rooms.png	../dataset/jp/test/51_close_wall.png
+../dataset/jp/test/52_input.jpg	../dataset/jp/test/52_wall.png	../dataset/jp/test/52_close.png	../dataset/jp/test/52_rooms.png	../dataset/jp/test/52_close_wall.png
+../dataset/jp/test/53_input.jpg	../dataset/jp/test/53_wall.png	../dataset/jp/test/53_close.png	../dataset/jp/test/53_rooms.png	../dataset/jp/test/53_close_wall.png
+../dataset/jp/test/54_input.jpg	../dataset/jp/test/54_wall.png	../dataset/jp/test/54_close.png	../dataset/jp/test/54_rooms.png	../dataset/jp/test/54_close_wall.png
+../dataset/jp/test/55_input.jpg	../dataset/jp/test/55_wall.png	../dataset/jp/test/55_close.png	../dataset/jp/test/55_rooms.png	../dataset/jp/test/55_close_wall.png
+../dataset/jp/test/56_input.jpg	../dataset/jp/test/56_wall.png	../dataset/jp/test/56_close.png	../dataset/jp/test/56_rooms.png	../dataset/jp/test/56_close_wall.png
+../dataset/jp/test/57_input.jpg	../dataset/jp/test/57_wall.png	../dataset/jp/test/57_close.png	../dataset/jp/test/57_rooms.png	../dataset/jp/test/57_close_wall.png
+../dataset/jp/test/58_input.jpg	../dataset/jp/test/58_wall.png	../dataset/jp/test/58_close.png	../dataset/jp/test/58_rooms.png	../dataset/jp/test/58_close_wall.png
+../dataset/jp/test/59_input.jpg	../dataset/jp/test/59_wall.png	../dataset/jp/test/59_close.png	../dataset/jp/test/59_rooms.png	../dataset/jp/test/59_close_wall.png
+../dataset/jp/test/5_input.jpg	../dataset/jp/test/5_wall.png	../dataset/jp/test/5_close.png	../dataset/jp/test/5_rooms.png	../dataset/jp/test/5_close_wall.png
+../dataset/jp/test/60_input.jpg	../dataset/jp/test/60_wall.png	../dataset/jp/test/60_close.png	../dataset/jp/test/60_rooms.png	../dataset/jp/test/60_close_wall.png
+../dataset/jp/test/61_input.jpg	../dataset/jp/test/61_wall.png	../dataset/jp/test/61_close.png	../dataset/jp/test/61_rooms.png	../dataset/jp/test/61_close_wall.png
+../dataset/jp/test/62_input.jpg	../dataset/jp/test/62_wall.png	../dataset/jp/test/62_close.png	../dataset/jp/test/62_rooms.png	../dataset/jp/test/62_close_wall.png
+../dataset/jp/test/63_input.jpg	../dataset/jp/test/63_wall.png	../dataset/jp/test/63_close.png	../dataset/jp/test/63_rooms.png	../dataset/jp/test/63_close_wall.png
+../dataset/jp/test/64_input.jpg	../dataset/jp/test/64_wall.png	../dataset/jp/test/64_close.png	../dataset/jp/test/64_rooms.png	../dataset/jp/test/64_close_wall.png
+../dataset/jp/test/65_input.jpg	../dataset/jp/test/65_wall.png	../dataset/jp/test/65_close.png	../dataset/jp/test/65_rooms.png	../dataset/jp/test/65_close_wall.png
+../dataset/jp/test/66_input.jpg	../dataset/jp/test/66_wall.png	../dataset/jp/test/66_close.png	../dataset/jp/test/66_rooms.png	../dataset/jp/test/66_close_wall.png
+../dataset/jp/test/67_input.jpg	../dataset/jp/test/67_wall.png	../dataset/jp/test/67_close.png	../dataset/jp/test/67_rooms.png	../dataset/jp/test/67_close_wall.png
+../dataset/jp/test/68_input.jpg	../dataset/jp/test/68_wall.png	../dataset/jp/test/68_close.png	../dataset/jp/test/68_rooms.png	../dataset/jp/test/68_close_wall.png
+../dataset/jp/test/69_input.jpg	../dataset/jp/test/69_wall.png	../dataset/jp/test/69_close.png	../dataset/jp/test/69_rooms.png	../dataset/jp/test/69_close_wall.png
+../dataset/jp/test/6_input.jpg	../dataset/jp/test/6_wall.png	../dataset/jp/test/6_close.png	../dataset/jp/test/6_rooms.png	../dataset/jp/test/6_close_wall.png
+../dataset/jp/test/70_input.jpg	../dataset/jp/test/70_wall.png	../dataset/jp/test/70_close.png	../dataset/jp/test/70_rooms.png	../dataset/jp/test/70_close_wall.png
+../dataset/jp/test/71_input.jpg	../dataset/jp/test/71_wall.png	../dataset/jp/test/71_close.png	../dataset/jp/test/71_rooms.png	../dataset/jp/test/71_close_wall.png
+../dataset/jp/test/72_input.jpg	../dataset/jp/test/72_wall.png	../dataset/jp/test/72_close.png	../dataset/jp/test/72_rooms.png	../dataset/jp/test/72_close_wall.png
+../dataset/jp/test/73_input.jpg	../dataset/jp/test/73_wall.png	../dataset/jp/test/73_close.png	../dataset/jp/test/73_rooms.png	../dataset/jp/test/73_close_wall.png
+../dataset/jp/test/74_input.jpg	../dataset/jp/test/74_wall.png	../dataset/jp/test/74_close.png	../dataset/jp/test/74_rooms.png	../dataset/jp/test/74_close_wall.png
+../dataset/jp/test/75_input.jpg	../dataset/jp/test/75_wall.png	../dataset/jp/test/75_close.png	../dataset/jp/test/75_rooms.png	../dataset/jp/test/75_close_wall.png
+../dataset/jp/test/76_input.jpg	../dataset/jp/test/76_wall.png	../dataset/jp/test/76_close.png	../dataset/jp/test/76_rooms.png	../dataset/jp/test/76_close_wall.png
+../dataset/jp/test/77_input.jpg	../dataset/jp/test/77_wall.png	../dataset/jp/test/77_close.png	../dataset/jp/test/77_rooms.png	../dataset/jp/test/77_close_wall.png
+../dataset/jp/test/78_input.jpg	../dataset/jp/test/78_wall.png	../dataset/jp/test/78_close.png	../dataset/jp/test/78_rooms.png	../dataset/jp/test/78_close_wall.png
+../dataset/jp/test/79_input.jpg	../dataset/jp/test/79_wall.png	../dataset/jp/test/79_close.png	../dataset/jp/test/79_rooms.png	../dataset/jp/test/79_close_wall.png
+../dataset/jp/test/7_input.jpg	../dataset/jp/test/7_wall.png	../dataset/jp/test/7_close.png	../dataset/jp/test/7_rooms.png	../dataset/jp/test/7_close_wall.png
+../dataset/jp/test/80_input.jpg	../dataset/jp/test/80_wall.png	../dataset/jp/test/80_close.png	../dataset/jp/test/80_rooms.png	../dataset/jp/test/80_close_wall.png
+../dataset/jp/test/81_input.jpg	../dataset/jp/test/81_wall.png	../dataset/jp/test/81_close.png	../dataset/jp/test/81_rooms.png	../dataset/jp/test/81_close_wall.png
+../dataset/jp/test/82_input.jpg	../dataset/jp/test/82_wall.png	../dataset/jp/test/82_close.png	../dataset/jp/test/82_rooms.png	../dataset/jp/test/82_close_wall.png
+../dataset/jp/test/83_input.jpg	../dataset/jp/test/83_wall.png	../dataset/jp/test/83_close.png	../dataset/jp/test/83_rooms.png	../dataset/jp/test/83_close_wall.png
+../dataset/jp/test/84_input.jpg	../dataset/jp/test/84_wall.png	../dataset/jp/test/84_close.png	../dataset/jp/test/84_rooms.png	../dataset/jp/test/84_close_wall.png
+../dataset/jp/test/85_input.jpg	../dataset/jp/test/85_wall.png	../dataset/jp/test/85_close.png	../dataset/jp/test/85_rooms.png	../dataset/jp/test/85_close_wall.png
+../dataset/jp/test/86_input.jpg	../dataset/jp/test/86_wall.png	../dataset/jp/test/86_close.png	../dataset/jp/test/86_rooms.png	../dataset/jp/test/86_close_wall.png
+../dataset/jp/test/87_input.jpg	../dataset/jp/test/87_wall.png	../dataset/jp/test/87_close.png	../dataset/jp/test/87_rooms.png	../dataset/jp/test/87_close_wall.png
+../dataset/jp/test/88_input.jpg	../dataset/jp/test/88_wall.png	../dataset/jp/test/88_close.png	../dataset/jp/test/88_rooms.png	../dataset/jp/test/88_close_wall.png
+../dataset/jp/test/89_input.jpg	../dataset/jp/test/89_wall.png	../dataset/jp/test/89_close.png	../dataset/jp/test/89_rooms.png	../dataset/jp/test/89_close_wall.png
+../dataset/jp/test/8_input.jpg	../dataset/jp/test/8_wall.png	../dataset/jp/test/8_close.png	../dataset/jp/test/8_rooms.png	../dataset/jp/test/8_close_wall.png
+../dataset/jp/test/90_input.jpg	../dataset/jp/test/90_wall.png	../dataset/jp/test/90_close.png	../dataset/jp/test/90_rooms.png	../dataset/jp/test/90_close_wall.png
+../dataset/jp/test/91_input.jpg	../dataset/jp/test/91_wall.png	../dataset/jp/test/91_close.png	../dataset/jp/test/91_rooms.png	../dataset/jp/test/91_close_wall.png
+../dataset/jp/test/92_input.jpg	../dataset/jp/test/92_wall.png	../dataset/jp/test/92_close.png	../dataset/jp/test/92_rooms.png	../dataset/jp/test/92_close_wall.png
+../dataset/jp/test/93_input.jpg	../dataset/jp/test/93_wall.png	../dataset/jp/test/93_close.png	../dataset/jp/test/93_rooms.png	../dataset/jp/test/93_close_wall.png
+../dataset/jp/test/94_input.jpg	../dataset/jp/test/94_wall.png	../dataset/jp/test/94_close.png	../dataset/jp/test/94_rooms.png	../dataset/jp/test/94_close_wall.png
+../dataset/jp/test/95_input.jpg	../dataset/jp/test/95_wall.png	../dataset/jp/test/95_close.png	../dataset/jp/test/95_rooms.png	../dataset/jp/test/95_close_wall.png
+../dataset/jp/test/96_input.jpg	../dataset/jp/test/96_wall.png	../dataset/jp/test/96_close.png	../dataset/jp/test/96_rooms.png	../dataset/jp/test/96_close_wall.png
+../dataset/jp/test/97_input.jpg	../dataset/jp/test/97_wall.png	../dataset/jp/test/97_close.png	../dataset/jp/test/97_rooms.png	../dataset/jp/test/97_close_wall.png
+../dataset/jp/test/98_input.jpg	../dataset/jp/test/98_wall.png	../dataset/jp/test/98_close.png	../dataset/jp/test/98_rooms.png	../dataset/jp/test/98_close_wall.png
+../dataset/jp/test/99_input.jpg	../dataset/jp/test/99_wall.png	../dataset/jp/test/99_close.png	../dataset/jp/test/99_rooms.png	../dataset/jp/test/99_close_wall.png
+../dataset/jp/test/9_input.jpg	../dataset/jp/test/9_wall.png	../dataset/jp/test/9_close.png	../dataset/jp/test/9_rooms.png	../dataset/jp/test/9_close_wall.png

dataset/r3d_test.txt

@@ -0,0 +1,53 @@
+../dataset/newyork/test/21.jpg	../dataset/newyork/test/21_wall.png	../dataset/newyork/test/21_close.png	../dataset/newyork/test/21_rooms.png	../dataset/newyork/test/21_close_wall.png
+../dataset/newyork/test/30691830.jpg	../dataset/newyork/test/30691830_wall.png	../dataset/newyork/test/30691830_close.png	../dataset/newyork/test/30691830_rooms.png	../dataset/newyork/test/30691830_close_wall.png
+../dataset/newyork/test/31074492.jpg	../dataset/newyork/test/31074492_wall.png	../dataset/newyork/test/31074492_close.png	../dataset/newyork/test/31074492_rooms.png	../dataset/newyork/test/31074492_close_wall.png
+../dataset/newyork/test/31837524.jpg	../dataset/newyork/test/31837524_wall.png	../dataset/newyork/test/31837524_close.png	../dataset/newyork/test/31837524_rooms.png	../dataset/newyork/test/31837524_close_wall.png
+../dataset/newyork/test/31851141.jpg	../dataset/newyork/test/31851141_wall.png	../dataset/newyork/test/31851141_close.png	../dataset/newyork/test/31851141_rooms.png	../dataset/newyork/test/31851141_close_wall.png
+../dataset/newyork/test/31873188.jpg	../dataset/newyork/test/31873188_wall.png	../dataset/newyork/test/31873188_close.png	../dataset/newyork/test/31873188_rooms.png	../dataset/newyork/test/31873188_close_wall.png
+../dataset/newyork/test/31889856.jpg	../dataset/newyork/test/31889856_wall.png	../dataset/newyork/test/31889856_close.png	../dataset/newyork/test/31889856_rooms.png	../dataset/newyork/test/31889856_close_wall.png
+../dataset/newyork/test/43949851.jpg	../dataset/newyork/test/43949851_wall.png	../dataset/newyork/test/43949851_close.png	../dataset/newyork/test/43949851_rooms.png	../dataset/newyork/test/43949851_close_wall.png
+../dataset/newyork/test/44777104.jpg	../dataset/newyork/test/44777104_wall.png	../dataset/newyork/test/44777104_close.png	../dataset/newyork/test/44777104_rooms.png	../dataset/newyork/test/44777104_close_wall.png
+../dataset/newyork/test/45157357.jpg	../dataset/newyork/test/45157357_wall.png	../dataset/newyork/test/45157357_close.png	../dataset/newyork/test/45157357_rooms.png	../dataset/newyork/test/45157357_close_wall.png
+../dataset/newyork/test/45299197.jpg	../dataset/newyork/test/45299197_wall.png	../dataset/newyork/test/45299197_close.png	../dataset/newyork/test/45299197_rooms.png	../dataset/newyork/test/45299197_close_wall.png
+../dataset/newyork/test/45348658.jpg	../dataset/newyork/test/45348658_wall.png	../dataset/newyork/test/45348658_close.png	../dataset/newyork/test/45348658_rooms.png	../dataset/newyork/test/45348658_close_wall.png
+../dataset/newyork/test/45719584.jpg	../dataset/newyork/test/45719584_wall.png	../dataset/newyork/test/45719584_close.png	../dataset/newyork/test/45719584_rooms.png	../dataset/newyork/test/45719584_close_wall.png
+../dataset/newyork/test/45720004.jpg	../dataset/newyork/test/45720004_wall.png	../dataset/newyork/test/45720004_close.png	../dataset/newyork/test/45720004_rooms.png	../dataset/newyork/test/45720004_close_wall.png
+../dataset/newyork/test/45724132.jpg	../dataset/newyork/test/45724132_wall.png	../dataset/newyork/test/45724132_close.png	../dataset/newyork/test/45724132_rooms.png	../dataset/newyork/test/45724132_close_wall.png
+../dataset/newyork/test/45724363.jpg	../dataset/newyork/test/45724363_wall.png	../dataset/newyork/test/45724363_close.png	../dataset/newyork/test/45724363_rooms.png	../dataset/newyork/test/45724363_close_wall.png
+../dataset/newyork/test/45724372.jpg	../dataset/newyork/test/45724372_wall.png	../dataset/newyork/test/45724372_close.png	../dataset/newyork/test/45724372_rooms.png	../dataset/newyork/test/45724372_close_wall.png
+../dataset/newyork/test/45740533.jpg	../dataset/newyork/test/45740533_wall.png	../dataset/newyork/test/45740533_close.png	../dataset/newyork/test/45740533_rooms.png	../dataset/newyork/test/45740533_close_wall.png
+../dataset/newyork/test/45765448.jpg	../dataset/newyork/test/45765448_wall.png	../dataset/newyork/test/45765448_close.png	../dataset/newyork/test/45765448_rooms.png	../dataset/newyork/test/45765448_close_wall.png
+../dataset/newyork/test/45775069.jpg	../dataset/newyork/test/45775069_wall.png	../dataset/newyork/test/45775069_close.png	../dataset/newyork/test/45775069_rooms.png	../dataset/newyork/test/45775069_close_wall.png
+../dataset/newyork/test/45780715.jpg	../dataset/newyork/test/45780715_wall.png	../dataset/newyork/test/45780715_close.png	../dataset/newyork/test/45780715_rooms.png	../dataset/newyork/test/45780715_close_wall.png
+../dataset/newyork/test/46543250.jpg	../dataset/newyork/test/46543250_wall.png	../dataset/newyork/test/46543250_close.png	../dataset/newyork/test/46543250_rooms.png	../dataset/newyork/test/46543250_close_wall.png
+../dataset/newyork/test/47464145.jpg	../dataset/newyork/test/47464145_wall.png	../dataset/newyork/test/47464145_close.png	../dataset/newyork/test/47464145_rooms.png	../dataset/newyork/test/47464145_close_wall.png
+../dataset/newyork/test/47485670.jpg	../dataset/newyork/test/47485670_wall.png	../dataset/newyork/test/47485670_close.png	../dataset/newyork/test/47485670_rooms.png	../dataset/newyork/test/47485670_close_wall.png
+../dataset/newyork/test/47489612.jpg	../dataset/newyork/test/47489612_wall.png	../dataset/newyork/test/47489612_close.png	../dataset/newyork/test/47489612_rooms.png	../dataset/newyork/test/47489612_close_wall.png
+../dataset/newyork/test/47499272.jpg	../dataset/newyork/test/47499272_wall.png	../dataset/newyork/test/47499272_close.png	../dataset/newyork/test/47499272_rooms.png	../dataset/newyork/test/47499272_close_wall.png
+../dataset/newyork/test/47499362.jpg	../dataset/newyork/test/47499362_wall.png	../dataset/newyork/test/47499362_close.png	../dataset/newyork/test/47499362_rooms.png	../dataset/newyork/test/47499362_close_wall.png
+../dataset/newyork/test/47505362.jpg	../dataset/newyork/test/47505362_wall.png	../dataset/newyork/test/47505362_close.png	../dataset/newyork/test/47505362_rooms.png	../dataset/newyork/test/47505362_close_wall.png
+../dataset/newyork/test/47525504.jpg	../dataset/newyork/test/47525504_wall.png	../dataset/newyork/test/47525504_close.png	../dataset/newyork/test/47525504_rooms.png	../dataset/newyork/test/47525504_close_wall.png
+../dataset/newyork/test/47541842.jpg	../dataset/newyork/test/47541842_wall.png	../dataset/newyork/test/47541842_close.png	../dataset/newyork/test/47541842_rooms.png	../dataset/newyork/test/47541842_close_wall.png
+../dataset/newyork/test/47541845.jpg	../dataset/newyork/test/47541845_wall.png	../dataset/newyork/test/47541845_close.png	../dataset/newyork/test/47541845_rooms.png	../dataset/newyork/test/47541845_close_wall.png
+../dataset/newyork/test/47541857.jpg	../dataset/newyork/test/47541857_wall.png	../dataset/newyork/test/47541857_close.png	../dataset/newyork/test/47541857_rooms.png	../dataset/newyork/test/47541857_close_wall.png
+../dataset/newyork/test/47541860.jpg	../dataset/newyork/test/47541860_wall.png	../dataset/newyork/test/47541860_close.png	../dataset/newyork/test/47541860_rooms.png	../dataset/newyork/test/47541860_close_wall.png
+../dataset/newyork/test/47541863.jpg	../dataset/newyork/test/47541863_wall.png	../dataset/newyork/test/47541863_close.png	../dataset/newyork/test/47541863_rooms.png	../dataset/newyork/test/47541863_close_wall.png
+../dataset/newyork/test/47541866.jpg	../dataset/newyork/test/47541866_wall.png	../dataset/newyork/test/47541866_close.png	../dataset/newyork/test/47541866_rooms.png	../dataset/newyork/test/47541866_close_wall.png
+../dataset/newyork/test/47542733.jpg	../dataset/newyork/test/47542733_wall.png	../dataset/newyork/test/47542733_close.png	../dataset/newyork/test/47542733_rooms.png	../dataset/newyork/test/47542733_close_wall.png
+../dataset/newyork/test/47542745.jpg	../dataset/newyork/test/47542745_wall.png	../dataset/newyork/test/47542745_close.png	../dataset/newyork/test/47542745_rooms.png	../dataset/newyork/test/47542745_close_wall.png
+../dataset/newyork/test/47545139.jpg	../dataset/newyork/test/47545139_wall.png	../dataset/newyork/test/47545139_close.png	../dataset/newyork/test/47545139_rooms.png	../dataset/newyork/test/47545139_close_wall.png
+../dataset/newyork/test/47545145.jpg	../dataset/newyork/test/47545145_wall.png	../dataset/newyork/test/47545145_close.png	../dataset/newyork/test/47545145_rooms.png	../dataset/newyork/test/47545145_close_wall.png
+../dataset/newyork/test/47545148.jpg	../dataset/newyork/test/47545148_wall.png	../dataset/newyork/test/47545148_close.png	../dataset/newyork/test/47545148_rooms.png	../dataset/newyork/test/47545148_close_wall.png
+../dataset/newyork/test/47545160.jpg	../dataset/newyork/test/47545160_wall.png	../dataset/newyork/test/47545160_close.png	../dataset/newyork/test/47545160_rooms.png	../dataset/newyork/test/47545160_close_wall.png
+../dataset/newyork/test/47546432.jpg	../dataset/newyork/test/47546432_wall.png	../dataset/newyork/test/47546432_close.png	../dataset/newyork/test/47546432_rooms.png	../dataset/newyork/test/47546432_close_wall.png
+../dataset/newyork/test/47546639.jpg	../dataset/newyork/test/47546639_wall.png	../dataset/newyork/test/47546639_close.png	../dataset/newyork/test/47546639_rooms.png	../dataset/newyork/test/47546639_close_wall.png
+../dataset/newyork/test/47546846.jpg	../dataset/newyork/test/47546846_wall.png	../dataset/newyork/test/47546846_close.png	../dataset/newyork/test/47546846_rooms.png	../dataset/newyork/test/47546846_close_wall.png
+../dataset/newyork/test/47547656.jpg	../dataset/newyork/test/47547656_wall.png	../dataset/newyork/test/47547656_close.png	../dataset/newyork/test/47547656_rooms.png	../dataset/newyork/test/47547656_close_wall.png
+../dataset/newyork/test/47548484.jpg	../dataset/newyork/test/47548484_wall.png	../dataset/newyork/test/47548484_close.png	../dataset/newyork/test/47548484_rooms.png	../dataset/newyork/test/47548484_close_wall.png
+../dataset/newyork/test/47548487.jpg	../dataset/newyork/test/47548487_wall.png	../dataset/newyork/test/47548487_close.png	../dataset/newyork/test/47548487_rooms.png	../dataset/newyork/test/47548487_close_wall.png
+../dataset/newyork/test/55.jpg	../dataset/newyork/test/55_wall.png	../dataset/newyork/test/55_close.png	../dataset/newyork/test/55_rooms.png	../dataset/newyork/test/55_close_wall.png
+../dataset/newyork/test/60.jpg	../dataset/newyork/test/60_wall.png	../dataset/newyork/test/60_close.png	../dataset/newyork/test/60_rooms.png	../dataset/newyork/test/60_close_wall.png
+../dataset/newyork/test/62.jpg	../dataset/newyork/test/62_wall.png	../dataset/newyork/test/62_close.png	../dataset/newyork/test/62_rooms.png	../dataset/newyork/test/62_close_wall.png
+../dataset/newyork/test/65.jpg	../dataset/newyork/test/65_wall.png	../dataset/newyork/test/65_close.png	../dataset/newyork/test/65_rooms.png	../dataset/newyork/test/65_close_wall.png
+../dataset/newyork/test/75.jpg	../dataset/newyork/test/75_wall.png	../dataset/newyork/test/75_close.png	../dataset/newyork/test/75_rooms.png	../dataset/newyork/test/75_close_wall.png
+../dataset/newyork/test/9.jpg	../dataset/newyork/test/9_wall.png	../dataset/newyork/test/9_close.png	../dataset/newyork/test/9_rooms.png	../dataset/newyork/test/9_close_wall.png

dataset/r3d_train.txt

@@ -0,0 +1,179 @@
+../dataset/newyork/train/10.jpg	../dataset/newyork/train/10_wall.png	../dataset/newyork/train/10_close.png	../dataset/newyork/train/10_rooms.png	../dataset/newyork/train/10_close_wall.png
+../dataset/newyork/train/28025487.jpg	../dataset/newyork/train/28025487_wall.png	../dataset/newyork/train/28025487_close.png	../dataset/newyork/train/28025487_rooms.png	../dataset/newyork/train/28025487_close_wall.png
+../dataset/newyork/train/28906422.jpg	../dataset/newyork/train/28906422_wall.png	../dataset/newyork/train/28906422_close.png	../dataset/newyork/train/28906422_rooms.png	../dataset/newyork/train/28906422_close_wall.png
+../dataset/newyork/train/2.jpg	../dataset/newyork/train/2_wall.png	../dataset/newyork/train/2_close.png	../dataset/newyork/train/2_rooms.png	../dataset/newyork/train/2_close_wall.png
+../dataset/newyork/train/30044076.jpg	../dataset/newyork/train/30044076_wall.png	../dataset/newyork/train/30044076_close.png	../dataset/newyork/train/30044076_rooms.png	../dataset/newyork/train/30044076_close_wall.png
+../dataset/newyork/train/30049107.jpg	../dataset/newyork/train/30049107_wall.png	../dataset/newyork/train/30049107_close.png	../dataset/newyork/train/30049107_rooms.png	../dataset/newyork/train/30049107_close_wall.png
+../dataset/newyork/train/30615117.jpg	../dataset/newyork/train/30615117_wall.png	../dataset/newyork/train/30615117_close.png	../dataset/newyork/train/30615117_rooms.png	../dataset/newyork/train/30615117_close_wall.png
+../dataset/newyork/train/30939153.jpg	../dataset/newyork/train/30939153_wall.png	../dataset/newyork/train/30939153_close.png	../dataset/newyork/train/30939153_rooms.png	../dataset/newyork/train/30939153_close_wall.png
+../dataset/newyork/train/30957516.jpg	../dataset/newyork/train/30957516_wall.png	../dataset/newyork/train/30957516_close.png	../dataset/newyork/train/30957516_rooms.png	../dataset/newyork/train/30957516_close_wall.png
+../dataset/newyork/train/31036152.jpg	../dataset/newyork/train/31036152_wall.png	../dataset/newyork/train/31036152_close.png	../dataset/newyork/train/31036152_rooms.png	../dataset/newyork/train/31036152_close_wall.png
+../dataset/newyork/train/31234182.jpg	../dataset/newyork/train/31234182_wall.png	../dataset/newyork/train/31234182_close.png	../dataset/newyork/train/31234182_rooms.png	../dataset/newyork/train/31234182_close_wall.png
+../dataset/newyork/train/31272420.jpg	../dataset/newyork/train/31272420_wall.png	../dataset/newyork/train/31272420_close.png	../dataset/newyork/train/31272420_rooms.png	../dataset/newyork/train/31272420_close_wall.png
+../dataset/newyork/train/31318404.jpg	../dataset/newyork/train/31318404_wall.png	../dataset/newyork/train/31318404_close.png	../dataset/newyork/train/31318404_rooms.png	../dataset/newyork/train/31318404_close_wall.png
+../dataset/newyork/train/31418847.jpg	../dataset/newyork/train/31418847_wall.png	../dataset/newyork/train/31418847_close.png	../dataset/newyork/train/31418847_rooms.png	../dataset/newyork/train/31418847_close_wall.png
+../dataset/newyork/train/31431717.jpg	../dataset/newyork/train/31431717_wall.png	../dataset/newyork/train/31431717_close.png	../dataset/newyork/train/31431717_rooms.png	../dataset/newyork/train/31431717_close_wall.png
+../dataset/newyork/train/31450071.jpg	../dataset/newyork/train/31450071_wall.png	../dataset/newyork/train/31450071_close.png	../dataset/newyork/train/31450071_rooms.png	../dataset/newyork/train/31450071_close_wall.png
+../dataset/newyork/train/31483593.jpg	../dataset/newyork/train/31483593_wall.png	../dataset/newyork/train/31483593_close.png	../dataset/newyork/train/31483593_rooms.png	../dataset/newyork/train/31483593_close_wall.png
+../dataset/newyork/train/31491612.jpg	../dataset/newyork/train/31491612_wall.png	../dataset/newyork/train/31491612_close.png	../dataset/newyork/train/31491612_rooms.png	../dataset/newyork/train/31491612_close_wall.png
+../dataset/newyork/train/31566489.jpg	../dataset/newyork/train/31566489_wall.png	../dataset/newyork/train/31566489_close.png	../dataset/newyork/train/31566489_rooms.png	../dataset/newyork/train/31566489_close_wall.png
+../dataset/newyork/train/31567842.jpg	../dataset/newyork/train/31567842_wall.png	../dataset/newyork/train/31567842_close.png	../dataset/newyork/train/31567842_rooms.png	../dataset/newyork/train/31567842_close_wall.png
+../dataset/newyork/train/31573533.jpg	../dataset/newyork/train/31573533_wall.png	../dataset/newyork/train/31573533_close.png	../dataset/newyork/train/31573533_rooms.png	../dataset/newyork/train/31573533_close_wall.png
+../dataset/newyork/train/31677402.jpg	../dataset/newyork/train/31677402_wall.png	../dataset/newyork/train/31677402_close.png	../dataset/newyork/train/31677402_rooms.png	../dataset/newyork/train/31677402_close_wall.png
+../dataset/newyork/train/31683135.jpg	../dataset/newyork/train/31683135_wall.png	../dataset/newyork/train/31683135_close.png	../dataset/newyork/train/31683135_rooms.png	../dataset/newyork/train/31683135_close_wall.png
+../dataset/newyork/train/31727418.jpg	../dataset/newyork/train/31727418_wall.png	../dataset/newyork/train/31727418_close.png	../dataset/newyork/train/31727418_rooms.png	../dataset/newyork/train/31727418_close_wall.png
+../dataset/newyork/train/31814460.jpg	../dataset/newyork/train/31814460_wall.png	../dataset/newyork/train/31814460_close.png	../dataset/newyork/train/31814460_rooms.png	../dataset/newyork/train/31814460_close_wall.png
+../dataset/newyork/train/31820961.jpg	../dataset/newyork/train/31820961_wall.png	../dataset/newyork/train/31820961_close.png	../dataset/newyork/train/31820961_rooms.png	../dataset/newyork/train/31820961_close_wall.png
+../dataset/newyork/train/31826949.jpg	../dataset/newyork/train/31826949_wall.png	../dataset/newyork/train/31826949_close.png	../dataset/newyork/train/31826949_rooms.png	../dataset/newyork/train/31826949_close_wall.png
+../dataset/newyork/train/31829949.jpg	../dataset/newyork/train/31829949_wall.png	../dataset/newyork/train/31829949_close.png	../dataset/newyork/train/31829949_rooms.png	../dataset/newyork/train/31829949_close_wall.png
+../dataset/newyork/train/31830006.jpg	../dataset/newyork/train/31830006_wall.png	../dataset/newyork/train/31830006_close.png	../dataset/newyork/train/31830006_rooms.png	../dataset/newyork/train/31830006_close_wall.png
+../dataset/newyork/train/31830138.jpg	../dataset/newyork/train/31830138_wall.png	../dataset/newyork/train/31830138_close.png	../dataset/newyork/train/31830138_rooms.png	../dataset/newyork/train/31830138_close_wall.png
+../dataset/newyork/train/31830141.jpg	../dataset/newyork/train/31830141_wall.png	../dataset/newyork/train/31830141_close.png	../dataset/newyork/train/31830141_rooms.png	../dataset/newyork/train/31830141_close_wall.png
+../dataset/newyork/train/31830270.jpg	../dataset/newyork/train/31830270_wall.png	../dataset/newyork/train/31830270_close.png	../dataset/newyork/train/31830270_rooms.png	../dataset/newyork/train/31830270_close_wall.png
+../dataset/newyork/train/31833933.jpg	../dataset/newyork/train/31833933_wall.png	../dataset/newyork/train/31833933_close.png	../dataset/newyork/train/31833933_rooms.png	../dataset/newyork/train/31833933_close_wall.png
+../dataset/newyork/train/31834719.jpg	../dataset/newyork/train/31834719_wall.png	../dataset/newyork/train/31834719_close.png	../dataset/newyork/train/31834719_rooms.png	../dataset/newyork/train/31834719_close_wall.png
+../dataset/newyork/train/31834734.jpg	../dataset/newyork/train/31834734_wall.png	../dataset/newyork/train/31834734_close.png	../dataset/newyork/train/31834734_rooms.png	../dataset/newyork/train/31834734_close_wall.png
+../dataset/newyork/train/31835886.jpg	../dataset/newyork/train/31835886_wall.png	../dataset/newyork/train/31835886_close.png	../dataset/newyork/train/31835886_rooms.png	../dataset/newyork/train/31835886_close_wall.png
+../dataset/newyork/train/31847853.jpg	../dataset/newyork/train/31847853_wall.png	../dataset/newyork/train/31847853_close.png	../dataset/newyork/train/31847853_rooms.png	../dataset/newyork/train/31847853_close_wall.png
+../dataset/newyork/train/31850325.jpg	../dataset/newyork/train/31850325_wall.png	../dataset/newyork/train/31850325_close.png	../dataset/newyork/train/31850325_rooms.png	../dataset/newyork/train/31850325_close_wall.png
+../dataset/newyork/train/31850409.jpg	../dataset/newyork/train/31850409_wall.png	../dataset/newyork/train/31850409_close.png	../dataset/newyork/train/31850409_rooms.png	../dataset/newyork/train/31850409_close_wall.png
+../dataset/newyork/train/31852926.jpg	../dataset/newyork/train/31852926_wall.png	../dataset/newyork/train/31852926_close.png	../dataset/newyork/train/31852926_rooms.png	../dataset/newyork/train/31852926_close_wall.png
+../dataset/newyork/train/31852929.jpg	../dataset/newyork/train/31852929_wall.png	../dataset/newyork/train/31852929_close.png	../dataset/newyork/train/31852929_rooms.png	../dataset/newyork/train/31852929_close_wall.png
+../dataset/newyork/train/31852932.jpg	../dataset/newyork/train/31852932_wall.png	../dataset/newyork/train/31852932_close.png	../dataset/newyork/train/31852932_rooms.png	../dataset/newyork/train/31852932_close_wall.png
+../dataset/newyork/train/31857804.jpg	../dataset/newyork/train/31857804_wall.png	../dataset/newyork/train/31857804_close.png	../dataset/newyork/train/31857804_rooms.png	../dataset/newyork/train/31857804_close_wall.png
+../dataset/newyork/train/31868853.jpg	../dataset/newyork/train/31868853_wall.png	../dataset/newyork/train/31868853_close.png	../dataset/newyork/train/31868853_rooms.png	../dataset/newyork/train/31868853_close_wall.png
+../dataset/newyork/train/31870182.jpg	../dataset/newyork/train/31870182_wall.png	../dataset/newyork/train/31870182_close.png	../dataset/newyork/train/31870182_rooms.png	../dataset/newyork/train/31870182_close_wall.png
+../dataset/newyork/train/31870983.jpg	../dataset/newyork/train/31870983_wall.png	../dataset/newyork/train/31870983_close.png	../dataset/newyork/train/31870983_rooms.png	../dataset/newyork/train/31870983_close_wall.png
+../dataset/newyork/train/31871118.jpg	../dataset/newyork/train/31871118_wall.png	../dataset/newyork/train/31871118_close.png	../dataset/newyork/train/31871118_rooms.png	../dataset/newyork/train/31871118_close_wall.png
+../dataset/newyork/train/31871448.jpg	../dataset/newyork/train/31871448_wall.png	../dataset/newyork/train/31871448_close.png	../dataset/newyork/train/31871448_rooms.png	../dataset/newyork/train/31871448_close_wall.png
+../dataset/newyork/train/31872336.jpg	../dataset/newyork/train/31872336_wall.png	../dataset/newyork/train/31872336_close.png	../dataset/newyork/train/31872336_rooms.png	../dataset/newyork/train/31872336_close_wall.png
+../dataset/newyork/train/31872645.jpg	../dataset/newyork/train/31872645_wall.png	../dataset/newyork/train/31872645_close.png	../dataset/newyork/train/31872645_rooms.png	../dataset/newyork/train/31872645_close_wall.png
+../dataset/newyork/train/31873326.jpg	../dataset/newyork/train/31873326_wall.png	../dataset/newyork/train/31873326_close.png	../dataset/newyork/train/31873326_rooms.png	../dataset/newyork/train/31873326_close_wall.png
+../dataset/newyork/train/31874937.jpg	../dataset/newyork/train/31874937_wall.png	../dataset/newyork/train/31874937_close.png	../dataset/newyork/train/31874937_rooms.png	../dataset/newyork/train/31874937_close_wall.png
+../dataset/newyork/train/31878534.jpg	../dataset/newyork/train/31878534_wall.png	../dataset/newyork/train/31878534_close.png	../dataset/newyork/train/31878534_rooms.png	../dataset/newyork/train/31878534_close_wall.png
+../dataset/newyork/train/31878567.jpg	../dataset/newyork/train/31878567_wall.png	../dataset/newyork/train/31878567_close.png	../dataset/newyork/train/31878567_rooms.png	../dataset/newyork/train/31878567_close_wall.png
+../dataset/newyork/train/31878750.jpg	../dataset/newyork/train/31878750_wall.png	../dataset/newyork/train/31878750_close.png	../dataset/newyork/train/31878750_rooms.png	../dataset/newyork/train/31878750_close_wall.png
+../dataset/newyork/train/31878855.jpg	../dataset/newyork/train/31878855_wall.png	../dataset/newyork/train/31878855_close.png	../dataset/newyork/train/31878855_rooms.png	../dataset/newyork/train/31878855_close_wall.png
+../dataset/newyork/train/31878867.jpg	../dataset/newyork/train/31878867_wall.png	../dataset/newyork/train/31878867_close.png	../dataset/newyork/train/31878867_rooms.png	../dataset/newyork/train/31878867_close_wall.png
+../dataset/newyork/train/31878870.jpg	../dataset/newyork/train/31878870_wall.png	../dataset/newyork/train/31878870_close.png	../dataset/newyork/train/31878870_rooms.png	../dataset/newyork/train/31878870_close_wall.png
+../dataset/newyork/train/31882362.jpg	../dataset/newyork/train/31882362_wall.png	../dataset/newyork/train/31882362_close.png	../dataset/newyork/train/31882362_rooms.png	../dataset/newyork/train/31882362_close_wall.png
+../dataset/newyork/train/31883016.jpg	../dataset/newyork/train/31883016_wall.png	../dataset/newyork/train/31883016_close.png	../dataset/newyork/train/31883016_rooms.png	../dataset/newyork/train/31883016_close_wall.png
+../dataset/newyork/train/31883034.jpg	../dataset/newyork/train/31883034_wall.png	../dataset/newyork/train/31883034_close.png	../dataset/newyork/train/31883034_rooms.png	../dataset/newyork/train/31883034_close_wall.png
+../dataset/newyork/train/31883331.jpg	../dataset/newyork/train/31883331_wall.png	../dataset/newyork/train/31883331_close.png	../dataset/newyork/train/31883331_rooms.png	../dataset/newyork/train/31883331_close_wall.png
+../dataset/newyork/train/31887483.jpg	../dataset/newyork/train/31887483_wall.png	../dataset/newyork/train/31887483_close.png	../dataset/newyork/train/31887483_rooms.png	../dataset/newyork/train/31887483_close_wall.png
+../dataset/newyork/train/31887492.jpg	../dataset/newyork/train/31887492_wall.png	../dataset/newyork/train/31887492_close.png	../dataset/newyork/train/31887492_rooms.png	../dataset/newyork/train/31887492_close_wall.png
+../dataset/newyork/train/31889847.jpg	../dataset/newyork/train/31889847_wall.png	../dataset/newyork/train/31889847_close.png	../dataset/newyork/train/31889847_rooms.png	../dataset/newyork/train/31889847_close_wall.png
+../dataset/newyork/train/31890228.jpg	../dataset/newyork/train/31890228_wall.png	../dataset/newyork/train/31890228_close.png	../dataset/newyork/train/31890228_rooms.png	../dataset/newyork/train/31890228_close_wall.png
+../dataset/newyork/train/38877131.jpg	../dataset/newyork/train/38877131_wall.png	../dataset/newyork/train/38877131_close.png	../dataset/newyork/train/38877131_rooms.png	../dataset/newyork/train/38877131_close_wall.png
+../dataset/newyork/train/39.jpg	../dataset/newyork/train/39_wall.png	../dataset/newyork/train/39_close.png	../dataset/newyork/train/39_rooms.png	../dataset/newyork/train/39_close_wall.png
+../dataset/newyork/train/3.jpg	../dataset/newyork/train/3_wall.png	../dataset/newyork/train/3_close.png	../dataset/newyork/train/3_rooms.png	../dataset/newyork/train/3_close_wall.png
+../dataset/newyork/train/41459443.jpg	../dataset/newyork/train/41459443_wall.png	../dataset/newyork/train/41459443_close.png	../dataset/newyork/train/41459443_rooms.png	../dataset/newyork/train/41459443_close_wall.png
+../dataset/newyork/train/42761030.jpg	../dataset/newyork/train/42761030_wall.png	../dataset/newyork/train/42761030_close.png	../dataset/newyork/train/42761030_rooms.png	../dataset/newyork/train/42761030_close_wall.png
+../dataset/newyork/train/43169833.jpg	../dataset/newyork/train/43169833_wall.png	../dataset/newyork/train/43169833_close.png	../dataset/newyork/train/43169833_rooms.png	../dataset/newyork/train/43169833_close_wall.png
+../dataset/newyork/train/43661446.jpg	../dataset/newyork/train/43661446_wall.png	../dataset/newyork/train/43661446_close.png	../dataset/newyork/train/43661446_rooms.png	../dataset/newyork/train/43661446_close_wall.png
+../dataset/newyork/train/43778570.jpg	../dataset/newyork/train/43778570_wall.png	../dataset/newyork/train/43778570_close.png	../dataset/newyork/train/43778570_rooms.png	../dataset/newyork/train/43778570_close_wall.png
+../dataset/newyork/train/44356523.jpg	../dataset/newyork/train/44356523_wall.png	../dataset/newyork/train/44356523_close.png	../dataset/newyork/train/44356523_rooms.png	../dataset/newyork/train/44356523_close_wall.png
+../dataset/newyork/train/44591497.jpg	../dataset/newyork/train/44591497_wall.png	../dataset/newyork/train/44591497_close.png	../dataset/newyork/train/44591497_rooms.png	../dataset/newyork/train/44591497_close_wall.png
+../dataset/newyork/train/44637031.jpg	../dataset/newyork/train/44637031_wall.png	../dataset/newyork/train/44637031_close.png	../dataset/newyork/train/44637031_rooms.png	../dataset/newyork/train/44637031_close_wall.png
+../dataset/newyork/train/44867164.jpg	../dataset/newyork/train/44867164_wall.png	../dataset/newyork/train/44867164_close.png	../dataset/newyork/train/44867164_rooms.png	../dataset/newyork/train/44867164_close_wall.png
+../dataset/newyork/train/45057754.jpg	../dataset/newyork/train/45057754_wall.png	../dataset/newyork/train/45057754_close.png	../dataset/newyork/train/45057754_rooms.png	../dataset/newyork/train/45057754_close_wall.png
+../dataset/newyork/train/45098284.jpg	../dataset/newyork/train/45098284_wall.png	../dataset/newyork/train/45098284_close.png	../dataset/newyork/train/45098284_rooms.png	../dataset/newyork/train/45098284_close_wall.png
+../dataset/newyork/train/45175987.jpg	../dataset/newyork/train/45175987_wall.png	../dataset/newyork/train/45175987_close.png	../dataset/newyork/train/45175987_rooms.png	../dataset/newyork/train/45175987_close_wall.png
+../dataset/newyork/train/45243139.jpg	../dataset/newyork/train/45243139_wall.png	../dataset/newyork/train/45243139_close.png	../dataset/newyork/train/45243139_rooms.png	../dataset/newyork/train/45243139_close_wall.png
+../dataset/newyork/train/45293770.jpg	../dataset/newyork/train/45293770_wall.png	../dataset/newyork/train/45293770_close.png	../dataset/newyork/train/45293770_rooms.png	../dataset/newyork/train/45293770_close_wall.png
+../dataset/newyork/train/45321949.jpg	../dataset/newyork/train/45321949_wall.png	../dataset/newyork/train/45321949_close.png	../dataset/newyork/train/45321949_rooms.png	../dataset/newyork/train/45321949_close_wall.png
+../dataset/newyork/train/45352198.jpg	../dataset/newyork/train/45352198_wall.png	../dataset/newyork/train/45352198_close.png	../dataset/newyork/train/45352198_rooms.png	../dataset/newyork/train/45352198_close_wall.png
+../dataset/newyork/train/45552859.jpg	../dataset/newyork/train/45552859_wall.png	../dataset/newyork/train/45552859_close.png	../dataset/newyork/train/45552859_rooms.png	../dataset/newyork/train/45552859_close_wall.png
+../dataset/newyork/train/45562633.jpg	../dataset/newyork/train/45562633_wall.png	../dataset/newyork/train/45562633_close.png	../dataset/newyork/train/45562633_rooms.png	../dataset/newyork/train/45562633_close_wall.png
+../dataset/newyork/train/45591070.jpg	../dataset/newyork/train/45591070_wall.png	../dataset/newyork/train/45591070_close.png	../dataset/newyork/train/45591070_rooms.png	../dataset/newyork/train/45591070_close_wall.png
+../dataset/newyork/train/45591514.jpg	../dataset/newyork/train/45591514_wall.png	../dataset/newyork/train/45591514_close.png	../dataset/newyork/train/45591514_rooms.png	../dataset/newyork/train/45591514_close_wall.png
+../dataset/newyork/train/45608287.jpg	../dataset/newyork/train/45608287_wall.png	../dataset/newyork/train/45608287_close.png	../dataset/newyork/train/45608287_rooms.png	../dataset/newyork/train/45608287_close_wall.png
+../dataset/newyork/train/45613198.jpg	../dataset/newyork/train/45613198_wall.png	../dataset/newyork/train/45613198_close.png	../dataset/newyork/train/45613198_rooms.png	../dataset/newyork/train/45613198_close_wall.png
+../dataset/newyork/train/45633769.jpg	../dataset/newyork/train/45633769_wall.png	../dataset/newyork/train/45633769_close.png	../dataset/newyork/train/45633769_rooms.png	../dataset/newyork/train/45633769_close_wall.png
+../dataset/newyork/train/45665893.jpg	../dataset/newyork/train/45665893_wall.png	../dataset/newyork/train/45665893_close.png	../dataset/newyork/train/45665893_rooms.png	../dataset/newyork/train/45665893_close_wall.png
+../dataset/newyork/train/45708481.jpg	../dataset/newyork/train/45708481_wall.png	../dataset/newyork/train/45708481_close.png	../dataset/newyork/train/45708481_rooms.png	../dataset/newyork/train/45708481_close_wall.png
+../dataset/newyork/train/45714223.jpg	../dataset/newyork/train/45714223_wall.png	../dataset/newyork/train/45714223_close.png	../dataset/newyork/train/45714223_rooms.png	../dataset/newyork/train/45714223_close_wall.png
+../dataset/newyork/train/45716029.jpg	../dataset/newyork/train/45716029_wall.png	../dataset/newyork/train/45716029_close.png	../dataset/newyork/train/45716029_rooms.png	../dataset/newyork/train/45716029_close_wall.png
+../dataset/newyork/train/45719912.jpg	../dataset/newyork/train/45719912_wall.png	../dataset/newyork/train/45719912_close.png	../dataset/newyork/train/45719912_rooms.png	../dataset/newyork/train/45719912_close_wall.png
+../dataset/newyork/train/45720001.jpg	../dataset/newyork/train/45720001_wall.png	../dataset/newyork/train/45720001_close.png	../dataset/newyork/train/45720001_rooms.png	../dataset/newyork/train/45720001_close_wall.png
+../dataset/newyork/train/45720007.jpg	../dataset/newyork/train/45720007_wall.png	../dataset/newyork/train/45720007_close.png	../dataset/newyork/train/45720007_rooms.png	../dataset/newyork/train/45720007_close_wall.png
+../dataset/newyork/train/45724006.jpg	../dataset/newyork/train/45724006_wall.png	../dataset/newyork/train/45724006_close.png	../dataset/newyork/train/45724006_rooms.png	../dataset/newyork/train/45724006_close_wall.png
+../dataset/newyork/train/45724069.jpg	../dataset/newyork/train/45724069_wall.png	../dataset/newyork/train/45724069_close.png	../dataset/newyork/train/45724069_rooms.png	../dataset/newyork/train/45724069_close_wall.png
+../dataset/newyork/train/45724129.jpg	../dataset/newyork/train/45724129_wall.png	../dataset/newyork/train/45724129_close.png	../dataset/newyork/train/45724129_rooms.png	../dataset/newyork/train/45724129_close_wall.png
+../dataset/newyork/train/45724327.jpg	../dataset/newyork/train/45724327_wall.png	../dataset/newyork/train/45724327_close.png	../dataset/newyork/train/45724327_rooms.png	../dataset/newyork/train/45724327_close_wall.png
+../dataset/newyork/train/45724339.jpg	../dataset/newyork/train/45724339_wall.png	../dataset/newyork/train/45724339_close.png	../dataset/newyork/train/45724339_rooms.png	../dataset/newyork/train/45724339_close_wall.png
+../dataset/newyork/train/45724342.jpg	../dataset/newyork/train/45724342_wall.png	../dataset/newyork/train/45724342_close.png	../dataset/newyork/train/45724342_rooms.png	../dataset/newyork/train/45724342_close_wall.png
+../dataset/newyork/train/45724345.jpg	../dataset/newyork/train/45724345_wall.png	../dataset/newyork/train/45724345_close.png	../dataset/newyork/train/45724345_rooms.png	../dataset/newyork/train/45724345_close_wall.png
+../dataset/newyork/train/45724375.jpg	../dataset/newyork/train/45724375_wall.png	../dataset/newyork/train/45724375_close.png	../dataset/newyork/train/45724375_rooms.png	../dataset/newyork/train/45724375_close_wall.png
+../dataset/newyork/train/45724399.jpg	../dataset/newyork/train/45724399_wall.png	../dataset/newyork/train/45724399_close.png	../dataset/newyork/train/45724399_rooms.png	../dataset/newyork/train/45724399_close_wall.png
+../dataset/newyork/train/45724414.jpg	../dataset/newyork/train/45724414_wall.png	../dataset/newyork/train/45724414_close.png	../dataset/newyork/train/45724414_rooms.png	../dataset/newyork/train/45724414_close_wall.png
+../dataset/newyork/train/45724468.jpg	../dataset/newyork/train/45724468_wall.png	../dataset/newyork/train/45724468_close.png	../dataset/newyork/train/45724468_rooms.png	../dataset/newyork/train/45724468_close_wall.png
+../dataset/newyork/train/45725026.jpg	../dataset/newyork/train/45725026_wall.png	../dataset/newyork/train/45725026_close.png	../dataset/newyork/train/45725026_rooms.png	../dataset/newyork/train/45725026_close_wall.png
+../dataset/newyork/train/45725035.jpg	../dataset/newyork/train/45725035_wall.png	../dataset/newyork/train/45725035_close.png	../dataset/newyork/train/45725035_rooms.png	../dataset/newyork/train/45725035_close_wall.png
+../dataset/newyork/train/45728986.jpg	../dataset/newyork/train/45728986_wall.png	../dataset/newyork/train/45728986_close.png	../dataset/newyork/train/45728986_rooms.png	../dataset/newyork/train/45728986_close_wall.png
+../dataset/newyork/train/45731434.jpg	../dataset/newyork/train/45731434_wall.png	../dataset/newyork/train/45731434_close.png	../dataset/newyork/train/45731434_rooms.png	../dataset/newyork/train/45731434_close_wall.png
+../dataset/newyork/train/45737995.jpg	../dataset/newyork/train/45737995_wall.png	../dataset/newyork/train/45737995_close.png	../dataset/newyork/train/45737995_rooms.png	../dataset/newyork/train/45737995_close_wall.png
+../dataset/newyork/train/45740521.jpg	../dataset/newyork/train/45740521_wall.png	../dataset/newyork/train/45740521_close.png	../dataset/newyork/train/45740521_rooms.png	../dataset/newyork/train/45740521_close_wall.png
+../dataset/newyork/train/45740536.jpg	../dataset/newyork/train/45740536_wall.png	../dataset/newyork/train/45740536_close.png	../dataset/newyork/train/45740536_rooms.png	../dataset/newyork/train/45740536_close_wall.png
+../dataset/newyork/train/45740878.jpg	../dataset/newyork/train/45740878_wall.png	../dataset/newyork/train/45740878_close.png	../dataset/newyork/train/45740878_rooms.png	../dataset/newyork/train/45740878_close_wall.png
+../dataset/newyork/train/45740968.jpg	../dataset/newyork/train/45740968_wall.png	../dataset/newyork/train/45740968_close.png	../dataset/newyork/train/45740968_rooms.png	../dataset/newyork/train/45740968_close_wall.png
+../dataset/newyork/train/45741076.jpg	../dataset/newyork/train/45741076_wall.png	../dataset/newyork/train/45741076_close.png	../dataset/newyork/train/45741076_rooms.png	../dataset/newyork/train/45741076_close_wall.png
+../dataset/newyork/train/45741079.jpg	../dataset/newyork/train/45741079_wall.png	../dataset/newyork/train/45741079_close.png	../dataset/newyork/train/45741079_rooms.png	../dataset/newyork/train/45741079_close_wall.png
+../dataset/newyork/train/45741118.jpg	../dataset/newyork/train/45741118_wall.png	../dataset/newyork/train/45741118_close.png	../dataset/newyork/train/45741118_rooms.png	../dataset/newyork/train/45741118_close_wall.png
+../dataset/newyork/train/45741127.jpg	../dataset/newyork/train/45741127_wall.png	../dataset/newyork/train/45741127_close.png	../dataset/newyork/train/45741127_rooms.png	../dataset/newyork/train/45741127_close_wall.png
+../dataset/newyork/train/45743284.jpg	../dataset/newyork/train/45743284_wall.png	../dataset/newyork/train/45743284_close.png	../dataset/newyork/train/45743284_rooms.png	../dataset/newyork/train/45743284_close_wall.png
+../dataset/newyork/train/45756454.jpg	../dataset/newyork/train/45756454_wall.png	../dataset/newyork/train/45756454_close.png	../dataset/newyork/train/45756454_rooms.png	../dataset/newyork/train/45756454_close_wall.png
+../dataset/newyork/train/45763060.jpg	../dataset/newyork/train/45763060_wall.png	../dataset/newyork/train/45763060_close.png	../dataset/newyork/train/45763060_rooms.png	../dataset/newyork/train/45763060_close_wall.png
+../dataset/newyork/train/45764671.jpg	../dataset/newyork/train/45764671_wall.png	../dataset/newyork/train/45764671_close.png	../dataset/newyork/train/45764671_rooms.png	../dataset/newyork/train/45764671_close_wall.png
+../dataset/newyork/train/45764830.jpg	../dataset/newyork/train/45764830_wall.png	../dataset/newyork/train/45764830_close.png	../dataset/newyork/train/45764830_rooms.png	../dataset/newyork/train/45764830_close_wall.png
+../dataset/newyork/train/45765070.jpg	../dataset/newyork/train/45765070_wall.png	../dataset/newyork/train/45765070_close.png	../dataset/newyork/train/45765070_rooms.png	../dataset/newyork/train/45765070_close_wall.png
+../dataset/newyork/train/45765874.jpg	../dataset/newyork/train/45765874_wall.png	../dataset/newyork/train/45765874_close.png	../dataset/newyork/train/45765874_rooms.png	../dataset/newyork/train/45765874_close_wall.png
+../dataset/newyork/train/45774964.jpg	../dataset/newyork/train/45774964_wall.png	../dataset/newyork/train/45774964_close.png	../dataset/newyork/train/45774964_rooms.png	../dataset/newyork/train/45774964_close_wall.png
+../dataset/newyork/train/45775138.jpg	../dataset/newyork/train/45775138_wall.png	../dataset/newyork/train/45775138_close.png	../dataset/newyork/train/45775138_rooms.png	../dataset/newyork/train/45775138_close_wall.png
+../dataset/newyork/train/45775222.jpg	../dataset/newyork/train/45775222_wall.png	../dataset/newyork/train/45775222_close.png	../dataset/newyork/train/45775222_rooms.png	../dataset/newyork/train/45775222_close_wall.png
+../dataset/newyork/train/45775225.jpg	../dataset/newyork/train/45775225_wall.png	../dataset/newyork/train/45775225_close.png	../dataset/newyork/train/45775225_rooms.png	../dataset/newyork/train/45775225_close_wall.png
+../dataset/newyork/train/45775354.jpg	../dataset/newyork/train/45775354_wall.png	../dataset/newyork/train/45775354_close.png	../dataset/newyork/train/45775354_rooms.png	../dataset/newyork/train/45775354_close_wall.png
+../dataset/newyork/train/45775501.jpg	../dataset/newyork/train/45775501_wall.png	../dataset/newyork/train/45775501_close.png	../dataset/newyork/train/45775501_rooms.png	../dataset/newyork/train/45775501_close_wall.png
+../dataset/newyork/train/45775504.jpg	../dataset/newyork/train/45775504_wall.png	../dataset/newyork/train/45775504_close.png	../dataset/newyork/train/45775504_rooms.png	../dataset/newyork/train/45775504_close_wall.png
+../dataset/newyork/train/45775894.jpg	../dataset/newyork/train/45775894_wall.png	../dataset/newyork/train/45775894_close.png	../dataset/newyork/train/45775894_rooms.png	../dataset/newyork/train/45775894_close_wall.png
+../dataset/newyork/train/45777601.jpg	../dataset/newyork/train/45777601_wall.png	../dataset/newyork/train/45777601_close.png	../dataset/newyork/train/45777601_rooms.png	../dataset/newyork/train/45777601_close_wall.png
+../dataset/newyork/train/45777694.jpg	../dataset/newyork/train/45777694_wall.png	../dataset/newyork/train/45777694_close.png	../dataset/newyork/train/45777694_rooms.png	../dataset/newyork/train/45777694_close_wall.png
+../dataset/newyork/train/45780106.jpg	../dataset/newyork/train/45780106_wall.png	../dataset/newyork/train/45780106_close.png	../dataset/newyork/train/45780106_rooms.png	../dataset/newyork/train/45780106_close_wall.png
+../dataset/newyork/train/45780376.jpg	../dataset/newyork/train/45780376_wall.png	../dataset/newyork/train/45780376_close.png	../dataset/newyork/train/45780376_rooms.png	../dataset/newyork/train/45780376_close_wall.png
+../dataset/newyork/train/45781483.jpg	../dataset/newyork/train/45781483_wall.png	../dataset/newyork/train/45781483_close.png	../dataset/newyork/train/45781483_rooms.png	../dataset/newyork/train/45781483_close_wall.png
+../dataset/newyork/train/45783298.jpg	../dataset/newyork/train/45783298_wall.png	../dataset/newyork/train/45783298_close.png	../dataset/newyork/train/45783298_rooms.png	../dataset/newyork/train/45783298_close_wall.png
+../dataset/newyork/train/45783466.jpg	../dataset/newyork/train/45783466_wall.png	../dataset/newyork/train/45783466_close.png	../dataset/newyork/train/45783466_rooms.png	../dataset/newyork/train/45783466_close_wall.png
+../dataset/newyork/train/45.jpg	../dataset/newyork/train/45_wall.png	../dataset/newyork/train/45_close.png	../dataset/newyork/train/45_rooms.png	../dataset/newyork/train/45_close_wall.png
+../dataset/newyork/train/46452431.jpg	../dataset/newyork/train/46452431_wall.png	../dataset/newyork/train/46452431_close.png	../dataset/newyork/train/46452431_rooms.png	../dataset/newyork/train/46452431_close_wall.png
+../dataset/newyork/train/46678955.jpg	../dataset/newyork/train/46678955_wall.png	../dataset/newyork/train/46678955_close.png	../dataset/newyork/train/46678955_rooms.png	../dataset/newyork/train/46678955_close_wall.png
+../dataset/newyork/train/46781618.jpg	../dataset/newyork/train/46781618_wall.png	../dataset/newyork/train/46781618_close.png	../dataset/newyork/train/46781618_rooms.png	../dataset/newyork/train/46781618_close_wall.png
+../dataset/newyork/train/46807061.jpg	../dataset/newyork/train/46807061_wall.png	../dataset/newyork/train/46807061_close.png	../dataset/newyork/train/46807061_rooms.png	../dataset/newyork/train/46807061_close_wall.png
+../dataset/newyork/train/46.jpg	../dataset/newyork/train/46_wall.png	../dataset/newyork/train/46_close.png	../dataset/newyork/train/46_rooms.png	../dataset/newyork/train/46_close_wall.png
+../dataset/newyork/train/47073524.jpg	../dataset/newyork/train/47073524_wall.png	../dataset/newyork/train/47073524_close.png	../dataset/newyork/train/47073524_rooms.png	../dataset/newyork/train/47073524_close_wall.png
+../dataset/newyork/train/47185109.jpg	../dataset/newyork/train/47185109_wall.png	../dataset/newyork/train/47185109_close.png	../dataset/newyork/train/47185109_rooms.png	../dataset/newyork/train/47185109_close_wall.png
+../dataset/newyork/train/47236967.jpg	../dataset/newyork/train/47236967_wall.png	../dataset/newyork/train/47236967_close.png	../dataset/newyork/train/47236967_rooms.png	../dataset/newyork/train/47236967_close_wall.png
+../dataset/newyork/train/47325578.jpg	../dataset/newyork/train/47325578_wall.png	../dataset/newyork/train/47325578_close.png	../dataset/newyork/train/47325578_rooms.png	../dataset/newyork/train/47325578_close_wall.png
+../dataset/newyork/train/47360870.jpg	../dataset/newyork/train/47360870_wall.png	../dataset/newyork/train/47360870_close.png	../dataset/newyork/train/47360870_rooms.png	../dataset/newyork/train/47360870_close_wall.png
+../dataset/newyork/train/47429369.jpg	../dataset/newyork/train/47429369_wall.png	../dataset/newyork/train/47429369_close.png	../dataset/newyork/train/47429369_rooms.png	../dataset/newyork/train/47429369_close_wall.png
+../dataset/newyork/train/47464136.jpg	../dataset/newyork/train/47464136_wall.png	../dataset/newyork/train/47464136_close.png	../dataset/newyork/train/47464136_rooms.png	../dataset/newyork/train/47464136_close_wall.png
+../dataset/newyork/train/47464142.jpg	../dataset/newyork/train/47464142_wall.png	../dataset/newyork/train/47464142_close.png	../dataset/newyork/train/47464142_rooms.png	../dataset/newyork/train/47464142_close_wall.png
+../dataset/newyork/train/47464151.jpg	../dataset/newyork/train/47464151_wall.png	../dataset/newyork/train/47464151_close.png	../dataset/newyork/train/47464151_rooms.png	../dataset/newyork/train/47464151_close_wall.png
+../dataset/newyork/train/47465963.jpg	../dataset/newyork/train/47465963_wall.png	../dataset/newyork/train/47465963_close.png	../dataset/newyork/train/47465963_rooms.png	../dataset/newyork/train/47465963_close_wall.png
+../dataset/newyork/train/47484836.jpg	../dataset/newyork/train/47484836_wall.png	../dataset/newyork/train/47484836_close.png	../dataset/newyork/train/47484836_rooms.png	../dataset/newyork/train/47484836_close_wall.png
+../dataset/newyork/train/47489621.jpg	../dataset/newyork/train/47489621_wall.png	../dataset/newyork/train/47489621_close.png	../dataset/newyork/train/47489621_rooms.png	../dataset/newyork/train/47489621_close_wall.png
+../dataset/newyork/train/47489648.jpg	../dataset/newyork/train/47489648_wall.png	../dataset/newyork/train/47489648_close.png	../dataset/newyork/train/47489648_rooms.png	../dataset/newyork/train/47489648_close_wall.png
+../dataset/newyork/train/47490062.jpg	../dataset/newyork/train/47490062_wall.png	../dataset/newyork/train/47490062_close.png	../dataset/newyork/train/47490062_rooms.png	../dataset/newyork/train/47490062_close_wall.png
+../dataset/newyork/train/47492936.jpg	../dataset/newyork/train/47492936_wall.png	../dataset/newyork/train/47492936_close.png	../dataset/newyork/train/47492936_rooms.png	../dataset/newyork/train/47492936_close_wall.png
+../dataset/newyork/train/47499269.jpg	../dataset/newyork/train/47499269_wall.png	../dataset/newyork/train/47499269_close.png	../dataset/newyork/train/47499269_rooms.png	../dataset/newyork/train/47499269_close_wall.png
+../dataset/newyork/train/47499620.jpg	../dataset/newyork/train/47499620_wall.png	../dataset/newyork/train/47499620_close.png	../dataset/newyork/train/47499620_rooms.png	../dataset/newyork/train/47499620_close_wall.png
+../dataset/newyork/train/47503913.jpg	../dataset/newyork/train/47503913_wall.png	../dataset/newyork/train/47503913_close.png	../dataset/newyork/train/47503913_rooms.png	../dataset/newyork/train/47503913_close_wall.png
+../dataset/newyork/train/47505359.jpg	../dataset/newyork/train/47505359_wall.png	../dataset/newyork/train/47505359_close.png	../dataset/newyork/train/47505359_rooms.png	../dataset/newyork/train/47505359_close_wall.png
+../dataset/newyork/train/47508827.jpg	../dataset/newyork/train/47508827_wall.png	../dataset/newyork/train/47508827_close.png	../dataset/newyork/train/47508827_rooms.png	../dataset/newyork/train/47508827_close_wall.png
+../dataset/newyork/train/47514899.jpg	../dataset/newyork/train/47514899_wall.png	../dataset/newyork/train/47514899_close.png	../dataset/newyork/train/47514899_rooms.png	../dataset/newyork/train/47514899_close_wall.png
+../dataset/newyork/train/47514920.jpg	../dataset/newyork/train/47514920_wall.png	../dataset/newyork/train/47514920_close.png	../dataset/newyork/train/47514920_rooms.png	../dataset/newyork/train/47514920_close_wall.png
+../dataset/newyork/train/47534687.jpg	../dataset/newyork/train/47534687_wall.png	../dataset/newyork/train/47534687_close.png	../dataset/newyork/train/47534687_rooms.png	../dataset/newyork/train/47534687_close_wall.png
+../dataset/newyork/train/4.jpg	../dataset/newyork/train/4_wall.png	../dataset/newyork/train/4_close.png	../dataset/newyork/train/4_rooms.png	../dataset/newyork/train/4_close_wall.png
+../dataset/newyork/train/50.jpg	../dataset/newyork/train/50_wall.png	../dataset/newyork/train/50_close.png	../dataset/newyork/train/50_rooms.png	../dataset/newyork/train/50_close_wall.png
+../dataset/newyork/train/52.jpg	../dataset/newyork/train/52_wall.png	../dataset/newyork/train/52_close.png	../dataset/newyork/train/52_rooms.png	../dataset/newyork/train/52_close_wall.png
+../dataset/newyork/train/57.jpg	../dataset/newyork/train/57_wall.png	../dataset/newyork/train/57_close.png	../dataset/newyork/train/57_rooms.png	../dataset/newyork/train/57_close_wall.png
+../dataset/newyork/train/7.jpg	../dataset/newyork/train/7_wall.png	../dataset/newyork/train/7_close.png	../dataset/newyork/train/7_rooms.png	../dataset/newyork/train/7_close_wall.png

deepfloorplan_inference.py

@@ -0,0 +1,55 @@
+import os
+import numpy as np
+import tensorflow as tf
+from PIL import Image
+import imageio
+from net import Network
+from utils.rgb_ind_convertor import ind2rgb, floorplan_fuse_map
+
+class DeepFloorPlanModel:
+    def __init__(self, model_dir='pretrained', input_size=(512, 512)):
+        self.input_size = input_size
+        self.model_dir = model_dir
+        self._build_graph()
+        self._load_weights()
+
+    def _build_graph(self):
+        tf.compat.v1.reset_default_graph()
+        self.sess = tf.compat.v1.Session()
+        self.x = tf.compat.v1.placeholder(shape=[1, self.input_size[0], self.input_size[1], 3], dtype=tf.float32, name='inputs')
+        self.network = Network()
+        logits1, logits2 = self.network.forward(self.x, init_with_pretrain_vgg=False)
+        self.rooms = self.network.convert_one_hot_to_image(logits1, act='softmax', dtype='int')
+        self.close_walls = self.network.convert_one_hot_to_image(logits2, act='softmax', dtype='int')
+        self.sess.run(tf.compat.v1.global_variables_initializer())
+        self.sess.run(tf.compat.v1.local_variables_initializer())
+        self.saver = tf.compat.v1.train.Saver()
+
+    def _load_weights(self):
+        ckpt = tf.train.latest_checkpoint(self.model_dir)
+        if ckpt is None:
+            raise FileNotFoundError(f"No checkpoint found in {self.model_dir}")
+        self.saver.restore(self.sess, ckpt)
+
+    def predict(self, image):
+        # Accepts a numpy array or PIL image, returns a numpy array (segmentation mask)
+        if isinstance(image, Image.Image):
+            image = np.array(image)
+        if image.shape[-1] == 4:
+            image = image[..., :3]
+        im_resized = np.array(Image.fromarray(image).resize(self.input_size, Image.BICUBIC)) / 255.0
+        im_resized = im_resized.astype(np.float32)
+        im_resized = np.reshape(im_resized, (1, self.input_size[0], self.input_size[1], 3))
+        out1, out2 = self.sess.run([self.rooms, self.close_walls], feed_dict={self.x: im_resized})
+        out1 = np.squeeze(out1)
+        out2 = np.squeeze(out2)
+        # Merge logic: set out1 pixels to 9/10 where out2==1/2
+        out1[out2==2] = 10
+        out1[out2==1] = 9
+        # Convert to RGB for visualization
+        out_rgb = ind2rgb(out1, color_map=floorplan_fuse_map)
+        out_rgb = out_rgb.astype(np.uint8)
+        return out_rgb
+
+    def close(self):
+        self.sess.close() 

demo.py

@@ -0,0 +1,89 @@
+import os
+import argparse
+import numpy as np
+import tensorflow as tf
+
+import imageio
+from PIL import Image
+
+from matplotlib import pyplot as plt
+
+os.environ['CUDA_VISIBLE_DEVICES'] = '0'
+
+os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'
+
+# input image path
+parser = argparse.ArgumentParser()
+
+parser.add_argument('--im_path', type=str, default='./demo/45765448.jpg',
+                    help='input image paths.')
+
+# color map
+floorplan_map = {
+	0: [255,255,255], # background
+	1: [192,192,224], # closet
+	2: [192,255,255], # batchroom/washroom
+	3: [224,255,192], # livingroom/kitchen/dining room
+	4: [255,224,128], # bedroom
+	5: [255,160, 96], # hall
+	6: [255,224,224], # balcony
+	7: [255,255,255], # not used
+	8: [255,255,255], # not used
+	9: [255, 60,128], # door & window
+	10:[  0,  0,  0]  # wall
+}
+
+def ind2rgb(ind_im, color_map=floorplan_map):
+	rgb_im = np.zeros((ind_im.shape[0], ind_im.shape[1], 3))
+
+	for i, rgb in color_map.items():
+		rgb_im[(ind_im==i)] = rgb
+
+	return rgb_im
+
+def main(args):
+	# load input
+	im = imageio.imread(args.im_path, mode='RGB')
+	im = im.astype(np.float32)
+	im = PIL.Image.fromarray(im).resize((512,512,3)) / 255.
+
+	# create tensorflow session
+	with tf.Session() as sess:
+		
+		# initialize
+		sess.run(tf.group(tf.global_variables_initializer(),
+					tf.local_variables_initializer()))
+
+		# restore pretrained model
+		saver = tf.train.import_meta_graph('./pretrained/pretrained_r3d.meta')
+		saver.restore(sess, './pretrained/pretrained_r3d')
+
+		# get default graph
+		graph = tf.get_default_graph()
+
+		# restore inputs & outpus tensor
+		x = graph.get_tensor_by_name('inputs:0')
+		room_type_logit = graph.get_tensor_by_name('Cast:0')
+		room_boundary_logit = graph.get_tensor_by_name('Cast_1:0')
+
+		# infer results
+		[room_type, room_boundary] = sess.run([room_type_logit, room_boundary_logit],\
+										feed_dict={x:im.reshape(1,512,512,3)})
+		room_type, room_boundary = np.squeeze(room_type), np.squeeze(room_boundary)
+
+		# merge results
+		floorplan = room_type.copy()
+		floorplan[room_boundary==1] = 9
+		floorplan[room_boundary==2] = 10
+		floorplan_rgb = ind2rgb(floorplan)
+
+		# plot results
+		plt.subplot(121)
+		plt.imshow(im)
+		plt.subplot(122)
+		plt.imshow(floorplan_rgb/255.)
+		plt.show()
+
+if __name__ == '__main__':
+	FLAGS, unparsed = parser.parse_known_args()
+	main(FLAGS)

main.py

@@ -0,0 +1,317 @@
+import argparse
+from net import *
+import os
+import time
+import random
+import numpy as np
+import tensorflow as tf
+import imageio
+from PIL import Image
+
+os.environ['CUDA_VISIBLE_DEVICES'] = GPU_ID
+
+os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'
+
+seed = 8964
+
+# input image path
+parser = argparse.ArgumentParser()
+
+parser.add_argument('--phase', type=str, default='Test',
+					help='Train/Test network.')
+
+class MODEL(Network):
+	"""docstring for MODEL"""
+	def __init__(self):
+		Network.__init__(self)
+		self.log_dir = 'pretrained'
+		self.eval_file = './dataset/r3d_test.txt'		
+		self.loss_type = 'balanced'
+
+	def convert_one_hot_to_image(self, one_hot, dtype='float', act=None):
+		if act == 'softmax':
+			one_hot = tf.nn.softmax(one_hot, axis=-1)
+
+		[n, h, w, c] = one_hot.shape.as_list()
+
+		im = tf.reshape(tf.argmax(one_hot, axis=-1), [n, h, w, 1])
+		if dtype == 'int':
+			im = tf.cast(im, dtype=tf.uint8)
+		else:
+			im = tf.cast(im, dtype=tf.float32)
+		return im
+
+	def cross_two_tasks_weight(self, y1, y2):
+		p1 = tf.reduce_sum(y1)
+		p2 = tf.reduce_sum(y2)
+
+		w1 = p2 / (p1 + p2)
+		w2 = p1 / (p1 + p2)
+
+		return w1, w2
+
+	def balanced_entropy(self, x, y):
+		# cliped_by_eps
+		eps = 1e-6
+		z = tf.nn.softmax(x)
+		cliped_z = tf.clip_by_value(z, eps, 1-eps)
+		log_z = tf.log(cliped_z)
+
+		num_classes = y.shape.as_list()[-1]
+		ind = tf.argmax(y, -1, output_type=tf.int32)
+		# ind = tf.reshape(ind, shape=[1, 512, 512, 1]) # for debugging
+
+		total = tf.reduce_sum(y) # total foreground pixels
+
+		m_c = [] # index mask
+		n_c = [] # each class foreground pixels
+		for c in range(num_classes):
+			m_c.append(tf.cast(tf.equal(ind, c), dtype=tf.int32))
+			n_c.append(tf.cast(tf.reduce_sum(m_c[-1]), dtype=tf.float32))
+
+		# compute count
+		c = []
+		for i in range(num_classes):
+			c.append(total - n_c[i])
+		tc = tf.add_n(c)
+
+		# use for compute loss
+		loss = 0.
+		for i in range(num_classes): 
+			w = c[i] / tc
+			m_c_one_hot = tf.one_hot((i*m_c[i]), num_classes, axis=-1)
+			y_c = m_c_one_hot*y
+
+			loss += w*tf.reduce_mean(-tf.reduce_sum(y_c*log_z, axis=1)) 
+
+		return (loss / num_classes) # mean 
+
+	def train(self, loader_dict, num_batch, max_step=40000):
+		images	 = loader_dict['images']
+		labels_r_hot = loader_dict['label_rooms']
+		labels_cw_hot  = loader_dict['label_boundaries']
+
+		max_ep = max_step // num_batch
+		print('max_step = {}, max_ep = {}, num_batch = {}'.format(max_step, max_ep, num_batch))
+
+		logits1, logits2 = self.forward(images, init_with_pretrain_vgg=False)
+
+		if self.loss_type == 'balanced':
+			# in-task loss balance
+			loss1 = self.balanced_entropy(logits1, labels_r_hot) # multi classes balance
+			loss2 = self.balanced_entropy(logits2, labels_cw_hot)
+		else:
+			loss1 = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits_v2(logits=logits1, labels=labels_r_hot, name='bce1'))
+			loss2 = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits_v2(logits=logits2, labels=labels_cw_hot, name='bce2'))
+
+		# compute cross loss balance weight
+		w1, w2 = self.cross_two_tasks_weight(labels_r_hot, labels_cw_hot)
+		loss = (w1*loss1 + w2*loss2)
+
+		optim = tf.train.AdamOptimizer(learning_rate=1e-4).minimize(loss, colocate_gradients_with_ops=True) # gradient ops assign to same device as forward ops
+
+		# # add image summary
+		# tf.summary.image('input', images)
+		# tf.summary.image('label_r', self.convert_one_hot_to_image(labels_r_hot))
+		# tf.summary.image('predict_room', self.convert_one_hot_to_image(logits1, act='softmax')) # room type to use argmax to visualize
+		# tf.summary.image('predict_close_wall', tf.nn.sigmoid(logits2)) # boundaries type to use argmax to visualize
+
+		# # add scalar summary
+		# tf.summary.scalar('bce', loss)
+
+		# define session
+		config = tf.ConfigProto(allow_soft_placement=True) 
+		config.gpu_options.allow_growth=True # prevent the program occupies all GPU memory
+		with tf.Session(config=config) as sess:
+			# init all variables in graph
+			sess.run(tf.group(tf.global_variables_initializer(),
+							tf.local_variables_initializer()))
+
+			# saver 
+			saver = tf.train.Saver(max_to_keep=10) 
+
+			# filewriter for log info
+			# log_dir = self.log_dir+'/run-%02d%02d-%02d%02d' % tuple(time.localtime(time.time()))[1:5]
+			# writer = tf.summary.FileWriter(log_dir)
+			# merged = tf.summary.merge_all()
+
+			# coordinator for queue runner
+			coord = tf.train.Coordinator()
+
+			# start queue 
+			threads = tf.train.start_queue_runners(sess=sess, coord=coord)
+
+			print("Start Training!")
+			total_times = 0			
+
+			for ep in range(max_ep): # epoch loop
+				for n in range(num_batch): # batch loop
+					tic = time.time()
+					# [loss_value, update_value, summaries] = sess.run([loss, optim, merged])	
+					[loss_value, update_value] = sess.run([loss, optim])	
+					duration = time.time()-tic
+
+					total_times += duration
+
+					step = int(ep*num_batch + n)
+					# write log 
+					print('step {}: loss = {:.3}; {:.2} data/sec, excuted {} minutes'.format(step,
+						loss_value, 1.0/duration, int(total_times/60)))
+					# writer.add_summary(summaries, global_step=step)
+				# save model parameters after 2 epoch training
+				if ep % 2 == 0:
+					saver.save(sess, self.log_dir+'/model', global_step=ep)
+					self.evaluate(sess=sess, epoch=ep)
+			saver.save(sess, self.log_dir+'/model', global_step=max_ep)
+			self.evaluate(sess=sess, epoch=max_ep)
+
+			# close session
+			coord.request_stop()
+			coord.join(threads)			
+			sess.close()	
+
+	def infer(self, save_dir='out', resize=True, merge=True):
+		print("generating test set of {}.... will save to [./{}]".format(self.eval_file, save_dir))
+		room_dir = os.path.join(save_dir, 'room')
+		close_wall_dir = os.path.join(save_dir, 'boundary')
+
+		if not os.path.exists(save_dir):
+			os.mkdir(save_dir)
+		if not os.path.exists(room_dir):
+			os.mkdir(room_dir)
+		if not os.path.exists(close_wall_dir):
+			os.mkdir(close_wall_dir)
+
+		x = tf.placeholder(shape=[1, 512, 512, 3], dtype=tf.float32)
+
+		logits1, logits2 = self.forward(x, init_with_pretrain_vgg=False)
+		rooms = self.convert_one_hot_to_image(logits1, act='softmax', dtype='int')
+		close_walls = self.convert_one_hot_to_image(logits2, act='softmax', dtype='int')
+
+		config = tf.ConfigProto(allow_soft_placement=True)
+		sess = tf.Session(config=config)
+		sess.run(tf.group(tf.global_variables_initializer(),
+						tf.local_variables_initializer()))
+
+		saver = tf.train.Saver() # restore all parameters
+		saver.restore(sess, save_path = tf.train.latest_checkpoint(self.log_dir))
+
+		# infer one by one
+		paths = open(self.eval_file, 'r').read().splitlines()
+		paths = [p.split('\t')[0] for p in paths]	
+		for p in paths:
+			im = imageio.imread(p, mode='RGB')  
+			im_x = imageio.imresize(im, (512,512,3)) / 255. # resize and normalize
+			im_x = np.reshape(im_x, (1,512,512,3))
+
+			[out1, out2] = sess.run([rooms, close_walls], feed_dict={x: im_x})
+			if resize:
+				# out1 = imresize(np.squeeze(out1), (im.shape[0], im.shape[1])) # resize back 
+				# out2 = imresize(np.squeeze(out2), (im.shape[0], im.shape[1])) # resize back 
+				out1_rgb = ind2rgb(np.squeeze(out1))
+				out1_rgb = imageio.imresize(out1_rgb, (im.shape[0], im.shape[1])) # resize back 
+				out2_rgb = ind2rgb(np.squeeze(out2), color_map=floorplan_boundary_map)
+				out2_rgb = imageio.imresize(out2_rgb, (im.shape[0], im.shape[1])) # resize back 
+			else:
+				out1_rgb = ind2rgb(np.squeeze(out1))
+				out2_rgb = ind2rgb(np.squeeze(out2), color_map=floorplan_boundary_map)
+
+			if merge:
+				out1 = np.squeeze(out1)
+				out2 = np.squeeze(out2)
+				out1[out2==2] = 10
+				out1[out2==1] = 9
+				# out3_rgb = ind2rgb(out1, color_map=floorplan_fuse_map_figure) # use for present
+				out3_rgb = ind2rgb(out1, color_map=floorplan_fuse_map) # use for present
+
+			name = p.split('/')[-1]	
+			save_path1 = os.path.join(room_dir, name.split('.jpg')[0]+'_rooms.png')		
+			save_path2 = os.path.join(close_wall_dir, name.split('.jpg')[0]+'_bd_rm.png')		
+			save_path3 = os.path.join(save_dir, name.split('.jpg')[0]+'_rooms.png')		
+
+			imageio.imwrite(save_path1, out1_rgb)
+			imageio.imwrite(save_path2, out2_rgb)
+			if merge:
+				imageio.imwrite(save_path3, out3_rgb)
+			# imsave(save_path4, out4)
+			
+			print('Saving prediction: {}'.format(name))	
+
+	def evaluate(self, sess, epoch, num_of_classes=11):
+		x = tf.placeholder(shape=[1, 512, 512, 3], dtype=tf.float32)
+		logits1, logits2 = self.forward(x, init_with_pretrain_vgg=False)
+		predict_bd = self.convert_one_hot_to_image(logits2, act='softmax', dtype='int')
+		predict_room = self.convert_one_hot_to_image(logits1, act='softmax', dtype='int')
+
+		paths = open(self.eval_file, 'r').read().splitlines()
+		image_paths = [p.split('\t')[0] for p in paths] # image 
+		gt2_paths = [p.split('\t')[2] for p in paths] # 2 denote doors (and windows)
+		gt3_paths = [p.split('\t')[3] for p in paths] # 3 denote rooms
+		gt4_paths = [p.split('\t')[-1] for p in paths] # last one denote close wall
+
+		n = len(paths)
+
+		hist = np.zeros((num_of_classes, num_of_classes))
+		for i in range(n):
+			im = imageio.imread(image_paths[i], mode='RGB')
+			# for fuse label
+			dd = imageio.imread(gt2_paths[i], mode='L')
+			rr = imageio.imread(gt3_paths[i], mode='RGB')
+			cw = imageio.imread(gt4_paths[i], mode='L')
+
+			im = imageio.imresize(im, (512, 512, 3)) / 255. # normalize input image
+			im = np.reshape(im, (1,512,512,3))
+			# merge label
+			rr = imageio.imresize(rr, (512, 512, 3))
+			rr_ind = rgb2ind(rr)
+			cw = imageio.imresize(cw, (512, 512)) / 255
+			dd = imageio.imresize(dd, (512, 512)) / 255
+			cw = (cw>0.5).astype(np.uint8)
+			dd = (dd>0.5).astype(np.uint8)
+			rr_ind[cw==1] = 10
+			rr_ind[dd==1] = 9
+
+			# merge prediciton
+			rm_ind, bd_ind = sess.run([predict_room, predict_bd], feed_dict={x: im})
+			rm_ind = np.squeeze(rm_ind)
+			bd_ind = np.squeeze(bd_ind)
+			rm_ind[bd_ind==2] = 10
+			rm_ind[bd_ind==1] = 9
+
+			hist += fast_hist(rm_ind.flatten(), rr_ind.flatten(), num_of_classes)
+
+		overall_acc = np.diag(hist).sum() / hist.sum()
+		mean_acc = np.diag(hist) / (hist.sum(1) + 1e-6)
+		# iu = np.diag(hist) / (hist.sum(1) + 1e-6 + hist.sum(0) - np.diag(hist))			
+		mean_acc9 = (np.nansum(mean_acc[:7])+mean_acc[-2]+mean_acc[-1]) / 9.
+
+		file = open('EVAL_'+self.log_dir, 'a')
+		print('Model at epoch {}: overall accuracy = {:.4}, mean_acc = {:.4}'.format(epoch, overall_acc, mean_acc9))
+		for i in range(mean_acc.shape[0]):
+			if i not in [7 ,8]: # ingore class 7 & 8 
+				print('\t\tepoch {}: {}th label: accuracy = {:.4}'.format(epoch, i, mean_acc[i]))		
+		file.close()
+
+def main(args):
+	tf.set_random_seed(seed)
+	np.random.seed(seed)
+	random.seed(seed)
+
+	model = MODEL()
+
+	if args.phase.lower() == 'train':
+		loader_dict, num_batch = data_loader_bd_rm_from_tfrecord(batch_size=1)
+
+		# START TRAINING
+		tic = time.time()
+		model.train(loader_dict, num_batch)
+		toc = time.time()
+		print('total training + evaluation time = {} minutes'.format((toc-tic)/60))
+	elif args.phase.lower() == 'test':	
+		model.infer()
+	else:
+		pass
+
+if __name__ == '__main__':
+	FLAGS, unparsed = parser.parse_known_args()
+	main(FLAGS)

net.py

@@ -0,0 +1,362 @@
+import numpy as np
+import tensorflow as tf # using tf 1.10.1
+
+from tensorflow.contrib.slim.nets import vgg
+
+import os
+import sys
+import glob
+import time
+import random
+
+from scipy import ndimage
+import imageio
+from PIL import Image
+
+sys.path.append('./utils/')
+from rgb_ind_convertor import *
+from util import fast_hist
+from tf_record import read_record, read_bd_rm_record
+
+GPU_ID = '0'
+
+def data_loader_bd_rm_from_tfrecord(batch_size=1):
+	paths = open('../dataset/r3d_train.txt', 'r').read().splitlines()
+
+	loader_dict = read_bd_rm_record('../dataset/r3d.tfrecords', batch_size=batch_size, size=512)
+
+	num_batch = len(paths) // batch_size
+
+	return loader_dict, num_batch
+
+class Network(object):
+	"""docstring for Network"""
+	def __init__(self, dtype=tf.float32):
+		print('Initial nn network object...')
+		self.dtype = dtype
+		self.pre_train_restore_map = {'vgg_16/conv1/conv1_1/weights':'FNet/conv1_1/W', # {'checkpoint_scope_var_name':'current_scope_var_name'} shape must be the same
+									'vgg_16/conv1/conv1_1/biases':'FNet/conv1_1/b',	
+									'vgg_16/conv1/conv1_2/weights':'FNet/conv1_2/W',
+									'vgg_16/conv1/conv1_2/biases':'FNet/conv1_2/b',	
+									'vgg_16/conv2/conv2_1/weights':'FNet/conv2_1/W',
+									'vgg_16/conv2/conv2_1/biases':'FNet/conv2_1/b',	
+									'vgg_16/conv2/conv2_2/weights':'FNet/conv2_2/W',
+									'vgg_16/conv2/conv2_2/biases':'FNet/conv2_2/b',	
+									'vgg_16/conv3/conv3_1/weights':'FNet/conv3_1/W',
+									'vgg_16/conv3/conv3_1/biases':'FNet/conv3_1/b',	
+									'vgg_16/conv3/conv3_2/weights':'FNet/conv3_2/W',
+									'vgg_16/conv3/conv3_2/biases':'FNet/conv3_2/b',	
+									'vgg_16/conv3/conv3_3/weights':'FNet/conv3_3/W',
+									'vgg_16/conv3/conv3_3/biases':'FNet/conv3_3/b',	
+									'vgg_16/conv4/conv4_1/weights':'FNet/conv4_1/W',
+									'vgg_16/conv4/conv4_1/biases':'FNet/conv4_1/b',	
+									'vgg_16/conv4/conv4_2/weights':'FNet/conv4_2/W',
+									'vgg_16/conv4/conv4_2/biases':'FNet/conv4_2/b',	
+									'vgg_16/conv4/conv4_3/weights':'FNet/conv4_3/W',
+									'vgg_16/conv4/conv4_3/biases':'FNet/conv4_3/b',	
+									'vgg_16/conv5/conv5_1/weights':'FNet/conv5_1/W',
+									'vgg_16/conv5/conv5_1/biases':'FNet/conv5_1/b',	
+									'vgg_16/conv5/conv5_2/weights':'FNet/conv5_2/W',
+									'vgg_16/conv5/conv5_2/biases':'FNet/conv5_2/b',	
+									'vgg_16/conv5/conv5_3/weights':'FNet/conv5_3/W',
+									'vgg_16/conv5/conv5_3/biases':'FNet/conv5_3/b'} 
+
+	# basic layer 
+	def _he_uniform(self, shape, regularizer=None, trainable=None, name=None):
+		name = 'W' if name is None else name+'/W'
+
+		# size = (k_h, k_w, in_dim, out_dim)
+		kernel_size = np.prod(shape[:2]) # k_h*k_w
+		fan_in = shape[-2]*kernel_size  # fan_out = shape[-1]*kernel_size
+
+		# compute the scale value
+		s = np.sqrt(1. /fan_in)
+
+		# create variable and specific GPU device
+		with tf.device('/device:GPU:'+GPU_ID):
+			w = tf.get_variable(name, shape, dtype=self.dtype,
+							initializer=tf.random_uniform_initializer(minval=-s, maxval=s),
+							regularizer=regularizer, trainable=trainable)
+
+		return w
+
+	def _constant(self, shape, value=0, regularizer=None, trainable=None, name=None):
+		name = 'b' if name is None else name+'/b'
+
+		with tf.device('/device:GPU:'+GPU_ID):
+			b = tf.get_variable(name, shape, dtype=self.dtype,
+							initializer=tf.constant_initializer(value=value),
+							regularizer=regularizer, trainable=trainable)
+
+		return b
+
+	def _conv2d(self, tensor, dim, size=3, stride=1, rate=1, pad='SAME', act='relu', norm='none', G=16, bias=True, name='conv'):
+		"""pre activate => norm => conv
+		"""
+		in_dim = tensor.shape.as_list()[-1]
+		size = size if isinstance(size, (tuple, list)) else [size, size]
+		stride = stride if isinstance(stride, (tuple, list)) else [1, stride, stride, 1]
+		rate = rate if isinstance(rate, (tuple, list)) else [1, rate, rate, 1]
+		kernel_shape = [size[0], size[1], in_dim, dim]
+
+		w = self._he_uniform(kernel_shape, name=name)
+		b = self._constant(dim, name=name) if bias else 0
+
+		if act == 'relu':
+			tensor = tf.nn.relu(tensor, name=name+'/relu')
+		elif act == 'sigmoid':
+			tensor = tf.nn.sigmoid(tensor, name=name+'/sigmoid')
+		elif act == 'softplus':
+			tensor = tf.nn.softplus(tensor, name=name+'/softplus')
+		elif act =='leaky_relu':
+			tensor = tf.nn.leaky_relu(tensor, name=name+'/leaky_relu')
+		else:
+			norm = 'none'
+
+		if norm == 'gn': # group normalization after acitvation
+			# normalize
+			# tranpose: [bs, h, w, c] to [bs, c, h, w] following the paper
+			x = tf.transpose(tensor, [0, 3, 1, 2])
+			N, C, H, W = x.get_shape().as_list()
+			G = min(G, C)
+			x = tf.reshape(x, [-1, G, C // G, H, W])
+			mean, var = tf.nn.moments(x, [2, 3, 4], keep_dims=True)
+			x = (x - mean) / tf.sqrt(var + 1e-6)
+
+			# per channel gamma and beta
+			with tf.device('/device:GPU:'+GPU_ID):
+				gamma = tf.get_variable(name+'/gamma', [C], dtype=self.dtype, initializer=tf.constant_initializer(1.0))
+				beta = tf.get_variable(name+'/beta', [C], dtype=self.dtype, initializer=tf.constant_initializer(0.0))
+				gamma = tf.reshape(gamma, [1, C, 1, 1])
+				beta = tf.reshape(beta, [1, C, 1, 1])
+
+			tensor = tf.reshape(x, [-1, C, H, W]) * gamma + beta
+			# tranpose: [bs, c, h, w, c] to [bs, h, w, c] following the paper
+			tensor = tf.transpose(tensor, [0, 2, 3, 1])			
+
+		out = tf.nn.conv2d(tensor, w, strides=stride, padding=pad, dilations=rate, name=name) + b # default no bias
+
+		return out	
+
+	def _upconv2d(self, tensor, dim, size=4, stride=2, pad='SAME', act='relu', name='upconv'):
+		[batch_size, h, w, in_dim] = tensor.shape.as_list()
+
+		size = size if isinstance(size, (tuple, list)) else [size, size]
+		stride = stride if isinstance(stride, (tuple, list)) else [1, stride, stride, 1]
+
+		kernel_shape = [size[0], size[1], dim, in_dim]
+		W = self._he_uniform(kernel_shape, name=name)
+
+		if pad == 'SAME':
+			out_shape = [batch_size, h*stride[1], w*stride[2], dim]
+		else:
+			out_shape = [batch_size, (h-1)*stride[1]+size[0],
+									(w-1)*stride[2]+size[1], dim]
+
+		out = tf.nn.conv2d_transpose(tensor, W, output_shape=tf.stack(out_shape),
+									strides=stride, padding=pad, name=name)
+
+		# reset shape information
+		out.set_shape(out_shape)
+
+		if act == 'relu':
+			out = tf.nn.relu(out, name=name+'/relu')
+		elif act == 'sigmoid':
+			out = tf.nn.sigmoid(out, name=name+'/sigmoid')
+		else:
+			pass
+
+		return out		
+
+
+	def _max_pool2d(self, tensor, size=2, stride=2, pad='VALID'):
+		size = size if isinstance(size, (tuple, list)) else [1, size, size, 1]
+		stride = stride if isinstance(stride, (tuple, list)) else [1, stride, stride, 1]
+		# 
+		size = [1, size[0], size[1], 1] if len(size)==2 else size
+		stride = [1, stride[0], stride[1], 1] if len(stride)==2 else stride
+
+		out = tf.nn.max_pool(tensor, size, stride, pad)
+
+		return out
+
+	# following three function used for combining context features
+	def _constant_kernel(self, shape, value=1.0, diag=False, flip=False, regularizer=None, trainable=None, name=None):
+		name = 'fixed_w' if name is None else name+'/fixed_w'
+
+		with tf.device('/device:GPU:'+GPU_ID):
+			if not diag:
+				k = tf.get_variable(name, shape, dtype=self.dtype,
+						initializer=tf.constant_initializer(value=value),
+						regularizer=regularizer, trainable=trainable)
+			else:
+				w = tf.eye(shape[0], num_columns=shape[1])
+				if flip:
+					w = tf.reshape(w, (shape[0], shape[1], 1))
+					w = tf.image.flip_left_right(w)
+				w = tf.reshape(w, shape)
+				k = tf.get_variable(name, None, dtype=self.dtype, # constant initializer dont specific shape
+						initializer=w,
+						regularizer=regularizer, trainable=trainable)				
+
+		return k		
+
+	def _context_conv2d(self, tensor, dim=1, size=7, diag=False, flip=False, stride=1, name='cconv'):
+		"""
+		Implement using identity matrix, combine neighbour pixels without bias, current only accept depth 1 of input tensor
+
+		Args:
+			diag: create diagnoal identity matrix
+			transpose: transpose the diagnoal matrix
+		"""	
+		in_dim = tensor.shape.as_list()[-1] # suppose to be 1
+		size = size if isinstance(size, (tuple, list)) else [size, size]
+		stride = stride if isinstance(stride, (tuple, list)) else [1, stride, stride, 1]
+		kernel_shape = [size[0], size[1], in_dim, dim]		
+
+		w = self._constant_kernel(kernel_shape, diag=diag, flip=flip, trainable=False, name=name)
+		out = tf.nn.conv2d(tensor, w, strides=stride, padding='SAME', name=name)
+
+		return out
+
+	def _non_local_context(self, tensor1, tensor2, stride=4, name='non_local_context'):
+		"""Use 1/stride image size of identity one rank kernel to combine context features, default is half image size, embedding between encoder and decoder part
+		
+		Args:
+			stride: define the neighbour size 
+		"""
+		assert tensor1.shape.as_list() == tensor2.shape.as_list(), "input tensor should have same shape"
+
+		[N, H, W, C] = tensor1.shape.as_list()
+
+		hs = H // stride if (H // stride) > 1 else (stride-1)
+		vs = W // stride if (W // stride) > 1 else (stride-1)
+
+		hs = hs if (hs%2!=0) else hs+1
+		vs = hs if (vs%2!=0) else vs+1
+
+		# compute attention map
+		a = self._conv2d(tensor1, dim=C, name=name+'/fa1')
+		a = self._conv2d(a, dim=C, name=name+'/fa2')
+		a = self._conv2d(a, dim=1, size=1, act='linear', norm=None, name=name+'/a')
+		a = tf.nn.sigmoid(a, name=name+'/a_sigmoid')
+
+		# reduce the tensor depth 		
+		x = self._conv2d(tensor2, dim=C, name=name+'/fx1')
+		x = self._conv2d(x, dim=1, size=1, act='linear', norm=None, name=name+'/x')
+
+		# pre attention, prevent the text
+		x = a*x
+
+		h = self._context_conv2d(x, size=[hs, 1], name=name+'/cc_h') # h
+		v = self._context_conv2d(x, size=[1, vs], name=name+'/cc_v') # v
+		d1 = self._context_conv2d(x, size=[hs, vs], diag=True, name=name+'/cc_d1') # d
+		d2 = self._context_conv2d(x, size=[hs, vs], diag=True, flip=True, name=name+'/cc_d2') # d_t
+
+		# double attention, prevent blurring
+		c1 = a*(h+v+d1+d2)
+		# c1 = (h+v+d1+d2)
+
+		# expand to dim 
+		c1 = self._conv2d(c1, dim=C, size=1, act='linear', norm=None, name=name+'/expand')
+		# c1 = self._conv2d(c1, dim=C, name=name+'/conv1') # contextural feature
+
+		# further convolution to learn richer feature
+		features = tf.concat([tensor2, c1], axis=3, name=name+'/in_context_concat')
+		out = self._conv2d(features, dim=C, name=name+'/conv2')
+
+		# return out, a
+		return out, None
+
+	def _up_bilinear(self, tensor, dim, shape, name='upsample'):
+		# [N, H, W, C] = tensor.shape.as_list()
+
+		out = self._conv2d(tensor, dim=dim, size=1, act='linear', name=name+'/1x1_conv')
+		return tf.image.resize_images(out, shape)
+
+
+	def forward(self, inputs, init_with_pretrain_vgg=False, pre_trained_model='./vgg16/vgg_16.ckpt'):
+		# feature extraction part and also the share network 
+		reuse_fnet = len([v for v in tf.global_variables() if v.name.startswith('FNet')]) > 0
+		with tf.variable_scope('FNet', reuse=reuse_fnet):
+			# feature extraction
+			self.conv1_1 = self._conv2d(inputs, dim=64, name='conv1_1') 
+			self.conv1_2 = self._conv2d(self.conv1_1, dim=64, name='conv1_2')		
+			self.pool1  = self._max_pool2d(self.conv1_2) # 256 => /2			
+
+			self.conv2_1 = self._conv2d(self.pool1, dim=128, name='conv2_1')
+			self.conv2_2 = self._conv2d(self.conv2_1, dim=128, name='conv2_2')	
+			self.pool2 = self._max_pool2d(self.conv2_2) # 128 => /4		
+
+			self.conv3_1 = self._conv2d(self.pool2, dim=256, name='conv3_1')
+			self.conv3_2 = self._conv2d(self.conv3_1, dim=256, name='conv3_2')	
+			self.conv3_3 = self._conv2d(self.conv3_2, dim=256, name='conv3_3')	
+			self.pool3 = self._max_pool2d(self.conv3_3) # 64 => /8		
+
+			self.conv4_1 = self._conv2d(self.pool3, dim=512, name='conv4_1')	
+			self.conv4_2 = self._conv2d(self.conv4_1, dim=512, name='conv4_2')		
+			self.conv4_3 = self._conv2d(self.conv4_2, dim=512, name='conv4_3')	
+			self.pool4 = self._max_pool2d(self.conv4_3)	# 32 => /16		
+
+			self.conv5_1 = self._conv2d(self.pool4, dim=512, name='conv5_1')	
+			self.conv5_2 = self._conv2d(self.conv5_1, dim=512, name='conv5_2')		
+			self.conv5_3 = self._conv2d(self.conv5_2, dim=512, name='conv5_3')		
+			self.pool5 = self._max_pool2d(self.conv5_3)	# 16 => /32		
+
+			# init feature extraction part from pre-train vgg16
+			if init_with_pretrain_vgg:
+				tf.train.init_from_checkpoint(pre_trained_model, self.pre_train_restore_map)
+
+			# input size for logits predict
+			[n, h, w, c] = inputs.shape.as_list()
+
+		reuse_cw_net = len([v for v in tf.global_variables() if v.name.startswith('CWNet')]) > 0
+		with tf.variable_scope('CWNet', reuse=reuse_cw_net):
+			# upsample
+			up2 = (self._upconv2d(self.pool5, dim=256, act='linear', name='up2_1') # 32 => /16
+					+ self._conv2d(self.pool4, dim=256, act='linear', name='pool4_s'))
+			self.up2_cw = self._conv2d(up2, dim=256, name='up2_3')
+
+			up4 = (self._upconv2d(self.up2_cw, dim=128, act='linear', name='up4_1') # 64 => /8
+					+ self._conv2d(self.pool3, dim=128, act='linear', name='pool3_s'))
+			self.up4_cw = self._conv2d(up4, dim=128, name='up4_3')
+
+			up8 = (self._upconv2d(self.up4_cw, dim=64, act='linear', name='up8_1') # 128 => /4
+					+ self._conv2d(self.pool2, dim=64, act='linear', name='pool2_s'))
+			self.up8_cw = self._conv2d(up8, dim=64, name='up8_2')
+
+			up16 = (self._upconv2d(self.up8_cw, dim=32, act='linear', name='up16_1') # 256 => /2
+					+ self._conv2d(self.pool1, dim=32, act='linear', name='pool1_s'))
+			self.up16_cw = self._conv2d(up16, dim=32, name='up16_2')
+
+			# predict logits
+			logits_cw = self._up_bilinear(self.up16_cw, dim=3, shape=(h, w), name='logits')	
+
+		# decode network for room type detection
+		reuse_rnet = len([v for v in tf.global_variables() if v.name.startswith('RNet')]) > 0
+		with tf.variable_scope('RNet', reuse=reuse_rnet):
+			# upsample
+			up2 = (self._upconv2d(self.pool5, dim=256, act='linear', name='up2_1') # 32 => /16
+					+ self._conv2d(self.pool4, dim=256, act='linear', name='pool4_s'))
+			up2 = self._conv2d(up2, dim=256, name='up2_2')
+			up2, _ = self._non_local_context(self.up2_cw, up2, name='context_up2')
+
+			up4 = (self._upconv2d(up2, dim=128, act='linear', name='up4_1') # 64 => /8
+					+ self._conv2d(self.pool3, dim=128, act='linear', name='pool3_s'))
+			up4 = self._conv2d(up4, dim=128, name='up4_2')
+			up4, _ = self._non_local_context(self.up4_cw, up4, name='context_up4')
+
+			up8 = (self._upconv2d(up4, dim=64, act='linear', name='up8_1') # 128 => /4
+					+ self._conv2d(self.pool2, dim=64, act='linear', name='pool2_s'))
+			up8 = self._conv2d(up8, dim=64, name='up8_2')
+			up8, _ = self._non_local_context(self.up8_cw, up8, name='context_up8')
+
+			up16 = (self._upconv2d(up8, dim=32, act='linear', name='up16_1') # 256 => /2
+					+ self._conv2d(self.pool1, dim=32, act='linear', name='pool1_s'))
+			up16 = self._conv2d(up16, dim=32, name='up16_2')
+			self.up16_r, self.a = self._non_local_context(self.up16_cw, up16, name='context_up16')
+
+			# predict logits
+			logits_r = self._up_bilinear(self.up16_r, dim=9, shape=(h, w), name='logits')	
+
+			return logits_r, logits_cw	

postprocess.py

@@ -0,0 +1,83 @@
+import argparse
+
+import os
+import sys
+import glob
+import time
+import random
+
+import numpy as np
+import imageio
+from PIL import Image
+from matplotlib import pyplot as plt
+
+sys.path.append('./utils/')
+from rgb_ind_convertor import *
+from util import *
+
+parser = argparse.ArgumentParser()
+
+parser.add_argument('--result_dir', type=str, default='./out',
+					help='The folder that save network predictions.')
+
+def post_process(input_dir, save_dir, merge=True):
+	if not os.path.exists(save_dir):
+		os.mkdir(save_dir)
+
+	input_paths = sorted(glob.glob(os.path.join(input_dir, '*.png')))
+	names = [i.split('/')[-1] for i in input_paths]
+	out_paths = [os.path.join(save_dir, i) for i in names]
+
+	n = len(input_paths)
+	# n = 1
+	for i in range(n):
+		im = imageio.imread(input_paths[i], mode='RGB')
+		im_ind = rgb2ind(im, color_map=floorplan_fuse_map)
+		# seperate image into room-seg & boundary-seg
+		rm_ind = im_ind.copy()
+		rm_ind[im_ind==9] = 0
+		rm_ind[im_ind==10] = 0
+
+		bd_ind = np.zeros(im_ind.shape, dtype=np.uint8)
+		bd_ind[im_ind==9] = 9
+		bd_ind[im_ind==10] = 10
+
+		hard_c = (bd_ind>0).astype(np.uint8)
+
+		# region from room prediction it self
+		rm_mask = np.zeros(rm_ind.shape)
+		rm_mask[rm_ind>0] = 1			
+		# region from close_wall line		
+		cw_mask = hard_c
+		# refine close wall mask by filling the grap between bright line	
+		cw_mask = fill_break_line(cw_mask)
+			
+		fuse_mask = cw_mask + rm_mask
+		fuse_mask[fuse_mask>=1] = 255
+
+		# refine fuse mask by filling the hole
+		fuse_mask = flood_fill(fuse_mask)
+		fuse_mask = fuse_mask // 255	
+
+		# one room one label
+		new_rm_ind = refine_room_region(cw_mask, rm_ind)
+
+		# ignore the background mislabeling
+		new_rm_ind = fuse_mask*new_rm_ind
+
+		print('Saving {}th refined room prediciton to {}'.format(i, out_paths[i]))
+		if merge:
+			new_rm_ind[bd_ind==9] = 9
+			new_rm_ind[bd_ind==10] = 10
+			rgb = ind2rgb(new_rm_ind, color_map=floorplan_fuse_map)
+		else:
+			rgb = ind2rgb(new_rm_ind)
+		imageio.imwrite(out_paths[i], rgb)
+
+if __name__ == '__main__':
+	FLAGS, unparsed = parser.parse_known_args()
+
+	input_dir = FLAGS.result_dir
+	save_dir = os.path.join(input_dir, 'post')
+
+	post_process(input_dir, save_dir)

pretrained/download_links.txt

@@ -0,0 +1 @@
+download link: https://mycuhk-my.sharepoint.com/:f:/g/personal/1155052510_link_cuhk_edu_hk/EgyJhisy04hNnxKncWl5zksBf9zDKDpMJ7c0V-q53_pxuA?e=P0BjZd

requirements.txt

@@ -0,0 +1,7 @@
+tensorflow==1.15.5
+numpy>=1.18.0
+Pillow>=8.0.0
+imageio>=2.9.0
+gradio>=3.0.0
+matplotlib>=3.0.0
+scipy>=1.4.0 

scores.py

@@ -0,0 +1,123 @@
+import argparse
+import numpy as np
+import imageio
+from PIL import Image
+
+import os
+import sys
+import glob
+import time
+
+sys.path.append('./utils/')
+from metrics import fast_hist
+from rgb_ind_convertor import *
+
+parser = argparse.ArgumentParser()
+
+parser.add_argument('--dataset', type=str, default='R3D',
+					help='define the benchmark')
+
+parser.add_argument('--result_dir', type=str, default='./out',
+					help='define the storage folder of network prediction')
+
+def evaluate_semantic(benchmark_path, result_dir, num_of_classes=11, need_merge_result=False, im_downsample=False, gt_downsample=False):
+	gt_paths = open(benchmark_path, 'r').read().splitlines()
+	d_paths = [p.split('\t')[2] for p in gt_paths] # 1 denote wall, 2 denote door, 3 denote room
+	r_paths = [p.split('\t')[3] for p in gt_paths] # 1 denote wall, 2 denote door, 3 denote room
+	cw_paths = [p.split('\t')[-1] for p in gt_paths] # 1 denote wall, 2 denote door, 3 denote room, last one denote close wall
+	im_paths = [os.path.join(result_dir, p.split('/')[-1]) for p in r_paths]
+	if need_merge_result:
+		im_paths = [os.path.join(result_dir+'/room', p.split('/')[-1]) for p in r_paths]
+		im_d_paths = [os.path.join(result_dir+'/door', p.split('/')[-1]) for p in d_paths]
+		im_cw_paths = [os.path.join(result_dir+'/close_wall', p.split('/')[-1]) for p in cw_paths]
+
+	n = len(im_paths)
+	# n = 1
+	hist = np.zeros((num_of_classes, num_of_classes))
+	for i in range(n):
+		im = imageio.imread(im_paths[i], mode='RGB')
+		if need_merge_result:
+			im_d = imageio.imread(im_d_paths[i], mode='L')
+			im_cw = imageio.imread(im_cw_paths[i], mode='L')
+		# create fuse semantic label
+		cw = imageio.imread(cw_paths[i], mode='L')
+		dd = imageio.imread(d_paths[i], mode='L')
+		rr = imageio.imread(r_paths[i], mode='RGB')
+
+		if im_downsample:
+			im = PIL.Image.fromarray(im).resize((512, 512), Image.Resampling.LANCZOS)
+			if need_merge_result:
+				im_d = PIL.Image.fromarray(im_d).resize((512, 512), Image.Resampling.LANCZOS)
+				im_cw = PIL.Image.fromarray(im_cw).resize((512, 512), Image.Resampling.LANCZOS)
+				im_d = np.array(im_d) / 255
+				im_cw = np.array(im_cw) / 255
+
+		if gt_downsample:
+			cw = PIL.Image.fromarray(cw).resize((512, 512), Image.Resampling.LANCZOS)
+			dd = PIL.Image.fromarray(dd).resize((512, 512), Image.Resampling.LANCZOS)
+			rr = PIL.Image.fromarray(rr).resize((512, 512, 3), Image.Resampling.LANCZOS)
+
+		# normalize
+		cw = cw / 255
+		dd = dd / 255
+
+		im_ind = rgb2ind(im, color_map=floorplan_fuse_map) 
+		if im_ind.sum()==0:
+			im_ind = rgb2ind(im+1)
+		rr_ind = rgb2ind(rr, color_map=floorplan_fuse_map) 
+		if rr_ind.sum()==0:
+			rr_ind = rgb2ind(rr+1)
+
+		if need_merge_result:
+			im_d  = (im_d>0.5).astype(np.uint8)
+			im_cw = (im_cw>0.5).astype(np.uint8)
+			im_ind[im_cw==1] = 10
+			im_ind[im_d ==1] = 9
+
+		# merge the label and produce 
+		cw = (cw>0.5).astype(np.uint8)
+		dd = (dd>0.5).astype(np.uint8)
+		rr_ind[cw==1] = 10
+		rr_ind[dd==1] = 9
+
+		name = im_paths[i].split('/')[-1]
+		r_name = r_paths[i].split('/')[-1]
+		
+		print('Evaluating {}(im) <=> {}(gt)...'.format(name, r_name))
+
+		hist += fast_hist(im_ind.flatten(), rr_ind.flatten(), num_of_classes)
+
+	print('*'*60)
+	# overall accuracy
+	acc = np.diag(hist).sum() / hist.sum()
+	print('overall accuracy {:.4}'.format(acc))
+	# per-class accuracy, avoid div zero
+	acc = np.diag(hist) / (hist.sum(1) + 1e-6)
+	print('room-type: mean accuracy {:.4}, room-type+bd: mean accuracy {:.4}'.format(np.nanmean(acc[:7]), (np.nansum(acc[:7])+np.nansum(acc[-2:]))/9.))
+	for t in range(0, acc.shape[0]):
+		if t not in [7, 8]:
+			print('room type {}th, accuracy = {:.4}'.format(t, acc[t]))
+
+	print('*'*60)
+	# per-class IU, avoid div zero
+	iu = np.diag(hist) / (hist.sum(1) + 1e-6 + hist.sum(0) - np.diag(hist))
+	print('room-type: mean IoU {:.4}, room-type+bd: mean IoU {:.4}'.format(np.nanmean(iu[:7]), (np.nansum(iu[:7])+np.nansum(iu[-2:]))/9.))
+	for t in range(iu.shape[0]):
+		if t not in [7,8]: # ignore class 7 & 8
+			print('room type {}th, IoU = {:.4}'.format(t, iu[t]))
+
+if __name__ == '__main__':
+	FLAGS, unparsed = parser.parse_known_args()
+
+	if FLAGS.dataset.lower() == 'r2v':
+		benchmark_path = './dataset/r2v_test.txt'
+	else:
+		benchmark_path = './dataset/r3d_test.txt'
+
+	result_dir = FLAGS.result_dir
+
+	tic = time.time()
+	evaluate_semantic(benchmark_path, result_dir, need_merge_result=False, im_downsample=False, gt_downsample=True) # same as previous line but 11 classes by combining the opening and wall line
+
+	print("*"*60)
+	print("Evaluate time: {} sec".format(time.time()-tic))

utils/create_tfrecord.py

@@ -0,0 +1,65 @@
+"""
+Please prepare the raw image datas save to one folder, 
+makesure the path is match to the train_file/test_file.
+"""
+
+from tf_record import *
+import imageio
+from PIL import Image
+
+train_file = '../dataset/r2v_train.txt'
+test_file = '../dataset/r2v_test.txt'
+
+# debug
+if __name__ == '__main__':
+	# write to TFRecord
+	train_paths = open(train_file, 'r').read().splitlines()
+	# test_paths = open(test_file, 'r').read().splitlines()
+
+	# write_record(train_paths, name='../dataset/jp_train.tfrecords')
+	# write_record(test_paths, name='../dataset/newyork_test.tfrecords')
+
+	# write_seg_record(train_paths, name='../dataset/jp_seg_train.tfrecords')
+	# write_seg_record(train_paths, name='../dataset/newyork_seg_train.tfrecords')
+
+	write_bd_rm_record(train_paths, name='../dataset/jp_train.tfrecords')
+	# write_bd_rm_record(train_paths, name='../dataset/all_train3.tfrecords')
+
+	# read from TFRecord
+	# loader_list = read_record('../dataset/jp_train.tfrecords')
+	# loader_list = read_seg_record('../dataset/jp_seg_train.tfrecords')
+
+	# loader_list = read_bd_rm_record('../dataset/newyork_bd_rm_train.tfrecords')
+	# loader_list = read_bd_rm_record('../dataset/jp_bd_rm_train.tfrecords')
+
+	# images = loader_list['images']
+	# bd_ind = loader_list['label_boundaries']
+	# rm_ind = loader_list['label_rooms']
+
+	# with tf.Session() as sess:
+	# 	# init all variables in graph
+	# 	sess.run(tf.group(tf.global_variables_initializer(),
+	# 					tf.local_variables_initializer()))
+		
+	# 	coord = tf.train.Coordinator()
+	# 	threads = tf.train.start_queue_runners(sess=sess, coord=coord)
+
+	# 	image, bd, rm = sess.run([images, bd_ind, rm_ind])
+
+	# 	print 'sess run image shape = ',image.shape
+	# 	print 'sess run wall shape = ', bd.shape
+	# 	print 'sess run room shape =', rm.shape
+
+	# 	bd = np.argmax(np.squeeze(bd), axis=-1)
+	# 	rm = np.argmax(np.squeeze(rm), axis=-1)
+	# 	plt.subplot(231)
+	# 	plt.imshow(np.squeeze(image))
+	# 	plt.subplot(233)
+	# 	plt.imshow(bd)
+	# 	plt.subplot(234)
+	# 	plt.imshow(rm)
+	# 	plt.show()
+
+	# 	coord.request_stop()
+	# 	coord.join(threads)
+	# 	sess.close()

utils/rgb_ind_convertor.py

@@ -0,0 +1,79 @@
+import numpy as np
+from PIL import Image
+ 
+# use for index 2 rgb
+floorplan_room_map = {
+	0: [  0,  0,  0], # background
+	1: [192,192,224], # closet
+	2: [192,255,255], # bathroom/washroom
+	3: [224,255,192], # livingroom/kitchen/diningroom
+	4: [255,224,128], # bedroom
+	5: [255,160, 96], # hall
+	6: [255,224,224], # balcony
+	7: [224,224,224], # not used
+	8: [224,224,128]  # not used
+}
+
+# boundary label
+floorplan_boundary_map = {
+	0: [  0,  0,  0], # background
+	1: [255,60,128],  # opening (door&window)
+	2: [255,255,255]  # wall line	
+}
+
+# boundary label for presentation
+floorplan_boundary_map_figure = {
+	0: [255,255,255], # background
+	1: [255, 60,128],  # opening (door&window)
+	2: [  0,  0,  0]  # wall line	
+}
+
+# merge all label into one multi-class label
+floorplan_fuse_map = {
+	0: [  0,  0,  0], # background
+	1: [192,192,224], # closet
+	2: [192,255,255], # batchroom/washroom
+	3: [224,255,192], # livingroom/kitchen/dining room
+	4: [255,224,128], # bedroom
+	5: [255,160, 96], # hall
+	6: [255,224,224], # balcony
+	7: [224,224,224], # not used
+	8: [224,224,128], # not used
+	9: [255,60,128],  # extra label for opening (door&window)
+	10: [255,255,255]  # extra label for wall line
+}
+
+# invert the color of wall line and background for presentation
+floorplan_fuse_map_figure = {
+	0: [255,255,255], # background
+	1: [192,192,224], # closet
+	2: [192,255,255], # batchroom/washroom
+	3: [224,255,192], # livingroom/kitchen/dining room
+	4: [255,224,128], # bedroom
+	5: [255,160, 96], # hall
+	6: [255,224,224], # balcony
+	7: [224,224,224], # not used
+	8: [224,224,128], # not used
+	9: [255,60,128],  # extra label for opening (door&window)
+	10: [ 0, 0,  0]  # extra label for wall line
+}
+
+def rgb2ind(im, color_map=floorplan_room_map):
+	ind = np.zeros((im.shape[0], im.shape[1]))
+
+	for i, rgb in color_map.items():
+		ind[(im==rgb).all(2)] = i
+
+	# return ind.astype(int) # int => int64
+	return ind.astype(np.uint8) # force to uint8
+
+def ind2rgb(ind_im, color_map=floorplan_room_map):
+	rgb_im = np.zeros((ind_im.shape[0], ind_im.shape[1], 3))
+
+	for i, rgb in color_map.items():
+		rgb_im[(ind_im==i)] = rgb
+
+	return rgb_im
+
+def unscale_imsave(path, im, cmin=0, cmax=255):
+	Image.fromarray(im, 'L').save(path)

utils/tf_record.py

@@ -0,0 +1,357 @@
+import numpy as np
+
+import tensorflow as tf
+
+import imageio
+from PIL import Image
+from matplotlib import pyplot as plt
+from rgb_ind_convertor import *
+
+import os
+import sys
+import glob 
+import time
+
+def load_raw_images(path):
+	paths = path.split('\t')
+
+	image = imageio.imread(paths[0], mode='RGB')
+	wall  = imageio.imread(paths[1], mode='L')
+	close = imageio.imread(paths[2], mode='L')
+	room  = imageio.imread(paths[3], mode='RGB')
+	close_wall = imageio.imread(paths[4], mode='L')
+
+	# NOTE: imresize will rescale the image to range [0, 255], also cast data into uint8 or uint32
+	image = PIL.Image.fromarray(image).resize((512, 512), Image.BICUBIC)
+	wall = PIL.Image.fromarray(wall).resize((512, 512), Image.BICUBIC)
+	close = PIL.Image.fromarray(close).resize((512, 512), Image.BICUBIC)
+	close_wall = PIL.Image.fromarray(close_wall).resize((512, 512), Image.BICUBIC)
+	room = PIL.Image.fromarray(room).resize((512, 512), Image.BICUBIC)
+
+	room_ind = rgb2ind(room)
+
+	# make sure the dtype is uint8
+	image = np.array(image).astype(np.uint8)
+	wall = np.array(wall).astype(np.uint8)
+	close = np.array(close).astype(np.uint8)
+	close_wall = np.array(close_wall).astype(np.uint8)
+	room_ind = room_ind.astype(np.uint8)
+
+	# debug
+	# plt.subplot(231)
+	# plt.imshow(image)
+	# plt.subplot(233)
+	# plt.imshow(wall, cmap='gray')
+	# plt.subplot(234)
+	# plt.imshow(close, cmap='gray')
+	# plt.subplot(235)
+	# plt.imshow(room_ind)
+	# plt.subplot(236)
+	# plt.imshow(close_wall, cmap='gray')
+	# plt.show()
+
+	return image, wall, close, room_ind, close_wall
+
+def _int64_feature(value):
+	return tf.train.Feature(int64_list=tf.train.Int64List(value=[value]))
+
+def _bytes_feature(value):
+	return tf.train.Feature(bytes_list=tf.train.BytesList(value=[value]))
+
+def write_record(paths, name='dataset.tfrecords'):
+	writer = tf.python_io.TFRecordWriter(name)
+	
+	for i in range(len(paths)):
+		# Load the image
+		image, wall, close, room_ind, close_wall = load_raw_images(paths[i])
+
+		# Create a feature
+		feature = {'image': _bytes_feature(tf.compat.as_bytes(image.tostring())),
+					'wall': _bytes_feature(tf.compat.as_bytes(wall.tostring())),
+					'close': _bytes_feature(tf.compat.as_bytes(close.tostring())),
+					'room': _bytes_feature(tf.compat.as_bytes(room_ind.tostring())),
+					'close_wall': _bytes_feature(tf.compat.as_bytes(close_wall.tostring()))}
+		
+		# Create an example protocol buffer
+		example = tf.train.Example(features=tf.train.Features(feature=feature))
+    
+		# Serialize to string and write on the file
+		writer.write(example.SerializeToString())
+    
+	writer.close()
+
+def read_record(data_path, batch_size=1, size=512):
+	feature = {'image': tf.FixedLenFeature(shape=(), dtype=tf.string),
+				'wall': tf.FixedLenFeature(shape=(), dtype=tf.string),
+				'close': tf.FixedLenFeature(shape=(), dtype=tf.string),
+				'room': tf.FixedLenFeature(shape=(), dtype=tf.string),
+				'close_wall': tf.FixedLenFeature(shape=(), dtype=tf.string)}
+
+	# Create a list of filenames and pass it to a queue
+	filename_queue = tf.train.string_input_producer([data_path], num_epochs=None, shuffle=False, capacity=batch_size*128)
+	
+	# Define a reader and read the next record
+	reader = tf.TFRecordReader()
+	_, serialized_example = reader.read(filename_queue)
+
+	# Decode the record read by the reader
+	features = tf.parse_single_example(serialized_example, features=feature)
+
+	# Convert the image data from string back to the numbers
+	image = tf.decode_raw(features['image'], tf.uint8)
+	wall = tf.decode_raw(features['wall'], tf.uint8)
+	close = tf.decode_raw(features['close'], tf.uint8)
+	room = tf.decode_raw(features['room'], tf.uint8)
+	close_wall = tf.decode_raw(features['close_wall'], tf.uint8)
+
+	# Cast data
+	image = tf.cast(image, dtype=tf.float32)
+	wall = tf.cast(wall, dtype=tf.float32)
+	close = tf.cast(close, dtype=tf.float32)
+	# room = tf.cast(room, dtype=tf.float32)
+	close_wall = tf.cast(close_wall, dtype=tf.float32)
+
+	# Reshape image data into the original shape
+	image = tf.reshape(image, [size, size, 3])
+	wall = tf.reshape(wall, [size, size, 1])
+	close = tf.reshape(close, [size, size, 1])
+	room = tf.reshape(room, [size, size])
+	close_wall = tf.reshape(close_wall, [size, size, 1])
+
+
+	# Any preprocessing here ...
+	# normalize 
+	image = tf.divide(image, tf.constant(255.0))
+	wall = tf.divide(wall, tf.constant(255.0))
+	close = tf.divide(close, tf.constant(255.0))
+	close_wall = tf.divide(close_wall, tf.constant(255.0))
+
+	# Genereate one hot room label
+	room_one_hot = tf.one_hot(room, 9, axis=-1)
+
+	# Creates batches by randomly shuffling tensors
+	images, walls, closes, rooms, close_walls = tf.train.shuffle_batch([image, wall, close, room_one_hot, close_wall], 
+						batch_size=batch_size, capacity=batch_size*128, num_threads=1, min_after_dequeue=batch_size*32)	
+
+	# images, walls = tf.train.shuffle_batch([image, wall], 
+						# batch_size=batch_size, capacity=batch_size*128, num_threads=1, min_after_dequeue=batch_size*32)	
+
+	return {'images': images, 'walls': walls, 'closes': closes, 'rooms': rooms, 'close_walls': close_walls}
+	# return {'images': images, 'walls': walls}
+
+# ------------------------------------------------------------------------------------------------------------------------------------- *
+# Following are only for segmentation task, merge all label into one 
+
+def load_seg_raw_images(path):
+	paths = path.split('\t')
+
+	image = imageio.imread(paths[0], mode='RGB')
+	close = imageio.imread(paths[2], mode='L')
+	room  = imageio.imread(paths[3], mode='RGB')
+	close_wall = imageio.imread(paths[4], mode='L')
+
+	# NOTE: imresize will rescale the image to range [0, 255], also cast data into uint8 or uint32
+	image = PIL.Image.fromarray(image).resize((512, 512), Image.BICUBIC)
+	close = PIL.Image.fromarray(close).resize((512, 512), Image.BICUBIC) / 255
+	close_wall = PIL.Image.fromarray(close_wall).resize((512, 512), Image.BICUBIC) / 255
+	room = PIL.Image.fromarray(room).resize((512, 512), Image.BICUBIC)
+
+	room_ind = rgb2ind(room)
+
+	# merge result
+	d_ind = (close>0.5).astype(np.uint8)
+	cw_ind = (close_wall>0.5).astype(np.uint8)
+	room_ind[cw_ind==1] = 10
+	room_ind[d_ind==1] = 9
+
+	# make sure the dtype is uint8
+	image = np.array(image).astype(np.uint8)
+	room_ind = room_ind.astype(np.uint8)
+
+	# debug
+	# merge = ind2rgb(room_ind, color_map=floorplan_fuse_map)
+	# plt.subplot(131)
+	# plt.imshow(image)
+	# plt.subplot(132)
+	# plt.imshow(room_ind)
+	# plt.subplot(133)
+	# plt.imshow(merge/256.)
+	# plt.show()
+
+	return image, room_ind
+
+def write_seg_record(paths, name='dataset.tfrecords'):
+	writer = tf.python_io.TFRecordWriter(name)
+	
+	for i in range(len(paths)):
+		# Load the image
+		image, room_ind = load_seg_raw_images(paths[i])
+
+		# Create a feature
+		feature = {'image': _bytes_feature(tf.compat.as_bytes(image.tostring())),
+					'label': _bytes_feature(tf.compat.as_bytes(room_ind.tostring()))}
+		
+		# Create an example protocol buffer
+		example = tf.train.Example(features=tf.train.Features(feature=feature))
+    
+		# Serialize to string and write on the file
+		writer.write(example.SerializeToString())
+    
+	writer.close()
+
+def read_seg_record(data_path, batch_size=1, size=512):
+	feature = {'image': tf.FixedLenFeature(shape=(), dtype=tf.string),
+				'label': tf.FixedLenFeature(shape=(), dtype=tf.string)}
+
+	# Create a list of filenames and pass it to a queue
+	filename_queue = tf.train.string_input_producer([data_path], num_epochs=None, shuffle=False, capacity=batch_size*128)
+	
+	# Define a reader and read the next record
+	reader = tf.TFRecordReader()
+	_, serialized_example = reader.read(filename_queue)
+
+	# Decode the record read by the reader
+	features = tf.parse_single_example(serialized_example, features=feature)
+
+	# Convert the image data from string back to the numbers
+	image = tf.decode_raw(features['image'], tf.uint8)
+	label = tf.decode_raw(features['label'], tf.uint8)
+
+	# Cast data
+	image = tf.cast(image, dtype=tf.float32)
+
+	# Reshape image data into the original shape
+	image = tf.reshape(image, [size, size, 3])
+	label = tf.reshape(label, [size, size])
+
+
+	# Any preprocessing here ...
+	# normalize 
+	image = tf.divide(image, tf.constant(255.0))
+
+	# Genereate one hot room label
+	label_one_hot = tf.one_hot(label, 11, axis=-1)
+
+	# Creates batches by randomly shuffling tensors
+	images, labels = tf.train.shuffle_batch([image, label_one_hot], 
+						batch_size=batch_size, capacity=batch_size*128, num_threads=1, min_after_dequeue=batch_size*32)	
+
+	# images, walls = tf.train.shuffle_batch([image, wall], 
+						# batch_size=batch_size, capacity=batch_size*128, num_threads=1, min_after_dequeue=batch_size*32)	
+
+	return {'images': images, 'labels': labels}
+
+# --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- *
+# ------------------------------------------------------------------------------------------------------------------------------------- *
+# Following are only for multi-task network. Two labels(boundary and room.)
+
+def load_bd_rm_images(path):
+	paths = path.split('\t')
+
+	image = imageio.imread(paths[0], mode='RGB')
+	close = imageio.imread(paths[2], mode='L')
+	room  = imageio.imread(paths[3], mode='RGB')
+	close_wall = imageio.imread(paths[4], mode='L')
+
+	# NOTE: imresize will rescale the image to range [0, 255], also cast data into uint8 or uint32
+	image = PIL.Image.fromarray(image).resize((512, 512), Image.BICUBIC)
+	close = PIL.Image.fromarray(close).resize((512, 512), Image.BICUBIC) / 255.
+	close_wall = PIL.Image.fromarray(close_wall).resize((512, 512), Image.BICUBIC) / 255.
+	room = PIL.Image.fromarray(room).resize((512, 512), Image.BICUBIC)
+
+	room_ind = rgb2ind(room)
+
+	# merge result
+	d_ind = (close>0.5).astype(np.uint8)
+	cw_ind = (close_wall>0.5).astype(np.uint8)
+
+	cw_ind[cw_ind==1] = 2
+	cw_ind[d_ind==1] = 1
+
+	# make sure the dtype is uint8
+	image = np.array(image).astype(np.uint8)
+	room_ind = room_ind.astype(np.uint8)
+	cw_ind = cw_ind.astype(np.uint8)
+
+	# debugging
+	# merge = ind2rgb(room_ind, color_map=floorplan_fuse_map)
+	# rm = ind2rgb(room_ind)
+	# bd = ind2rgb(cw_ind, color_map=floorplan_boundary_map)
+	# plt.subplot(131)
+	# plt.imshow(image)
+	# plt.subplot(132)
+	# plt.imshow(rm/256.)
+	# plt.subplot(133)
+	# plt.imshow(bd/256.)
+	# plt.show()
+
+	return image, cw_ind, room_ind, d_ind
+
+def write_bd_rm_record(paths, name='dataset.tfrecords'):
+	writer = tf.python_io.TFRecordWriter(name)
+	
+	for i in range(len(paths)):
+		# Load the image
+		image, cw_ind, room_ind, d_ind = load_bd_rm_images(paths[i])
+
+		# Create a feature
+		feature = {'image': _bytes_feature(tf.compat.as_bytes(image.tostring())),
+					'boundary': _bytes_feature(tf.compat.as_bytes(cw_ind.tostring())),
+					'room': _bytes_feature(tf.compat.as_bytes(room_ind.tostring())),
+					'door': _bytes_feature(tf.compat.as_bytes(d_ind.tostring()))}
+		
+		# Create an example protocol buffer
+		example = tf.train.Example(features=tf.train.Features(feature=feature))
+    
+		# Serialize to string and write on the file
+		writer.write(example.SerializeToString())
+    
+	writer.close()
+
+def read_bd_rm_record(data_path, batch_size=1, size=512):
+	feature = {'image': tf.FixedLenFeature(shape=(), dtype=tf.string),
+				'boundary': tf.FixedLenFeature(shape=(), dtype=tf.string),
+				'room': tf.FixedLenFeature(shape=(), dtype=tf.string),
+				'door': tf.FixedLenFeature(shape=(), dtype=tf.string)}
+
+	# Create a list of filenames and pass it to a queue
+	filename_queue = tf.train.string_input_producer([data_path], num_epochs=None, shuffle=False, capacity=batch_size*128)
+	
+	# Define a reader and read the next record
+	reader = tf.TFRecordReader()
+	_, serialized_example = reader.read(filename_queue)
+
+	# Decode the record read by the reader
+	features = tf.parse_single_example(serialized_example, features=feature)
+
+	# Convert the image data from string back to the numbers
+	image = tf.decode_raw(features['image'], tf.uint8)
+	boundary = tf.decode_raw(features['boundary'], tf.uint8)
+	room = tf.decode_raw(features['room'], tf.uint8)
+	door = tf.decode_raw(features['door'], tf.uint8)
+
+	# Cast data
+	image = tf.cast(image, dtype=tf.float32)
+
+	# Reshape image data into the original shape
+	image = tf.reshape(image, [size, size, 3])
+	boundary = tf.reshape(boundary, [size, size])
+	room = tf.reshape(room, [size, size])
+	door = tf.reshape(door, [size, size])
+
+	# Any preprocessing here ...
+	# normalize 
+	image = tf.divide(image, tf.constant(255.0))
+
+	# Genereate one hot room label
+	label_boundary = tf.one_hot(boundary, 3, axis=-1)
+	label_room = tf.one_hot(room, 9, axis=-1)
+
+	# Creates batches by randomly shuffling tensors
+	images, label_boundaries, label_rooms, label_doors = tf.train.shuffle_batch([image, label_boundary, label_room, door], 
+						batch_size=batch_size, capacity=batch_size*128, num_threads=1, min_after_dequeue=batch_size*32)	
+
+	# images, walls = tf.train.shuffle_batch([image, wall], 
+						# batch_size=batch_size, capacity=batch_size*128, num_threads=1, min_after_dequeue=batch_size*32)	
+
+	return {'images': images, 'label_boundaries': label_boundaries, 'label_rooms': label_rooms, 'label_doors': label_doors}

utils/util.py

@@ -0,0 +1,65 @@
+import cv2
+import numpy as np
+from scipy import ndimage
+
+def fast_hist(im, gt, n=9):
+    """
+    n is num_of_classes
+    """
+    k = (gt >= 0) & (gt < n)
+    return np.bincount(n * gt[k].astype(int) + im[k], minlength=n**2).reshape(n, n)
+
+def flood_fill(test_array, h_max=255):
+	"""
+	fill in the hole 
+	"""
+	input_array = np.copy(test_array) 
+	el = ndimage.generate_binary_structure(2,2).astype(int)
+	inside_mask = ndimage.binary_erosion(~np.isnan(input_array), structure=el)
+	output_array = np.copy(input_array)
+	output_array[inside_mask]=h_max
+	output_old_array = np.copy(input_array)
+	output_old_array.fill(0)   
+	el = ndimage.generate_binary_structure(2,1).astype(int)
+	while not np.array_equal(output_old_array, output_array):
+		output_old_array = np.copy(output_array)
+		output_array = np.maximum(input_array,ndimage.grey_erosion(output_array, size=(3,3), footprint=el))
+	return output_array
+
+def fill_break_line(cw_mask):
+	broken_line_h = np.array([[0,0,0,0,0],
+							[0,0,0,0,0],
+							[1,0,0,0,1],
+							[0,0,0,0,0],
+							[0,0,0,0,0]], dtype=np.uint8)	
+	broken_line_h2 = np.array([[0,0,0,0,0],
+							[0,0,0,0,0],
+							[1,1,0,1,1],
+							[0,0,0,0,0],
+							[0,0,0,0,0]], dtype=np.uint8)			
+	broken_line_v = np.transpose(broken_line_h)
+	broken_line_v2 = np.transpose(broken_line_h2)
+	cw_mask = cv2.morphologyEx(cw_mask, cv2.MORPH_CLOSE, broken_line_h)
+	cw_mask = cv2.morphologyEx(cw_mask, cv2.MORPH_CLOSE, broken_line_v)
+	cw_mask = cv2.morphologyEx(cw_mask, cv2.MORPH_CLOSE, broken_line_h2)
+	cw_mask = cv2.morphologyEx(cw_mask, cv2.MORPH_CLOSE, broken_line_v2)
+
+	return cw_mask	
+
+def refine_room_region(cw_mask, rm_ind):
+	label_rm, num_label = ndimage.label((1-cw_mask))
+	new_rm_ind = np.zeros(rm_ind.shape)
+	for j in range(1, num_label+1):  
+		mask = (label_rm == j).astype(np.uint8)
+		ys, xs = np.where(mask!=0)
+		area = (np.amax(xs)-np.amin(xs))*(np.amax(ys)-np.amin(ys))
+		if area < 100:
+			continue
+		else:
+			room_types, type_counts = np.unique(mask*rm_ind, return_counts=True)
+			if len(room_types) > 1:
+				room_types = room_types[1:] # ignore background type which is zero
+				type_counts = type_counts[1:] # ignore background count
+			new_rm_ind += mask*room_types[np.argmax(type_counts)]
+
+	return new_rm_ind