Upload 198 files

.gitattributes

@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+TEXTOIR-main/open_intent_detection/methods/OpenMax/libMR/build/temp.linux-x86_64-3.8/libmr.o filter=lfs diff=lfs merge=lfs -text
+TEXTOIR-main/open_intent_detection/methods/OpenMax/libMR/libmr.cpython-36m-x86_64-linux-gnu.so filter=lfs diff=lfs merge=lfs -text
+TEXTOIR-main/open_intent_detection/methods/OpenMax/libMR/libmr.cpython-38-x86_64-linux-gnu.so filter=lfs diff=lfs merge=lfs -text

TEXTOIR-main/.gitignore

@@ -0,0 +1,5 @@
+__pycache__
+logs
+outputs
+cache
+

TEXTOIR-main/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2022 THUIAR
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

TEXTOIR-main/README.md

@@ -0,0 +1,164 @@
+# TEXT Open Intent Recognition (TEXTOIR)
+
+TEXTOIR is the first high-quality Text Open Intent Recognition platform. This repo contains a convenient toolkit with extensible interfaces, integrating a series of state-of-the-art algorithms of two tasks (open intent detection and open intent discovery). We also release the pipeline framework and the visualized platform in the repo [TEXTOIR-DEMO](https://github.com/thuiar/TEXTOIR-DEMO). 
+
+
+## Introduction
+TEXTOIR aims to provide a convenience toolkit for researchers to reproduce the related text open classification and clustering methods. It contains two tasks, which are defined as open intent detection and open intent discovery. Open intent detection aims to identify n-class known intents, and detect one-class open intent. Open intent discovery aims to leverage limited prior knowledge of known intents to find fine-grained known and open intent-wise clusters. Related papers and codes are collected in our previous released [reading list](https://github.com/thuiar/OKD-Reading-List).
+
+Open Intent Recognition:  
+![Example](figs/Intro.png "Example")
+
+## Updates 🔥 🔥 🔥 
+
+| Date 	| Announcements 	|
+|-	|-	|
+| 12/2023  | 🎆 🎆 New paper and SOTA in Open Intent Discovery. Refer to the directory [USNID](./open_intent_discovery/examples/run_semi_usnid.sh) for the codes. Read the paper -- [A Clustering Framework for Unsupervised and Semi-supervised New Intent Discovery (Published in IEEE TKDE 2023)](https://ieeexplore.ieee.org/document/10349963).  |
+| 04/2023  | 🎆 🎆 New paper and SOTA in Open Intent Detection. Refer to the directory [DA-ADB](./open_intent_detection/examples/run_DA-ADB.sh) for the codes. Read the paper -- [Learning Discriminative Representations and Decision Boundaries for Open Intent Detection (Published in IEEE/ACM TASLP 2023)](https://ieeexplore.ieee.org/document/10097558).  |
+| 09/2021 	| 🎆 🎆 The first integrated and visualized platform for text Open Intent Recognition TEXTOIR has been released. Refer to the directory [TEXTOIR-DEMO](https://github.com/thuiar/TEXTOIR-DEMO) for the demo codes. Read our paper [TEXTOIR: An Integrated and Visualized Platform for Text Open Intent Recognition (Published in ACL 2021)](https://aclanthology.org/2021.acl-demo.20.pdf).	|
+| 05/2021 	| New paper and baselines DeepAligned in Open Intent Discovery have been released. Read our paper [Discovering New Intents with Deep Aligned Clustering (Published in AAAI 2021)](https://ojs.aaai.org/index.php/AAAI/article/view/17689). 	|
+| 05/2021 	| New paper and baselines ADB in Open Intent Detection have been released. Read our paper [Deep Open Intent Classification with Adaptive Decision Boundary (Published in AAAI 2021)](https://ojs.aaai.org/index.php/AAAI/article/view/17690). 	|
+| 05/2020 	| New paper and baselines CDAC+ in Open Intent Discovery have been released. Read our paper [Discovering New Intents via Constrained Deep Adaptive Clustering with Cluster Refinement (Published in AAAI 2020)](https://ojs.aaai.org/index.php/AAAI/article/view/6353). 	|
+| 07/2019 	| New paper and baselines DeepUNK in Open Intent Detection have been released. Read our paper [Deep Unknown Intent Detection with Margin Loss (Published in ACL 2019)](https://aclanthology.org/P19-1548). 	|
+---------------------------------------------------------------------------
+
+ **We strongly recommend you to use our TEXTOIR toolkit, which has standard and unified interfaces (especially data setting) to obtain fair and persuable results on benchmark intent datasets!**
+ 
+## Benchmark Datasets
+
+| Datasets | Source |
+| :---: | :---: |
+| [BANKING](./data/banking) | [Paper](https://aclanthology.org/2020.nlp4convai-1.5/) |
+| [OOS](./data/oos) / [CLINC150](./data/clinc) | [Paper](https://aclanthology.org/D19-1131/) |
+| [StackOverflow](./data/stackoverflow) | [Paper](https://aclanthology.org/W15-1509.pdf) |
+
+## Integrated Models
+### Open Intent Detection
+
+| Model Name | Source | Published |
+| :---: | :---: | :---: |
+| [OpenMax*](./open_intent_detection/examples/run_OpenMax.sh) | [Paper](https://openaccess.thecvf.com/content_cvpr_2016/papers/Bendale_Towards_Open_Set_CVPR_2016_paper.pdf) [Code](https://github.com/abhijitbendale/OSDN) | CVPR 2016 |
+| [MSP](./open_intent_detection/examples/run_MSP.sh) | [Paper](https://arxiv.org/pdf/1610.02136.pdf) [Code](https://github.com/hendrycks/error-detection) | ICLR 2017 |
+| [DOC](./open_intent_detection/examples/run_DOC.sh) | [Paper](https://aclanthology.org/D17-1314.pdf) [Code](https://github.com/leishu02/EMNLP2017_DOC) | EMNLP 2017 |
+| [DeepUnk](./open_intent_detection/examples/run_DeepUnk.sh) | [Paper](https://aclanthology.org/P19-1548.pdf) [Code](https://github.com/thuiar/DeepUnkID) | ACL 2019 |
+| [SEG](./open_intent_detection/examples/run_SEG.sh) | [Paper](https://aclanthology.org/2020.acl-main.99) [Code](https://github.com/fanolabs/0shot-classification) | ACL 2020 |
+| [ADB](./open_intent_detection/examples/run_ADB.sh) | [Paper](https://ojs.aaai.org/index.php/AAAI/article/view/17690) [Code](https://github.com/thuiar/Adaptive-Decision-Boundary) | AAAI 2021 |
+| [(K+1)-way](./open_intent_detection/examples/run_K+1-way.sh) | [Paper](https://aclanthology.org/2021.acl-long.273) [Code](https://github.com/fanolabs/out-of-scope-intent-detection) | ACL 2021 |
+| [MDF](./open_intent_detection/examples/run_MDF.sh) | [Paper](https://aclanthology.org/2021.acl-long.85.pdf) [Code](https://github.com/rivercold/BERT-unsupervised-OOD) | ACL 2021 |
+| [ARPL*](./open_intent_detection/examples/run_ARPL.sh) | [Paper](https://ieeexplore.ieee.org/document/9521769) [Code](https://github.com/iCGY96/ARPL) | IEEE TPAMI 2022 |
+| [KNNCL](./open_intent_detection/examples/run_KNNCL.sh) | [Paper](https://aclanthology.org/2022.acl-long.352/) [Code](https://github.com/zyh190507/KnnContrastiveForOOD) | ACL 2022 |
+| [DA-ADB](./open_intent_detection/examples/run_DA-ADB.sh) | [Paper](https://ieeexplore.ieee.org/document/10097558) [Code](https://github.com/thuiar/TEXTOIR) | IEEE/ACM TASLP 2023 |
+
+### New Intent Discovery
+
+| Setting | Model Name | Source | Published |
+| :---: | :---: | :---: | :---: |
+| Unsupervised | [KM](./examples/run_KM.sh) | [Paper](https://www.cs.cmu.edu/~bhiksha/courses/mlsp.fall2010/class14/macqueen.pdf) | BSMSP 1967 |
+| Unsupervised | [AG](./examples/run_AG.sh) | [Paper](https://www.sciencedirect.com/science/article/abs/pii/0031320378900183) | PR 1978 |
+| Unsupervised | [SAE-KM](./examples/run_SAE.sh) | [Paper](https://www.jmlr.org/papers/volume11/vincent10a/vincent10a.pdf)  | JMLR 2010|
+| Unsupervised | [DEC](./examples/run_DEC.sh) | [Paper](http://proceedings.mlr.press/v48/xieb16.pdf) [Code](https://github.com/piiswrong/dec) | ICML 2016 |
+| Unsupervised | [DCN](./examples/run_DCN.sh) | [Paper](http://proceedings.mlr.press/v70/yang17b/yang17b.pdf) [Code](https://github.com/xuyxu/Deep-Clustering-Network) | ICML 2017 |
+| Unsupervised | [CC](./examples/run_CC.sh) | [Paper](https://yunfan-li.github.io/assets/pdf/Contrastive%20Clustering.pdf) [Code](https://github.com/Yunfan-Li/Contrastive-Clustering) | AAAI 2021 |
+| Unsupervised | [SCCL](./examples/run_SCCL.sh) | [Paper](http://proceedings.mlr.press/v70/yang17b/yang17b.pdf) [Code](https://github.com/xuyxu/Deep-Clustering-Network) | NAACL 2021 |
+| Unsupervised | [USNID](./examples/run_unsup_usnid) | [Paper](https://ieeexplore.ieee.org/document/10349963) [Code](https://github.com/thuiar/TEXTOIR/tree/main/open_intent_discovery) |   IEEE TKDE 2023 |
+| Semi-supervised | [KCL*](./examples/run_KCL.sh) | [Paper](https://openreview.net/pdf?id=ByRWCqvT-) [Code](https://github.com/GT-RIPL/L2C) | ICLR 2018 |
+| Semi-supervised | [MCL*](./examples/run_MCL.sh) | [Paper](https://openreview.net/pdf?id=SJzR2iRcK7) [Code](https://github.com/GT-RIPL/L2C) | ICLR 2019 |
+| Semi-supervised | [DTC*](./examples/run_DTC.sh) | [Paper](https://www.robots.ox.ac.uk/~vgg/research/DTC/files/iccv2019_DTC.pdf) [Code](https://github.com/k-han/DTC) | ICCV 2019 |
+| Semi-supervised | [CDAC+](./examples/run_CDACPlus.sh) | [Paper](https://ojs.aaai.org/index.php/AAAI/article/view/6353) [Code](https://github.com/thuiar/CDAC-plus) | AAAI 2020 |
+| Semi-supervised | [DeepAligned](./examples/run_DeepAligned.sh) | [Paper](https://ojs.aaai.org/index.php/AAAI/article/view/17689) [Code](https://github.com/thuiar/DeepAligned-Clustering) | AAAI 2021 |
+| Semi-supervised | [GCD](./examples/run_GCD.sh) | [Paper](https://www.robots.ox.ac.uk/~vgg/research/gcd/) [Code](https://github.com/sgvaze/generalized-category-discovery) | CVPR 2022 |
+| Semi-supervised | [MTP-CLNN](./examples/run_MTP_CLNN.sh) | [Paper](https://aclanthology.org/2022.acl-long.21.pdf) [Code](https://github.com/fanolabs/NID_ACLARR2022) | ACL 2022 |
+| Semi-supervised | [USNID](./examples/run_semi_usnid.sh) | [Paper](https://ieeexplore.ieee.org/document/10349963) [Code](https://github.com/thuiar/TEXTOIR/tree/main/open_intent_discovery) |  IEEE TKDE 2023 |
+
+
+(* denotes the CV model replaced with the BERT backbone)
+
+## Quick Start
+1. Use anaconda to create Python (version >= 3.6) environment
+```
+conda create --name textoir python=3.6
+conda activate textoir
+```
+
+2. Install PyTorch (Cuda version 11.2)
+```
+conda install pytorch torchvision torchaudio cudatoolkit=11.0 -c pytorch -c conda-forge  
+```
+3. Clone the TEXTOIR repository, and choose the task (Take open intent detection as an example).
+```
+git clone git@github.com:thuiar/TEXTOIR.git
+cd TEXTOIR
+cd open_intent_detection
+```
+4. Install related environmental dependencies
+```
+pip install -r requirements.txt
+```
+5. Run examples (Take ADB as an example)
+```
+sh examples/run_ADB.sh
+```
+
+* Note that if you cannot download the pre-trained model directly from HuggingFace transformers, you need to download it yourself. We provide the pre-trained bert model in the following link:  
+[Baidu Cloud Drive](https://pan.baidu.com/s/1k1zxK4xh0UyPhOU_-oPlow) with code: v8tk
+
+## Extensibility
+
+This toolkit is extensible and supports adding new methods, datasets, configurations, backbones, dataloaders, losses conveniently. More detailed information can be seen in the tutorials of the directories [open_intent_detection](./open_intent_detection/README.md) and [open_intent_discovery](./open_intent_discovery/README.md). 
+
+<!-- ### Extensibility
+This toolkit is extensible and supports adding new methods, datasets, configurations, backbones, dataloaders, losses conveniently. More detailed information can be seen in the directory [open_intent_detection](./open_intent_detection/README.md) and [open_intent_discovery](./open_intent_discovery/README.md) respectively. 
+
+### Reliability
+The codes in this repo have been confirmed and are reliable. 
+
+The experimental results are close to the reported ones in our AAAI 2021 papers [Discovering New Intents with DeepAligned Clustering](https://ojs.aaai.org/index.php/AAAI/article/view/17689) and [Deep Open Intent Classification with Adaptive Decision Boundary](https://ojs.aaai.org/index.php/AAAI/article/view/17690). Note that the results of some methods may fluctuate in a small range due to the selected random seeds, hyper-parameters, optimizers, etc. The final results are the average of 10 random seeds to reduce the influence of different selected known classes. -->
+
+## Citations
+
+If this work is helpful, or you want to use the codes and results in this repo, please cite the following papers:
+
+* [TEXTOIR: An Integrated and Visualized Platform for Text Open Intent Recognition](https://aclanthology.org/2021.acl-demo.20/)
+* [Learning Discriminative Representations and Decision Boundaries for Open Intent Detection](https://ieeexplore.ieee.org/document/10097558)
+* [A Clustering Framework for Unsupervised and Semi-supervised New Intent Discovery](https://ieeexplore.ieee.org/document/10349963)
+
+```
+@inproceedings{zhang-etal-2021-textoir,
+    title = "{TEXTOIR}: An Integrated and Visualized Platform for Text Open Intent Recognition",
+    author = "Zhang, Hanlei  and Li, Xiaoteng  and Xu, Hua  and Zhang, Panpan and Zhao, Kang  and Gao, Kai",
+    booktitle = "Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations",
+    pages = "167--174",
+    year = "2021",
+    url = "https://aclanthology.org/2021.acl-demo.20",
+    doi = "10.18653/v1/2021.acl-demo.20",
+}
+```
+```
+@article{DA-ADB, 
+    title = {Learning Discriminative Representations and Decision Boundaries for Open Intent Detection},  
+    author = {Zhang, Hanlei and Xu, Hua and Zhao, Shaojie and Zhou, Qianrui}, 
+    journal = {IEEE/ACM Transactions on Audio, Speech, and Language Processing},  
+    volume = {31},
+    pages = {1611-1623},
+    year = {2023}, 
+    doi = {10.1109/TASLP.2023.3265203} 
+} 
+```
+```
+@ARTICLE{USNID,
+  author={Zhang, Hanlei and Xu, Hua and Wang, Xin and Long, Fei and Gao, Kai},
+  journal={IEEE Transactions on Knowledge and Data Engineering}, 
+  title={A Clustering Framework for Unsupervised and Semi-supervised New Intent Discovery}, 
+  year={2023},
+  doi={10.1109/TKDE.2023.3340732}
+} 
+```
+
+
+## Contributors
+
+[Hanlei Zhang](https://github.com/HanleiZhang), [Shaojie Zhao](https://github.com/MurraryZhao), [Xin Wang](https://github.com/mrFocusXin), [Ting-En Lin](https://github.com/tnlin), [Qianrui Zhou](https://github.com/zhougr18), [Huisheng Mao](https://github.com/FlameSky-S). 
+
+## Bugs or questions?
+
+If you have any questions, please open issues and illustrate your problems as detailed as possible. If you want to integrate your method in our repo, please feel free to **pull request**!

TEXTOIR-main/__init__.py

@@ -0,0 +1,17 @@
+'''
+Text Open Intent Recognition (TEXTOIR) toolkit @THUIAR.
+
+If you use this code, please cite the following paper:
+@inproceedings{zhang-etal-2021-textoir,
+    title = "{TEXTOIR}: An Integrated and Visualized Platform for Text Open Intent Recognition",
+    author = "Zhang, Hanlei  and
+      Li, Xiaoteng  and
+      Xu, Hua  and
+      Zhang, Panpan  and
+      Zhao, Kang  and
+      Gao, Kai",
+    booktitle = "Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations",
+    year = "2021",
+    pages = "167--174",
+}
+'''

TEXTOIR-main/data/banking/dev.tsv

@@ -0,0 +1,1001 @@
+text	label
+Could you please explain why I received a message that my transfer is not possible? I've never had any problems making a transfer in the past.	beneficiary_not_allowed
+How did you guys get your exchange rate?	exchange_rate
+I am still waiting on my card?	card_arrival
+Why is the purchase I made this morning still pending?	pending_card_payment
+I made a mistake and need to cancel a transfer.	cancel_transfer
+How can I request cash back? The ATM just gave me a different amount than what I've been actually charged though...	wrong_amount_of_cash_received
+I think a fee was applied for paying with a card.	card_payment_fee_charged
+How long does it take for a cash withdrawal to show?	pending_cash_withdrawal
+I attempted a transfer, but it failed.	failed_transfer
+How do you decide what the exchange rates are?	exchange_rate
+what steps do i do to get a real card	order_physical_card
+Fees for currency exchange	exchange_charge
+Is it possible to obtain both a Visa and a Mastercard from you?	visa_or_mastercard
+why was a extra fee added to my account for transfer ?	transfer_fee_charged
+I am not happy with your company. Please delete my account asap.	terminate_account
+Why did I not get the amount I specified at the ATM?	wrong_amount_of_cash_received
+Why can't I withdraw cash from this ATM?	declined_cash_withdrawal
+Am I able to get both a Visa and MasterCard from you?	visa_or_mastercard
+I am not satisfied with your company.  Close my account.	terminate_account
+How can I exchange currencies in the app?	exchange_via_app
+How do I modify my account details?	edit_personal_details
+Will I be fined for topping up by transfer?	top_up_by_bank_transfer_charge
+I want to top up by card, what do you support?	supported_cards_and_currencies
+Tell me how to renew my new card?	activate_my_card
+The card can be used at which ATMs?	atm_support
+Can i choose a specific delivery date?	card_delivery_estimate
+want to know where funds come from	verify_source_of_funds
+My card isn't working at all, I need assistance. It's really frustrating.	card_not_working
+My Netflix subscription says that there is an issue with my card information, but I thought everything was fine. Why would it be declined?	card_not_working
+Why was I charged $1 in a transaction?	extra_charge_on_statement
+Why are all my transfers failing?  I have the details right and they won't complete.  I've been trying all day and they still won't go through, this is URGENT, please help.	declined_transfer
+Can I add money to top up my account via a bank transfer?	transfer_into_account
+I thought you offer crypto top up but it doesn't seem to be right ? Money just disappeared from my account, How it would work ?	top_up_reverted
+Where do you get those exchange rates?	exchange_rate
+The seller just told me he didn't get paid. After reviewing my account it did subtract the balance but now it has added it back to my account. I need you to please take care of this promptly.	reverted_card_payment?
+How do I sign up for a card if I'm in America?	country_support
+Can I get spare cards?	getting_spare_card
+I've transferred money into my account but the balance is the same	balance_not_updated_after_bank_transfer
+I haven't received my transfer funds yet, where are they?	pending_transfer
+I want to change my account details.	edit_personal_details
+I'm getting concerned here.  I sent a friend some money and she isn't seeing it.  What's going on?	transfer_not_received_by_recipient
+My card payment is just showing up as pending since forever, what's going on with that, when should it be going through?	pending_card_payment
+There is a payment with my card which i definitely did not make by me .Never seen that it before.	card_payment_not_recognised
+How much will I be charged for exchanging foreign currency	exchange_charge
+I have lost my credit card in the ATM!	card_swallowed
+I recently made a transfer that I would like to cancel	cancel_transfer
+My card needs to be topped up.	topping_up_by_card
+Why was I denied the option to transfer money to a selected beneficiary?	beneficiary_not_allowed
+I live in the EU, can I get one of your cards?	country_support
+Can I use this app for both USD and GBP?	exchange_via_app
+What do I need to see my source of money?	verify_source_of_funds
+Help! My card isn't working at the ATM and I know I used the right PIN. Trying to get my money.	declined_cash_withdrawal
+Why does my transfer say it is pending?	pending_transfer
+Someone withdrew money from my account, and it wasn't me.	cash_withdrawal_not_recognised
+How do I shut down my account and get out of this company?	terminate_account
+How long do I have  to wait for my transaction to be completed?	transfer_not_received_by_recipient
+Will you be able to exchange to EUR?	fiat_currency_support
+Hello, I have noticed a problem on my recent withdrawal at the banks atm machine.  I made a transaction and upon receiving my funds from the machine, a fee was assessed to my withdrawal.  Are these types of transactions free of charge?	cash_withdrawal_charge
+How can many disposable cards can I have per day?	disposable_card_limits
+Is there a specific source that the exchange rate for the transfer I'm planning on making is pulled from?	exchange_rate
+I want to use a payment card to top up my account. How can I do this?	supported_cards_and_currencies
+I can't get the card itself to work.	card_not_working
+I want a refund on the things i bought	request_refund
+In reviewing my statement, I show I have two charges for the same purchase at the same time, which I have not made in duplicate? How do I get this solved?	transaction_charged_twice
+I made a card payment, but it has been declined?	declined_card_payment
+What are the fees for using transfer to top off my account?	top_up_by_bank_transfer_charge
+Do you have an auto-top function and how do I use it if so?	automatic_top_up
+How can I find an ATM that accepts this card?	atm_support
+I wanted to buy something online yesterday, but I could not because the card got declined on the day of and the following day. What is wrong with the card?	declined_transfer
+I have the cash already, my account still shows up as pending.  How can my account be still pending?	pending_cash_withdrawal
+Do you know when the withdrawal will show?	pending_cash_withdrawal
+How much can I top up on my card?	top_up_limits
+Is topping up unlimited?	top_up_limits
+Can you tell me what I need to do to have my pin changed?	change_pin
+what is this extra charge for a withdrawal?	cash_withdrawal_charge
+how to recover my passcode	passcode_forgotten
+The transfer finally worked, but how long does it stay pending?	pending_transfer
+Where do I go to unblock my PIN?	pin_blocked
+Why did my credit card get declined for top up?	top_up_failed
+A transfer was not allowed to an account.	beneficiary_not_allowed
+Payment I did not make.	card_payment_not_recognised
+I couldn't pay with card in a shop	declined_card_payment
+I was trying to top up my account and it isn't working	top_up_failed
+Im not sure if a seller is giving me my money back! I asked them to refund my money for the item but It's been a week now and still nothing has been given back to me. Can you resolve this for me?	Refund_not_showing_up
+Do I need to use a limit on top-ups?	top_up_limits
+How can I find out why my transfer didn't go through?	failed_transfer
+I made a card payment but it didn't work. Why not?	declined_card_payment
+How much is it to use my US debit or credit card to top up?	top_up_by_card_charge
+Do you support direct deposits from my employer?	receiving_money
+My wallet is empty even though I topped it up an hour ago.	topping_up_by_card
+I haven't received my PIN yet. Do I need to get it from you?	get_physical_card
+Can I top up my balance with a cheque?	top_up_by_cash_or_cheque
+How long does it take for me to get my new card?	card_arrival
+"Hi, I had an issue with my card payment and It shows on my account that my payment went through. Then I got contacted from the seller saying ""they did not receive my payment"" then the payment was returned to my account. so I would like to see why it did that please."	reverted_card_payment?
+I see a direct debit payment on my statement that wasn't me	direct_debit_payment_not_recognised
+Can I have another card in addition to my first one?	getting_spare_card
+I'm not certain my top-up went through yet.	pending_top_up
+can I link a card of yours I already have?	card_linking
+How much would I be charged for a transfer?	top_up_by_bank_transfer_charge
+My pin isn't here yet	get_physical_card
+My card wants to be topped what do you guys support?	supported_cards_and_currencies
+I am having trouble transferring money to a beneficiery.	beneficiary_not_allowed
+Where did my extra fee for the transfer come from?	transfer_fee_charged
+I need help right now. I realize I sent my payment to the wrong account number after making a typo. Please cancel the transfer now since I can't do it from the app.	cancel_transfer
+Why have I been charged an extra pound?	extra_charge_on_statement
+Salary in GBP has been received. Does it need to be configured into another place?	receiving_money
+Give me a refund	request_refund
+Do i have to top-up with my credit card before I can transfer?	topping_up_by_card
+Can you do exchanges of EUR?	fiat_currency_support
+Why can't I transfer to another account?	beneficiary_not_allowed
+What is the process for activating my card and using it?	activate_my_card
+Why can't I see my top up?	topping_up_by_card
+There's a payment showing in the app today even though I have not used my card at all. I want a refund.	direct_debit_payment_not_recognised
+Is the ATM broken? I can't get cash from it.	declined_cash_withdrawal
+How can I check to see where my funds come from?	verify_source_of_funds
+I punched in the wrong pin too many times and I'm locked out.	pin_blocked
+How to receive virtual card?	getting_virtual_card
+My client was converting their Russian ruble into UK pound currency when they noticed their bill was much too high for the exchange. What happened?	card_payment_wrong_exchange_rate
+Hi I'm waiting for a refund for an item I bought. I have contacted the merchant directly but had nothing back. Can you tell me whether or not you are able to refund me for the item?	Refund_not_showing_up
+what is this extra charge with my purchase?	card_payment_fee_charged
+What countries are you issuing cards for?	country_support
+How do I get the passcode reset?	passcode_forgotten
+I am still waiting for a the cash I deposited this morning	balance_not_updated_after_cheque_or_cash_deposit
+Do you have a limit for someone's age?	age_limit
+Change of location, need to change details	edit_personal_details
+How can I make my card ready to use?	activate_my_card
+How do you top up with a cheque?	top_up_by_cash_or_cheque
+Why did I get less from the ATM?	wrong_amount_of_cash_received
+I wish to cancel a transfer	cancel_transfer
+Please explain to me how to know where my funds are from.	verify_source_of_funds
+Can I order a card please?	order_physical_card
+What are the steps to change my PIN to something else?	change_pin
+What is the Top up rate charge for using a card in the US?	top_up_by_card_charge
+Which ATMs accept this bill of fare ?	atm_support
+why hasn't my virtual card come yet?	getting_virtual_card
+How is my money transferred to the account	transfer_into_account
+I'm still waiting for my new card.	card_arrival
+How long does it take to receive my card?	card_delivery_estimate
+Will you guys charge me for topping up by transfer?	top_up_by_bank_transfer_charge
+What is taking so long with my top up?	pending_top_up
+can I choose delivery times to suite me?	card_delivery_estimate
+Hello. Please, would you tell me if you have or have not received funds for a refund that I recently requested for a big purchase? I've contacted the merchant, but I can't tell that they have taken any action on it.	Refund_not_showing_up
+What's the tracking on the card you sent?	card_arrival
+Which flat currencies are supported for holding and exchange?	fiat_currency_support
+What should I do if I think someone is using my card without my permission?	compromised_card
+How can I transfer money to a beneficiary?	beneficiary_not_allowed
+Why would the app deny my top-up?	top_up_failed
+I have a friend who needs money as soon as possibly. If I were to transfer it how long will it take to get to her?	transfer_not_received_by_recipient
+How long will a transaction to a friend take?	transfer_not_received_by_recipient
+I have a new email.	edit_personal_details
+Can you provide me a disposable virtual card?	get_disposable_virtual_card
+Can I change my pin through the app?	change_pin
+I have been double charged for a couple of things this week, and would appreciate a refund for the duplicate charges.	transaction_charged_twice
+What needs to be done to activate and use a card?	activate_my_card
+How long will this transfer from europe take?	transfer_timing
+My PIN has been blocked, what should I do?	pin_blocked
+How can I get my PIN unblocked?	pin_blocked
+How long does a check take to show up in the account balance?	balance_not_updated_after_cheque_or_cash_deposit
+I made a mistake and performed a transaction on the wrong account!	cancel_transfer
+Hi, I tried to top up my card today and it did not work. I have tried it in the past and it worked just fine but it won't do it anymore. Can you help me out?	top_up_failed
+Why was I charged twice for the same transaction?	transaction_charged_twice
+Transfers may show up as pending for up to one business day while they are processed, and then they'll depart for the beneficiary accounts. Please confirm that you have correctly entered your account details. SEPA transfers typically take up to two working days, while SWIFT transfers can take up to five working days.	pending_transfer
+How would I go about getting a physical card to use in stores?	order_physical_card
+My money is gone. I had topped up recently and saw it was there and went through, but now it's disappeared, what is happening?	top_up_reverted
+I might have been charged on my debit statement by fraud.	direct_debit_payment_not_recognised
+I didn't get my card back from the ATM.	card_swallowed
+Why was the wrong rate applied when I bought something in a different currency?	card_payment_wrong_exchange_rate
+A payment I didn't do shows in my app	card_payment_not_recognised
+How much is the charge to top up via credit card?	top_up_by_card_charge
+Do you hold money and if so, what currencies do you accept?	fiat_currency_support
+There is a withdrawal in my account I didn't make.	cash_withdrawal_not_recognised
+Why am I still not able to see the money I deposited a week ago? This is urgent!!	balance_not_updated_after_cheque_or_cash_deposit
+Where is the closest ATM I can use?	atm_support
+I heard you have virtual cards. How do I get one?	getting_virtual_card
+Where do I find PIN for my card?	pin_blocked
+I don't think I should have been charged a fee to use my card.	card_payment_fee_charged
+What are the fees imposed when I make a transaction?	card_payment_fee_charged
+Explain to me how to verify the top-up card.	verify_top_up
+I keep being asked to verify my identity. Is this normal?	why_verify_identity
+I need to find out where the card is that I ordered.	card_arrival
+I am still waiting for a transfer I did	balance_not_updated_after_bank_transfer
+I was charged a fee for the transfer, but can you tell me why?	transfer_fee_charged
+Why hasn't my top-up finished pending yet?	pending_top_up
+I need urgent help. I realize I sent my payment to the wrong account number. I made a typo. Please cancel the transfer immediately. I cannot do anything from the app.	cancel_transfer
+I ordered something, and now I have buyer's remorse. Since what I ordered hasn't come yet, can you just cancel the payment since I already know I want a refund for it?	request_refund
+When are cards shipped?	card_delivery_estimate
+The ATM keeps declining my Transaction . I tried two different ATMs already Would you please check if everything is okay with my account?	declined_cash_withdrawal
+How high is the exchange fee?	exchange_charge
+My ATM withdrawal is taking forever	pending_cash_withdrawal
+How do I get a card if I live in the US?	country_support
+I topped up recently and saw the money go through, but now it's just gone? What's going on	top_up_reverted
+Why was I charged an extra fee from the ATM?	cash_withdrawal_charge
+Which fiat currencies are supported by you?	fiat_currency_support
+A while ago i requested a refund from a seller. I keep checking my statement but I have not been refunded yet. I'm confused why i haven't gotten my money back yet but I need your help getting it please.	Refund_not_showing_up
+how do the cards work?	get_disposable_virtual_card
+How do I get extra cards?	getting_spare_card
+how do I link an old card?	card_linking
+How would I change my details?	edit_personal_details
+My card is missing!  Help!	lost_or_stolen_card
+What does it mean if my cash withdrawal does not show up properly?	pending_cash_withdrawal
+lost my card in atm	card_swallowed
+How do I replace a card that's going to expire?	card_about_to_expire
+Not sure what my password is.	passcode_forgotten
+Can someone help with my passcode?	passcode_forgotten
+I attempted to withdraw some money from an ATM this morning and received the wrong amount. Your app says I have enough money available in the account. Can someone tell me what's wrong?	wrong_amount_of_cash_received
+How does it work when I transfer money into my account?	transfer_into_account
+What currencies and cards do you support?	supported_cards_and_currencies
+Can it specifically be delivered on a certain date?	card_delivery_estimate
+My refund has not arrived, and I am unsure why. Who do I contact?	Refund_not_showing_up
+My app shows a payment I didn't make	card_payment_not_recognised
+I see cash in my app but I did not get it.	cash_withdrawal_not_recognised
+Why hasn't my refund been appiled to my account?	Refund_not_showing_up
+I had a dollar charged to my account that's still pending and wanted to know what the charge is for?	extra_charge_on_statement
+Why isn't the transferred money there?	balance_not_updated_after_bank_transfer
+I can't make purchases with my virtual card.	virtual_card_not_working
+What proof of identification is needed?	verify_my_identity
+Please help me!  My physical card isn't working right.	card_not_working
+There are duplicate transactions on my account	transaction_charged_twice
+How can I find your exchange rates?	exchange_rate
+Did I not get my refund? I don't see it.	Refund_not_showing_up
+Hearing about your verification results from us may take anywhere from 10 minutes to approximately one hour.  If this verification has in fact, failed, double-check to make sure that your images are clear.  Also make sure that your document photos have no blur or glare. They need to be readable.  You must also be 18 years of age or older and be a resident of Switzerland or the European Economic Area to open an account.	unable_to_verify_identity
+Why on earth do you need so much personal id info from me?	why_verify_identity
+How do I activate auto top-up when traveling?	automatic_top_up
+Why is my 'top up' not showing up in my wallet?	topping_up_by_card
+I'm not sure why my payment has been declined? Please tell me why, I thought it was all good.	declined_card_payment
+For some reason, I'm unable to transfer funds.	beneficiary_not_allowed
+Help!  The app has an ATM withdrawl that I didn't make.	cash_withdrawal_not_recognised
+I received a fee for obtaining cash, is this correct?	cash_withdrawal_charge
+Can I transfer money using the SWIFT service?	top_up_by_bank_transfer_charge
+How many transactions can be made with a disposable card?	disposable_card_limits
+How long does it take for the money to show up in my account?	transfer_timing
+I have used all of my PIN tries. What should I do now?	pin_blocked
+Will my other US credit card be accepted?	supported_cards_and_currencies
+Which currencies do you handle?	supported_cards_and_currencies
+I don't recognize this payment	card_payment_not_recognised
+Which merchants accept my card?	card_acceptance
+is it ok if i use both visa and mastercard	visa_or_mastercard
+How do I get my card PIN?	get_physical_card
+For the identity check, do I need any kind of documentation?	verify_my_identity
+Do you need to know my first and last name?	why_verify_identity
+How do I take a block off my card?	pin_blocked
+How long does it actually take for a card payment to go through? There's one stuck since a couple of days already that doesn't seem to move	pending_card_payment
+I moved, can I edit my personal details?	edit_personal_details
+Which fiat currencies are supported?	fiat_currency_support
+My refund isn't showing up on my statement.	Refund_not_showing_up
+There is  a charge I don't recognize.	card_payment_not_recognised
+My top up hasn't gone through	pending_top_up
+I put my check in the bank, but I haven't gotten my cash yet?	balance_not_updated_after_cheque_or_cash_deposit
+I've been trying to transfer funds to top-up my card, but the transaction is not going through.  Is there something I'm doing wrong?	top_up_failed
+When will I get my money? It's been too long.	transfer_not_received_by_recipient
+Where is the refund on my statement?	Refund_not_showing_up
+Please explain how disposable virtual card work.	get_disposable_virtual_card
+The app isn't letting me log in, but my info is correct.	unable_to_verify_identity
+why cant you just refund me instead of a merchant	Refund_not_showing_up
+I have gone through my statements and noticed there is a large payment that I did not make. Can i still dispute the payment? its been a while	direct_debit_payment_not_recognised
+oh my god I lost my card! what should I do?	lost_or_stolen_card
+I have been overcharged.	transaction_charged_twice
+Disaster, I've totally forgotten my passcode, can you help me?	passcode_forgotten
+how can i get it to add money automatically while travelling	automatic_top_up
+Will my GPB update my salary	receiving_money
+Can I top up with cash	top_up_by_cash_or_cheque
+I cannot seem to use my card.	card_not_working
+When did you start charging to make a withdrawal? How much is it now? I always thought it was free.	cash_withdrawal_charge
+Why is your exchange rate so bad?	card_payment_wrong_exchange_rate
+Hi, I sent a payment to the wrong account number - can you cancel it as soon as possible?	cancel_transfer
+I was mugged.  They took everything.  I can't use the app.  What do I do?	lost_or_stolen_phone
+What are the countries you support.	country_support
+I removed cash from an ATM earlier but it shows up as pending in the app. How can this still be pending, I already received the cash?	pending_cash_withdrawal
+I cannot seem to buy something online with my card? the payment comes back, what's going on?	reverted_card_payment?
+The exchange rate from my card payment isn't right.	card_payment_wrong_exchange_rate
+I have an emergency and lost my phone.  I need help accessing my account.	lost_or_stolen_phone
+What currencies do you accept for adding money to my account?	supported_cards_and_currencies
+Can you tell me what the problem with my card is? It was declined at a restaurant today.	card_not_working
+What is the time period for money to be transferred?	transfer_timing
+What do I do if I think someone has used my card?	compromised_card
+Why is my identity not being verified?	unable_to_verify_identity
+How can I information on my limit for top-ups?	top_up_limits
+When the time is right will this automatically replenish my balance?	automatic_top_up
+I'm waiting for my transaction to complete.	transfer_not_received_by_recipient
+I am seeing in the App a some cash withdrawal that I did not do	cash_withdrawal_not_recognised
+Hey a seller just contacted me that my money didn't come through. It was definitely deducted from my account, but now it appears it's back with me! Sort this out asap please.	reverted_card_payment?
+Why did you charge me a fee when I made a transfer?	transfer_fee_charged
+Where is the top-up verification code located?	verify_top_up
+I made a big purchase that I regretted and cancelled, the merchant said that my refund was submitted. Can you confirm to me that all of my money was truly returned to me?	Refund_not_showing_up
+Is top-up possible with Apple Pay?	apple_pay_or_google_pay
+Are there any restrictions for the disposable cards that I need to be aware of?	disposable_card_limits
+Am I allowed to carry an additional card?	getting_spare_card
+My package is taking too long to arrive and I don't want it anymore. Can I get a refund?	request_refund
+I'd like to open an account for my kids can I do that?	age_limit
+I did not do this payment	card_payment_not_recognised
+I sent a payment but it isn't showing up in the app	transfer_not_received_by_recipient
+How long does it take for a new purchase to stop being considered pending?	pending_card_payment
+What do I do if my top-up has not gone through?	pending_top_up
+I think the payment from my card was revereted	reverted_card_payment?
+How do I modify my details?	edit_personal_details
+Is there a way to link the card i just got?	card_linking
+Can i set up an auto top-up?	automatic_top_up
+What happened! I asked for 30 pounds at your ATM and only go 10. The machine lied about it too. It said I received the full 30 pounds. Someone better help me asap on this.	wrong_amount_of_cash_received
+Can friends I have top-up my account?	topping_up_by_card
+Is it possible to change from AUD to GBP?	exchange_via_app
+I deposited a cheque and its been days and I still haven't received the cash!!	balance_not_updated_after_cheque_or_cash_deposit
+I have been charged twice on my statement.	transaction_charged_twice
+What about an auto top-up, is that a option?	automatic_top_up
+I misplaced my phone so I can't use the app! Does this mean I can't use my card?	lost_or_stolen_phone
+The exchange rate for my payment last Saturday must be wrong because I was charged extra.	card_payment_wrong_exchange_rate
+What do I need in regards to identification?	verify_my_identity
+the exchange rate is wrong from something I got in another country	card_payment_wrong_exchange_rate
+I don't recognize this payment at all!  Help!	card_payment_not_recognised
+What's the top-up limit?	top_up_limits
+Are you able to delete my account?	terminate_account
+I would like to reactivate my card that I previously reported lost. I located the card this morning.	card_linking
+Can you tell me about why the account I have was charged an extra 1$	extra_charge_on_statement
+I need to order a new card.	order_physical_card
+Why was there a transfer fee on my account?	transfer_fee_charged
+I need a Visa and a Mastercard from you.	visa_or_mastercard
+Why do you keep declining my transfers? it's always been working really well so far but when I tried to buy something just now the card got declined, I tried couple of times already  but same thing is happening.	declined_transfer
+I have questions about how to transfer money to my account.	transfer_into_account
+Where did the  €1 fee in my statement come from	extra_charge_on_statement
+I need help with understanding why my card is not working.	card_not_working
+Will be Apple Watch be able to let me top up?	apple_pay_or_google_pay
+I was charged for getting cash.	cash_withdrawal_charge
+my money, where is it coming from?	verify_source_of_funds
+How can I set a new pin.	change_pin
+I think I was charged extra on my payment on last Saturday.  Maybe the exchange rate was wrong?  I'd like a refund.	card_payment_wrong_exchange_rate
+A transfer could not be completed.	failed_transfer
+What is the limit on disposable cards?	disposable_card_limits
+What is the proper way to activate a card?	activate_my_card
+I want to link my new card. Can you help?	card_linking
+Any chance of getting the card outside the UK?	country_support
+Where can I get a card in the EU?	country_support
+My wallet was just stolen, and they've already used my card! Please help, I can't afford to lose more money!	cash_withdrawal_not_recognised
+Is there a reason my top-up has been cancelled?	top_up_reverted
+Help me!  I don't know what my password is.	passcode_forgotten
+In my statement, there was an extra €1 fee	extra_charge_on_statement
+Can you  tell me how to get AUD not GBP?	exchange_via_app
+can i verify my identity i the app	verify_my_identity
+My cheque is taking a while	balance_not_updated_after_cheque_or_cash_deposit
+I bought something and received it, but the payment went back to my account. Is the seller okay?	reverted_card_payment?
+The wrong exchange rate was applied to me while pulling out cash.	wrong_exchange_rate_for_cash_withdrawal
+My top up is giving me problems. What is wrong with it?	top_up_failed
+I see a $1 charge in a transaction.	extra_charge_on_statement
+What steps do I have to take to change my PIN?	change_pin
+My top-up funds aren't available yet.	pending_top_up
+When can I expect my card?	card_delivery_estimate
+My card is stolen.  Help!	lost_or_stolen_card
+What is going on with my cash deposit. I'm looking at my account balance and it is not there. It has been almost a week. Someone better get back to me asap.	balance_not_updated_after_cheque_or_cash_deposit
+The store card reader would not accept my card.	declined_card_payment
+What businesses can I use this card with?	card_acceptance
+Where can the cards be delivered?	order_physical_card
+what are limits to top up	top_up_limits
+When will I receive my card?	card_delivery_estimate
+Can I use your app to exchange between USD and GBP	exchange_via_app
+I deposited cash into my account and I'm having trouble seeing it on my account.	balance_not_updated_after_cheque_or_cash_deposit
+I didn't cancel my top-up.	top_up_reverted
+My card payment's cancelled	reverted_card_payment?
+My new card hasn't came in.	card_arrival
+There are multiple instances where you added a fee to the payment, as well as some where you didn't. Do you have an outline of when a fee will be added?	card_payment_fee_charged
+Hi, I tried to withdraw some money from an ATM and the machine was broken.  It says that the transaction is still in progress.  Can you double check to see what's happening? I don't want to be charged since I was not able to withdraw the money.	pending_cash_withdrawal
+What is the fee for accepting money from a foreign country?	exchange_charge
+how long are my top ups pending	pending_top_up
+I got less money than I asked for.	wrong_amount_of_cash_received
+What fees are there for adding money using an international card	top_up_by_card_charge
+Why wouldn't the ATM give me my money?	declined_cash_withdrawal
+I was double charged at a restaurant can you take the second charge off?	transaction_charged_twice
+Money transfer help? How long?	transfer_timing
+Help me access the card I already got so I can information from it.	card_linking
+The passcode isn't working for me.	passcode_forgotten
+Is the 1 pound charge I can see an error or an incorrect entry?	extra_charge_on_statement
+I just deposited cash to my account and can't find it!	balance_not_updated_after_cheque_or_cash_deposit
+I made a payment with my card and it hasn't changed. How long do you think this will take?	pending_card_payment
+Is there any way to cancel a recent transfer?  I did not mean to make it and need to make sure it does not go through.	cancel_transfer
+My top up didn't go through	top_up_failed
+I got this stupid fee after taking cash out, why?	cash_withdrawal_charge
+My card's expiration date is close.	card_about_to_expire
+When is my balance going to update, I just transferred some money	balance_not_updated_after_bank_transfer
+How come when I tried to pay contactless at the bus it didnt work?	contactless_not_working
+Why does my identity need verification?	why_verify_identity
+Can I choose from either Visa or Mastercard?	visa_or_mastercard
+There's an unauthorized cash withdraw	cash_withdrawal_not_recognised
+I don't need to transfer anymore and I want to cancel it.	cancel_transfer
+I think there's a purchase at a store that I didn't go to.	card_payment_not_recognised
+Can you accept my other card from the U.S.?  It is the only other one I have.	supported_cards_and_currencies
+When I made a transfer I was charged extra.	transfer_fee_charged
+I have done a transfer within the country the day before yesterday, but it's still not there yet. Can you please check what's going on with that, the account number definitely is the right one, I've checked that already.	transfer_not_received_by_recipient
+I can't get money out of the ATM. Is my card actually working? It's the first time I tried using it	declined_cash_withdrawal
+A payment that I didn't make appears in my app.	card_payment_not_recognised
+Why did the ATM give me less money than what I requested?	wrong_amount_of_cash_received
+When will I have access to my pin number?	get_physical_card
+Send me my card now! ASAP	card_delivery_estimate
+On my disposable card, is there a transaction limit?	disposable_card_limits
+What is the price for an additional card?	getting_spare_card
+I need a single use card for shopping online	get_disposable_virtual_card
+How do I discontinue my account?	terminate_account
+I need a new card and I live in the United States.	country_support
+I saw a fee on my app from one of my payments and I wasn't sure where it came from. Is there a way you can tell me when there's an additional fee for my payments?	card_payment_fee_charged
+I've tried using my contactless in several locations today and it's not working anywhere. It seemed fine before. How do I get it to work again?	contactless_not_working
+There is a direct debit set up that I didn't create.	direct_debit_payment_not_recognised
+How can I open an account for my children?	age_limit
+What is the maximum number of transactions I can do with a disposable card?	disposable_card_limits
+Hello. I tried doing a topup and I thought it worked, but now the money is no longer in my account. Can you explain?	top_up_reverted
+What's the reason my new card hasn't come?	card_arrival
+How many digits identify the top-up verification code?	verify_top_up
+Whats the transfer process	transfer_into_account
+I was wondering, Can I get my pay in another currency?	receiving_money
+I was charged an incorrect exchange rate for a withdrawal.	wrong_exchange_rate_for_cash_withdrawal
+Can I change my card PIN?	change_pin
+How long is US delivery?	card_delivery_estimate
+How do I revert a transaction?	cancel_transfer
+What is the foreign exchange rate you will apply?	exchange_rate
+I would like my refund for this item I bought.	request_refund
+How do I transfer cash my into my account?	transfer_into_account
+I could not access my money at the ATM	declined_cash_withdrawal
+I am trying to transfer money and something is going wrong.	failed_transfer
+I believe that someone is using my card without my knowledge!	compromised_card
+I topped up but my money is not showing. Why is that?	top_up_reverted
+I'm sure I have enough money so why didn't the store accept my card?	declined_card_payment
+How do I get a new expiration date?	card_about_to_expire
+How do I use disposable cards?	get_disposable_virtual_card
+What reasons would a topup fail?	top_up_failed
+I attempted to get cash in the ATM but it was not authorized	declined_cash_withdrawal
+How long does it take for deliver to the US?	card_delivery_estimate
+What is the length of a transaction completion?	transfer_not_received_by_recipient
+How much is a physical card?	order_physical_card
+I have a new card and I can't add money to it.  I tried about 30 minutes ago and it is still pending.  What do I do?	pending_top_up
+I made a purchase but never expected to see a fee charged for the transaction.	card_payment_fee_charged
+I was charged for using my card?	card_payment_fee_charged
+If I don't have all my documents can they still verify my identity?	verify_my_identity
+If I were to use a European bank card for a top up, will I incur a charge?	top_up_by_card_charge
+Hi, I am interested buying crypto currency but unable to purchase it through the application. I do want to do the exchange, Could you please let me know what is the problem?	beneficiary_not_allowed
+How do I close my account?	terminate_account
+I want to change my PIN.	change_pin
+My app statement shows I took out cash from an ATM but I did not. What can I do?	cash_withdrawal_not_recognised
+Why are you asking so many questions about who I am	why_verify_identity
+I do not see the card PIN anywhere?	get_physical_card
+Can someone assist me with activating my card?	activate_my_card
+Is it possible for me to top up with Google Pay?	apple_pay_or_google_pay
+What is this extra fee that I got when I used my card to pay?	card_payment_fee_charged
+Can I stop a transaction?	cancel_transfer
+Can I reactivate my card? I reported it lost and found it this morning in my jacket.	card_linking
+How long will my transfer stay pending?	pending_transfer
+I can't seem to make a standard bank transfer. I have tried at least five times already but none of them are going through. Please tell me what is wrong?	failed_transfer
+The transfer I made isn't reflected in my balance	balance_not_updated_after_bank_transfer
+Where do I go to activate my card?	activate_my_card
+I deposited a check a few days ago and it's still not gone in! Please help me with this	balance_not_updated_after_cheque_or_cash_deposit
+hello I have lost my phone and I'm worried someone's going to steal money from me	lost_or_stolen_phone
+My card got charged twice for the same thing!	transaction_charged_twice
+When exchanging, how much are the rates?	exchange_rate
+Top up is not working even though I have my AMEX in apple pay.	apple_pay_or_google_pay
+I had to pay money for paying with my card.	card_payment_fee_charged
+why can't my beneficiary make transactions?	beneficiary_not_allowed
+I think my card is lost.  What do I do?	lost_or_stolen_card
+Why is it required to verify my identity?	why_verify_identity
+What are the countries that you operate in?	country_support
+What is the handling fee for transfer?	top_up_by_bank_transfer_charge
+The app failed to verify me.	unable_to_verify_identity
+Is there any fees for top ups?	top_up_by_card_charge
+The exchange rate is incorrect for my cash withdrawal transaction.	wrong_exchange_rate_for_cash_withdrawal
+How do I add an existing card to the app?	card_linking
+Why won't you take my transfer, I keep getting an error message.	beneficiary_not_allowed
+I prefer to get a Mastercard.	visa_or_mastercard
+Am I allowed to hold money in multiple currencies?	fiat_currency_support
+Why are my transfers being declined? It normally works but now it's declining my card. I've tried it a few times but it's still not working.	declined_transfer
+Can I increase my top-up maximum?	top_up_limits
+what is the exchange rate when i get cash	wrong_exchange_rate_for_cash_withdrawal
+Why was I charged a random fee for a transfer I made?	transfer_fee_charged
+I just bought an item and changed my mind.  I have not received it yet how can I get refunded.  Thank you.	request_refund
+Hello. I just noticed I was charged two times for a restaurant I visited this week. Can you reverse the additional charge?	transaction_charged_twice
+I have an unknown debit charge in my statement, why?	direct_debit_payment_not_recognised
+How do I go about getting a refund?	request_refund
+Why did the ATM eat my card?	card_swallowed
+Why can't I transfer money to a beneficiary	beneficiary_not_allowed
+How can I exchange to EUR?	fiat_currency_support
+Is there any way I could open an account for my daughter?	age_limit
+The payment on my card shows as pending.	pending_card_payment
+I would like to receive a refund for something I bought.	request_refund
+i need help. I updated my balance with a check yesterday and Im not seeing it. Can you look at my account to see what went wrong?	balance_not_updated_after_cheque_or_cash_deposit
+There is a payment with my card which i definitely did not do.	card_payment_not_recognised
+How can I dispute a direct debit transaction?	direct_debit_payment_not_recognised
+I'd like to know what the foreign exchange rate will be.	exchange_rate
+I am inquiring about your auto top up feature.	automatic_top_up
+My card was stolen and used to make several purchases. Please freeze my card so no one can use it.	compromised_card
+I need to figure out who has access to my card there are some payments I didn't make that are showing on the app.  Could you please help?	card_payment_not_recognised
+What ATM accepts Mastercard?	atm_support
+Will I be charged for a cash withdrawal?	cash_withdrawal_charge
+How long will the transfer take?	pending_transfer
+Where can I change my PIN?	change_pin
+Are there any fees if I want to add money using an international card?	top_up_by_card_charge
+I would like to know what the restrictions are for the disposable cards.	disposable_card_limits
+Can I receive payment in another currency?	receiving_money
+I am not seeing recent cash withdrawal on my account.	pending_cash_withdrawal
+The top up failed	top_up_failed
+How long do transfers take to show up in my account?	balance_not_updated_after_bank_transfer
+How old do I need to be?	age_limit
+What is the age requirement?	age_limit
+Waiting for the cash I deposited this morning still	balance_not_updated_after_cheque_or_cash_deposit
+How does one send me money?	receiving_money
+My salary is in GBP, is this something I need to configure?	receiving_money
+how do i find top up for cash deposit	top_up_by_cash_or_cheque
+Do I get charged if I am receiving money?	top_up_by_bank_transfer_charge
+How do I find the top-up verification code?	verify_top_up
+My card has not come in the mail yet.	card_arrival
+I was charged the wrong exchange rate on a purchase	card_payment_wrong_exchange_rate
+I think my card has been compromised in some way.	compromised_card
+It says that my payment is pending, what does that mean?	pending_card_payment
+Why is the exchange rate wrong for my cash withdrawal?	wrong_exchange_rate_for_cash_withdrawal
+Can I dispute a debit payment that I am seeing that I didn't make from a couple of months ago?	direct_debit_payment_not_recognised
+How long does it take for transfers to process?  I sent funds to a friend, and she says that she has not yet received anything.	transfer_not_received_by_recipient
+Changing my PIN	change_pin
+I must verify the source of my funds	verify_source_of_funds
+I need my card to be delivered fast	card_delivery_estimate
+do you have a fee for receiving a SEPA transfer?	top_up_by_bank_transfer_charge
+I cannot get my google pay to work.	apple_pay_or_google_pay
+What is the purpose of requiring my identification?	why_verify_identity
+There is a strange charge on my debit statement	direct_debit_payment_not_recognised
+It appears my transfer was declined.	declined_transfer
+Why is the transaction not posting?	pending_transfer
+I exchange currencies a lot.  Can I get a discount?	exchange_charge
+Hello, I went ahead and checked my account details to be sure, and they are definitely correct.  I've had a transfer pending for a while, and I can't understand why it hasn't gone through yet.	pending_transfer
+In a month, my card will reach its expiration date, am I required to request the next card?	card_about_to_expire
+What are ways to test why my card might not be working?	card_not_working
+When will my pending payment finish?	pending_card_payment
+Hi, I have done a topup, but my money was reverted. Please look into this issue.	top_up_reverted
+There has been a red flag on my top up.	top_up_failed
+You can use it anywhere that accepts Mastercard.	card_acceptance
+I was trying to use contactless earlier, but it didn't work.	contactless_not_working
+Why is the refund not on my statement?	Refund_not_showing_up
+The atm didn'y give me all the cash	wrong_amount_of_cash_received
+If I am in China, can I still order a new card and if so, how?	card_about_to_expire
+How much are you going to charge me if I top up my US card?	top_up_by_card_charge
+Is there an age limit to when my child can use these services?	age_limit
+Can I receive payments in another currency?	receiving_money
+Its been 3 days, How long will my transfer take to get to my wallet?	transfer_timing
+I requested a refund but can't see it.	Refund_not_showing_up
+Can I link my card I previously received?	card_linking
+What kind of charges should I expect to exchange currencies?	exchange_charge
+My card got trapped inside an ATM, what should I do?	card_swallowed
+Could you please help me?  I made my rent payment and have double checked that I sent it to the right account.  The person receiving it says he has not received it but it shows complete on my side.	transfer_not_received_by_recipient
+I need to get some Euros for my holiday. Am I able to exchange some of my money?	exchange_via_app
+Help me activate my new card.	activate_my_card
+I transferred money and it hasn't come yet	transfer_not_received_by_recipient
+Can I get a Visa card?	visa_or_mastercard
+How is my card payment still pending?	pending_card_payment
+Why hasn't my account updated to reflect my current balance?	balance_not_updated_after_bank_transfer
+I tried to charge my card but it was declined. A couple days ago I was able to charge it and it was accepted. Is something wrong? Will you double check for me please?	top_up_failed
+Is there any way to get another card on the same account?	getting_spare_card
+I did not receive my full withdrawal amount	wrong_amount_of_cash_received
+do i need to wait for my card before i get pin	get_physical_card
+Why do I need to have my identity verified?	why_verify_identity
+How do I receive a disposable virtual card?	get_disposable_virtual_card
+I just transferred some money into my account and do not see it updated yet, can you assist?	balance_not_updated_after_bank_transfer
+How do I change my last name?	edit_personal_details
+Where are you present globally?	country_support
+Where can i exchange between GBP and USD	exchange_via_app
+Would it be possible to open up an account for children?	age_limit
+Did you lose my money? I had added funds through top-up not long ago and I know they were in my account, but they seem to have disappeared. What happened?	top_up_reverted
+What is the procedure of unblocking my PIN?	pin_blocked
+I make payments for my place abroad using this account and since i use it so frequently I was wondering why did the fees increase suddenly rather than decreasing? I would think customers like me should receive incentives for using your service	transfer_fee_charged
+Where do I go to terminate my account?	terminate_account
+Someone took my card without my permission.	lost_or_stolen_card
+I would like to know how much I can top-up.	top_up_limits
+Where are the ATMs that I can use Mastercard?	atm_support
+When will you be taking orders for disposable virtual cards?	get_disposable_virtual_card
+I need a refund for something I bought.	request_refund
+Can this app exchange American and English currency?	exchange_via_app
+There is a direct debit that I didn't set up, what should I do?	direct_debit_payment_not_recognised
+Can I top up using cash?	top_up_by_cash_or_cheque
+How do I go about exchanging currencies with this app?	exchange_via_app
+Do all businesses take this card?	card_acceptance
+Is there a fee for transferring money	transfer_fee_charged
+Why are you not accepting my transfer!! I've tried a couple times already now and it just keeps showing an error message	beneficiary_not_allowed
+This is not right, I was told I could take out cash at the ATM for free?	cash_withdrawal_charge
+Why won't the machine accept my card anymore?	card_not_working
+I can't find my code	passcode_forgotten
+Which fiat currencies are supported for holding and exchanging?	fiat_currency_support
+I just pulled money out of the ATM and was charged a fee, WHy?	cash_withdrawal_charge
+I made a transaction at an atm and was charged extra for it.	cash_withdrawal_charge
+How can I transfer money to this account from another bank?	transfer_into_account
+What card types can I choose from?	visa_or_mastercard
+It's declining my transfers. Normally this works, but my card is being declined now and I've tried more than once.	declined_transfer
+I took out some cash from the ATM but it is still showing the transaction is pending.	pending_cash_withdrawal
+I haven't received my virtual card yet!!	getting_virtual_card
+Are there any fiat currencies that you will not work with?	fiat_currency_support
+The transfer keeps rejected, I tried to transfer some money to friends but it keeps getting rejected for some reason, Would you please check the issue?	failed_transfer
+Why is the payment I made still pending?	pending_card_payment
+Visa and Mastercard can both be received from you?	visa_or_mastercard
+I would like to pay by cheque.	top_up_by_cash_or_cheque
+I want to top up by transfering accounts, What do I do?	transfer_into_account
+My card arrived but I don't know how to get my PIN.	get_physical_card
+I don't know how to verify my identity.	verify_my_identity
+Is there a fee to get an actual card?	order_physical_card
+I received incorrect amount of cash from an ATM today. It was more than I wanted. What do I do now?	wrong_amount_of_cash_received
+I transferred some money but the balance stayed the same.	balance_not_updated_after_bank_transfer
+THere is some cash that is showing up on my app.	cash_withdrawal_not_recognised
+Why has my card payment not yet been applied to my account?	pending_card_payment
+Can I exchange currencies with the app?	exchange_via_app
+There was an unexpected fee for my cash withdrawal!	cash_withdrawal_charge
+Why has another fee been placed on my statement?	extra_charge_on_statement
+Hi, I made a transfer yesterday that I need to reverse.  I need to put the money in a different account.	cancel_transfer
+The payment for the card did not go through	declined_card_payment
+The money that I have transferred hasn't arrived yet.	balance_not_updated_after_bank_transfer
+Is there a way to track the new card you sent me?	card_arrival
+Where is my last money transaction	transfer_not_received_by_recipient
+My card payment has been declined	declined_card_payment
+Why hasn't the transaction I did to my friend arrive yet?	transfer_not_received_by_recipient
+I was just shopping online for some tickets and the payment was declined by you.  Why has this happened?  I am certain I have the right amount of funds in my account.	declined_card_payment
+I want to revert a transaction I did this morning	cancel_transfer
+There was an extra fee when I was at the ATM.  Why?	cash_withdrawal_charge
+Exchange rate for my card payment is wrong	card_payment_wrong_exchange_rate
+Can you explain why I was charged extra when I used the card?	card_payment_fee_charged
+There is an unexplained $1 charge on my statement, can you please explain this to me?	extra_charge_on_statement
+I don't understand why you aren't accepting my transfer. After a couple times trying I'm still getting an error message.	beneficiary_not_allowed
+How can I verify my top up?	verify_top_up
+Is there a transaction limit for a disposable card?	disposable_card_limits
+I want to start using my card.	activate_my_card
+How can I change my information?	edit_personal_details
+Can I get the Mastercard?	visa_or_mastercard
+Is topping up with Apple Pay possible?	apple_pay_or_google_pay
+My top up is not showing up in my wallet!	topping_up_by_card
+How is my ATM deposit not available yet?	pending_cash_withdrawal
+I asked for cash but I did not receive all the funds	wrong_amount_of_cash_received
+Where is my virtual card?	getting_virtual_card
+What is the price for a physical card?	order_physical_card
+If I am running low on credit can I auto top up?	automatic_top_up
+Where did the money I deposit go??	balance_not_updated_after_cheque_or_cash_deposit
+I'm due a refund and it is not on my statement.	Refund_not_showing_up
+Will I be charged for topping up?	top_up_by_card_charge
+When I was transferring money I was charged extra why?	transfer_fee_charged
+Can you tell me why I wasn't able to complete my transfer?	failed_transfer
+Please help me figure out the reason for the odd withdrawal amount from my account?	cash_withdrawal_not_recognised
+Should I order a new card before I freeze my original one?	compromised_card
+what is the matter?	direct_debit_payment_not_recognised
+How do I get a real card?	order_physical_card
+How long will I have to wait for my money to be in my account?	transfer_timing
+I'm still waiting for a money transfer to show.	pending_transfer
+I left my phone at the hotel.	lost_or_stolen_phone
+My top-up was cancelled and I am writing to ask if I need to complete the 3D Secure authentication section before I submit it.	top_up_reverted
+Can I use my account without verifying identity?	why_verify_identity
+Can another family member hold a card for this account in addition to mine?	getting_spare_card
+Where is my top-up generated from?	verify_source_of_funds
+I transferred money from the US, How long does it take?	transfer_timing
+Is it normal to get charged extra when i use my card?	card_payment_fee_charged
+My top-up appears to be pending.	pending_top_up
+How come I can't put money in my top up account?	top_up_reverted
+Why did the ATM cancel my transaction	declined_cash_withdrawal
+I'm going to cancel a purchase.	request_refund
+What exchange rate is it?	exchange_rate
+Hi, i was checking my old statements and found a discrepancy with a large amount transaction. I am sure it was not done by me. Please let me know if i could raise a dispute as its very late.	direct_debit_payment_not_recognised
+If I want to send out more cards, do you charge extra for that?	getting_spare_card
+I can't see my Google Pay top up. Is it not working correctly?	apple_pay_or_google_pay
+Where am I able to get my PIN unblocked?	pin_blocked
+Hae, I already completed my 3D secure authentication when I tried the crypto top up but it's still not working for me. I also don't understand what happened to the money in my account	top_up_reverted
+Can I get a physical card	order_physical_card
+Why was my payment declined? I keep trying my card but it's not working.	declined_card_payment
+DO I have to be 18 to open an account?	age_limit
+I went to top up my card and it was cancelled.	top_up_reverted
+I wasn't able to withdraw money at an ATM	declined_cash_withdrawal
+Why have I been charged a fee for cash withdrawal?	cash_withdrawal_charge
+Am I able to top up with Apple Pay?	apple_pay_or_google_pay
+I tried to get funds in hard cash but it was rejected!	declined_cash_withdrawal
+Is there a limit on disposable cards?	disposable_card_limits
+Is there a fee for topping up by charge card?	top_up_by_card_charge
+Can topping up be automatic?	automatic_top_up
+I transferred money, but I think I've been charged a fee for it?	transfer_fee_charged
+I would like to give another card to my daughter, how can I proceed?	getting_spare_card
+How does the exchange rate work?	exchange_rate
+Why are my transfers getting declined?	declined_transfer
+Will I get charged for sending out more cards?	getting_spare_card
+How do I get money from an ATM using my card? I can't seem to get it to work? I used the pin I had, but maybe it's wrong? What do I need to do to fix this?	declined_cash_withdrawal
+why would a merchant refuse a payment	reverted_card_payment?
+Am I supposed to verify my identity?	why_verify_identity
+Could you please put a freeze on my card?  There are some payments on it that I don't know where they came from.	card_payment_not_recognised
+Is there something wrong with the transferring functions? I keep trying to transfer funds and only get an error message.	beneficiary_not_allowed
+why does it take so long to verify my id	unable_to_verify_identity
+How can I hold and exchange fiat currencies?	fiat_currency_support
+I have been waiting for my top-up to complete and it still hasn't gone through.	pending_top_up
+I would like a virtual card...how do I get one?	getting_virtual_card
+What is this card payment, I do not recognize it.	card_payment_not_recognised
+Will you help me unblock my pin after putting in the wrong pin too many times?	pin_blocked
+I've been waiting for a top up to come through for over an hour and I need the funds now.	pending_top_up
+What is the source of the money in my account?	verify_source_of_funds
+Can I have my money refunded for an item	request_refund
+Is there a daily auto top-up limit?	automatic_top_up
+My transfer is quite urgent, and finally worked out, but I've been waiting for it for quite some time. Can you tell me how long to expect to wait for it?	pending_transfer
+I no longer have my phone.	lost_or_stolen_phone
+phone is gone, dont want others to access account information	lost_or_stolen_phone
+Is there a Mastercard ATM nearby?	atm_support
+make my account go away	terminate_account
+I was charged a fee for using my card. Can I check that in any way? It seems like there are occasions that I am charge for fees. Can you elaborate this?	card_payment_fee_charged
+I lost my passcode.	passcode_forgotten
+My payment was cancelled	reverted_card_payment?
+what are weekend rates for getting cash	wrong_exchange_rate_for_cash_withdrawal
+What do I need to do to receive my payroll in the account?	receiving_money
+When I received my cash; they had my exchange rate incorrect.	wrong_exchange_rate_for_cash_withdrawal
+how many transactions can go on a disposable card?	disposable_card_limits
+What should I do once my card expires?	card_about_to_expire
+Hi there, I came across a problem with my top up and my money is missing, was there a system error?	top_up_reverted
+How do I activate my card that just arrived?	activate_my_card
+I did a payment but it was reverted by the app	reverted_card_payment?
+Hi, I am disappointed to see such a bad exchange rate and hope you can assist me. Would you be able to confirm the official interbank exchange rate for me?	card_payment_wrong_exchange_rate
+I prefer using Mastercard, can I substitute?	visa_or_mastercard
+Are there restrictions to transfers to certain accounts?	beneficiary_not_allowed
+A fee shouldn't apply to me if my card was stolen.	lost_or_stolen_card
+Why is my friend not getting a transaction arrival yet?	transfer_not_received_by_recipient
+What is the longest a US transfer should take to appear in my account?	transfer_timing
+Are you ripping people off with the exchange rate?  It can't be that atrocious between banks.	card_payment_wrong_exchange_rate
+What countries are getting support?	country_support
+My card payment was reverted	reverted_card_payment?
+What will happen after my card expires?	card_about_to_expire
+Hi, I am new to this. Are there any additional fees or charges that I should be aware of? I was looking through the app and I thought I saw something in regards to fee-free allowances.	card_payment_fee_charged
+I would like to use cash to make a top-up?	top_up_by_cash_or_cheque
+I was supposed to receive my new card by now, but it hasn't came in.	card_arrival
+You have declined my  payment.	declined_card_payment
+Can you let me know why my cash withdrawal is still pending?	pending_cash_withdrawal
+How do I get this damn virtual card to work?	virtual_card_not_working
+Some cash withdrawal that I didn't make showed up in the app.	cash_withdrawal_not_recognised
+I did not ask for the amount of cash that I received.	wrong_amount_of_cash_received
+Could I reactivate my card please, I couldn't find it but it turns out it was in my jacket pocket.	card_linking
+I sent a payment to the wrong account number - can that transaction be reversed? I am worried about accidentally giving my money away.	cancel_transfer
+Tell me why my card payment would be reverted.	reverted_card_payment?
+I need my account checked because it appears there is a problem. I attempted to update my balance yesterday using cheque but it doesn't seem to have worked.  Should this not be faster than this?	balance_not_updated_after_cheque_or_cash_deposit
+Why can I not make a transfer?	failed_transfer
+I think there was an error in your exchange rate.	card_payment_wrong_exchange_rate
+I'm not sure why my card didn't work	declined_card_payment
+It can take between 10 minutes to an hour before you hear back from us about the verification result.  Please try again if your verification fails.  Make sure you double check that images are clear and all document photos are readable without blur or glare. You have to be 18 or older and live in the European Economic area or live in Switzerland to be able to open an account.	unable_to_verify_identity
+When I use my card, it says decline?	declined_card_payment
+Help my find the verification code on a top-up.	verify_top_up
+How do I make my top up complete?	pending_top_up
+Will I be charged for exchanging foreign currencies?	exchange_charge
+How do I add my new card to the app?	card_linking
+I moved to a new city and need to change my address	edit_personal_details
+I am still awaiting the money I put into my account this morning.	balance_not_updated_after_cheque_or_cash_deposit
+I can't verify identity through the app.	unable_to_verify_identity
+I don't understand why my payment was declined. I thought everything was ok.	declined_card_payment
+Why is a transfer I did recently still processing?	pending_transfer
+Where's the button to top up with a cheque?	top_up_by_cash_or_cheque
+My refund isn't showing up	Refund_not_showing_up
+A foreign purchase I made has the incorrect rate applied to it.	card_payment_wrong_exchange_rate
+Am I able to change my PIN at certain cash machines?	change_pin
+What countries are you available in?	country_support
+Which part of the app do I order virtual cards from?	getting_virtual_card
+I was charged a fee when I transferred my funds, why?	transfer_fee_charged
+How do I order a new card if my current one is about to expire?	card_about_to_expire
+It declined my transfer.	declined_transfer
+Your exchange rate was off.  I was charged extra on my payment that I made last Saturday.	card_payment_wrong_exchange_rate
+I tried using my credit card with top up but it didn't work! I thought it was supposed to?	top_up_failed
+Please help me as my online payment is getting declined since yesterday while i am trying to purchase something online.	declined_transfer
+I need help from someone in your department with finding my nearest ATM.	atm_support
+Do I get a discount if I exchange currencies a lot?	exchange_charge
+Why did you charge me extra?	extra_charge_on_statement
+I got some cash at an ATM earlier, but now app is showing that I withdrew more than I did.  Help!	wrong_amount_of_cash_received
+Why has my money been returned back to my card.	reverted_card_payment?
+I don't have a way to prove my identity.	unable_to_verify_identity
+Can you tell me what I need for identity validation?	verify_my_identity
+I checked the app and there is a payment that I don't recognize.	card_payment_not_recognised
+I only made one transaction at this place, but it's appearing multiple times.	transaction_charged_twice
+Why hasnt my top up gone through yet?? I thought it was suppose to be instant? Am I wrong? Why is it still pending?	pending_top_up
+what is the exchange rate for foreign ATMs	wrong_exchange_rate_for_cash_withdrawal
+I have not received an item i ordered yet so i want to cancel it and get a refund. Can you please guide me on how to go about it?	request_refund
+I only have one more US credit card, is it accepted?	supported_cards_and_currencies
+Can you tell me why a cheque deposit isn't showing in my balance?	balance_not_updated_after_cheque_or_cash_deposit
+Do you have the option for a disposable virtual card?	get_disposable_virtual_card
+I still don't have a card. How do I get one?	order_physical_card
+How do I link the new card that I just got from you guys?	card_linking
+I got cash from an ATM earlier but it shows up as pending in the app. How can this still be pending, I already received the cash?	pending_cash_withdrawal
+What happens with the duplicate charge in my account?	transaction_charged_twice
+To many charges on my card how do I go about fixing that?	transaction_charged_twice
+why is there a service fee on my statement?	extra_charge_on_statement
+Tell me why there is an extra charge when I used an ATM?	cash_withdrawal_charge
+I tried to get money out of an ATM in Notting Hill earlier, but it didn't work. Is my card actually working? It's the first time I tried using it	declined_cash_withdrawal
+I am unable to use my card for some reason.	card_not_working
+Is there a way to find an ATM around me?	atm_support
+Why is the exchange rate different than expected on my transaction?	wrong_exchange_rate_for_cash_withdrawal
+Why was my transaction declined?	failed_transfer
+I have a cash withdrawal that I don't recognize.	cash_withdrawal_not_recognised
+Why do I need to verify a top-up?	verify_top_up
+How do I protect my phone from someone using it after I lost it.	lost_or_stolen_phone
+Must I verify my identity?	why_verify_identity
+How do I verify the top up?	verify_top_up
+Tell me more about getting a virtual disposable card.	get_disposable_virtual_card
+I prefer Visa, can I get that instead?	visa_or_mastercard
+Why is the cash withdraw I made shown as pending?	pending_cash_withdrawal
+Help me verify my identification.	unable_to_verify_identity
+Can you help me track my card?	card_arrival
+How do I go forth on holding money in multiple currencies?	fiat_currency_support
+My daughter needs an acount.	age_limit
+A transfer I sent was received in less than the full amount I originally sent, and now I have to send additional funds to cover the full amount. What happened?	transfer_fee_charged
+Just received my replacement card, what steps do i need to take to activate it?	activate_my_card
+Why is my transfer stuck on pending?	pending_transfer
+What does a 'pending payment' mean?	pending_card_payment
+I've been waiting for a pending transfer to go through for a while now, and don't see any reason why I've been waiting this long.  I know that I have the correct account details, so that isn't the problem.	pending_transfer
+There is a cash withdrawal on my statement that I don't recognize	cash_withdrawal_not_recognised
+When will I see an updated balance after a transfer?	balance_not_updated_after_bank_transfer
+I need a second card on this account for my daughter.	getting_spare_card
+How long does it take transfers from Europe to complete?	transfer_timing
+There are transactions that I did not make on my account, I think someone has my information.	compromised_card
+Could you tell me about the restrictions that are on the disposable cards?	disposable_card_limits
+This morning, I tried to send money to my friends several times but each time the transfer fails.  Why does it keep failing?  The transfer gets rejected but I do not know what is causing it to be rejected.	failed_transfer
+Can I use my Apple Watch to pay for gas?	apple_pay_or_google_pay
+Is there a limit on how much I can top up?	top_up_limits
+there were two of the same charge to my account	transaction_charged_twice
+What are the charges if I exchange foreign currency?	exchange_charge
+My new card has arrived, what's the activation procedure?	activate_my_card
+There's something wrong on my bill. A transfer fee should never have been added.	transfer_fee_charged
+Hello, can you please inform me about international transfers? I am expecting a transfer I made from France two days ago and would like to know when it will arrive.	balance_not_updated_after_bank_transfer
+My cards were stolen	lost_or_stolen_phone
+Do I need to re-apply to order a new card when my old one's about to expire?	card_about_to_expire
+Can I change my in at all ATM's?	change_pin
+My card information seems to have been stolen! I don't know where these transactions came from and I've never even been to the city where they were made. I need my card blocked ASAP so this person can't keep using my card.	compromised_card
+What countries have access to this service.	country_support
+Can I have a discount if I exchange currency often?	exchange_charge
+How long does transfers take?	balance_not_updated_after_bank_transfer
+Is there a few for exchanging foreign currencies?	exchange_charge
+Is there a charge for receiving money?	top_up_by_bank_transfer_charge
+I want to top up by using a credit card, which ones do you support?	supported_cards_and_currencies
+I can't seem to be able to use my card	card_not_working
+There's a $200 withdrawal on my statement that seems fishy to me.	cash_withdrawal_not_recognised
+Why is my transfer not showing	balance_not_updated_after_bank_transfer
+How do I find the code needed in order to verify my top-up card?	verify_top_up
+I want to reactivate my card, I found it after thinking it was lost.	card_linking
+There is money missing from my account.	cash_withdrawal_not_recognised
+Show me how to look at the source of my current funds?	verify_source_of_funds
+I do not understand why there is a pending 1£ charge on my statement, I do not recall making this transaction. Can you please tell me what this is about?	extra_charge_on_statement
+Why would a merchant revert a card payment?	reverted_card_payment?
+I would like to get additional physical cards at this time	getting_spare_card
+The amount of cash I selected to withdraw is different than what I received.	wrong_amount_of_cash_received
+I tried withdrawing money in another country and the exchange rate was wrong.	wrong_exchange_rate_for_cash_withdrawal
+Please tell me how to get a refund for something I bought.	request_refund
+I thought I left my card at a restaurant, but they claim not to have it and now I don't have any idea where it is. I'm so worried that someone might be using it fraudulently. Is there any way to look into this?	lost_or_stolen_card
+Can I add to my account using a cheque?	top_up_by_cash_or_cheque
+My transfer has not arrived.	transfer_not_received_by_recipient
+I was unable to transfer money from my account.	failed_transfer
+your company is awful, i would like to delete my account please	terminate_account
+The ATM didn't give me the amount I asked for	wrong_amount_of_cash_received
+Why do I have a strange payment on my statement?	card_payment_not_recognised
+I need to renew a card that's about to expire	card_about_to_expire
+Report stolen card	lost_or_stolen_card
+I think I lost my phone.  Is there a way to prevent anyone from getting into my account on it?	lost_or_stolen_phone
+My cash withdrawal was declined.  Why?	declined_cash_withdrawal
+I just transferred some money. Why was I charged a fee?	transfer_fee_charged
+Can I pick between Visa and Mastercard?	visa_or_mastercard
+Will you accept my credit card?	supported_cards_and_currencies
+Can you tell me how I go about activating a new card?	activate_my_card
+Where is the cash I deposited?	balance_not_updated_after_cheque_or_cash_deposit
+Cancelling a transaction	cancel_transfer
+I need assistance with understanding which fiat currencies that you support.	fiat_currency_support
+If I bought something I didn't like, can I get a refund?	request_refund
+When will the money from a transfer be made available on my account?	transfer_timing
+Where do I access the auto top-up function?	automatic_top_up
+how much does it cost to add money to my card	top_up_by_card_charge
+Where can I find the verification code for the card top-up?	verify_top_up
+What is the verification process used for?	verify_top_up
+I sent some money but the intended recipient says it hasn't arrived	transfer_not_received_by_recipient
+I'm expecting to see a cash withdrawal but I'm still waiting.	pending_cash_withdrawal
+May monthly payment to Apple has been declined but I don't know why, they are going to re-try the payment and it must go through, can you tell me what is going on please?	declined_card_payment
+How often can I top up my account?	top_up_limits
+show me how to top up	topping_up_by_card
+I don't recognize what this payment on my statement would be for.	card_payment_not_recognised
+My disposable virtual card was rejected.	virtual_card_not_working
+How much will it cost me for a SEPA transfer?	top_up_by_bank_transfer_charge
+Hi, status of one of my card payment is still pending from a while. Please let e know how much time it will take to complete.	pending_card_payment
+Was there a problem with topping up?	top_up_failed
+Can I make a withdraw in my home currency at an ATM? I'm on vacation and am a bit worried because I don't have cash with me. Is it possible to withdraw here without any additional costs or fees?	wrong_exchange_rate_for_cash_withdrawal
+I would like to get an item refunded	request_refund
+Can you tell me what currencies I can use to add money?	supported_cards_and_currencies
+My card is about to expire.  Are there any costs with a replacement card?  How quick do you mail it to me?	card_about_to_expire
+please help me with this transfer, for some reason i can't transfer to a cryptocurency account	declined_transfer
+How do I know where my funds come from?	verify_source_of_funds
+How do I top up my account using transfer?	transfer_into_account
+How do I know when my new card is going to arrive?	card_arrival
+My product was faulty.	request_refund
+I am trying  to  transfer and  it has been  stopped why is  that	beneficiary_not_allowed
+What documents are required for proof of identity?	verify_my_identity
+Can my PIN be changed in any cash machine?	change_pin
+I want a new card	getting_spare_card
+I've been charged an extra £1 and I don't know why	extra_charge_on_statement
+What does the €1 fee mean?	extra_charge_on_statement
+What is needed for identity verification?	verify_my_identity
+I don't understand why I was charged an additional fee for this service. Can you clarify?	card_payment_fee_charged
+There is a missing transfer on my account	balance_not_updated_after_bank_transfer
+My account is locked because I used the wrong pin too many times.  Please help.p	pin_blocked
+Why hasn't my money transfer showed up yet?	balance_not_updated_after_bank_transfer
+My card has been compromised. I see a bunch of online shopping charges that I didn't make. I need to freeze it immediately.	compromised_card
+Why am I being charged again?	transaction_charged_twice
+I withdrew some cash and the exchange rate seems to be wrong.	wrong_exchange_rate_for_cash_withdrawal
+The exchange rate looks wrong on a holiday purchase	card_payment_wrong_exchange_rate
+I topped up my account but the app failed to process it.	top_up_failed
+With this app, will I be able to exchange currency?	exchange_via_app
+Why is my card payment still pending?	pending_card_payment
+Why can I not choose the card i want?	visa_or_mastercard
+Where do I get a disposable card	get_disposable_virtual_card
+Can I use my apple watch to top-up?	apple_pay_or_google_pay
+Can I make an account for my daughter?	age_limit
+I was charged twice for a few things this week when I only bought one. What do I do?	transaction_charged_twice
+I would like to delete my account. I'm not happy with the service you're providing	terminate_account
+Why do I need to verify my account?	why_verify_identity
+What's the transfer time from Europe?	transfer_timing
+Can you tell me how to get my card on the app?	card_linking
+Can I top up using my apple watch?	apple_pay_or_google_pay
+If my phone was stolen, do I need to do anything?	lost_or_stolen_phone
+What is the process when my card is due to expire?	card_about_to_expire
+How can I find a way to transfer money without it being rejected?	declined_transfer
+I have no way to prove my identity.	unable_to_verify_identity
+There is an urgent transfer I need to make from China.  How long will it take to get it processed?	transfer_timing
+I made a top up yesterday but it's still pending.	pending_top_up
+The applied exchange rate is wrong for the cash I got abroad.	wrong_exchange_rate_for_cash_withdrawal
+There is a payment showing that I know I didn't make. I didn't use my card that day and I'm sure of it. Can you please cancel that transaction and return my money? Thankyou	direct_debit_payment_not_recognised
+Let me know when applying extra charges for payments. I just checked the app and saw a fee that I wasn't warned about.	card_payment_fee_charged
+Why haven't I gotten my payment yet?	pending_card_payment
+I would like to top up via credit card, which credit cards do you support?	supported_cards_and_currencies
+I have been charged way to much so something is apparently wrong with my exchange. I changed Russian Ruble into UK pounds.	card_payment_wrong_exchange_rate
+Has my top up gone through?	pending_top_up
+How can I check on the status of my new card?	card_arrival
+Please explain the extra fee when I paid with the card.	card_payment_fee_charged
+I was charged an extra dollar on my account and would like to know the reason.	extra_charge_on_statement
+I can't find how to top up my account with a check.  Where is that?	top_up_by_cash_or_cheque
+My statement is showing an extra £1 charge and I am unsure why. Is there an explanation for this?	extra_charge_on_statement
+I requested $100 but only got $20	wrong_amount_of_cash_received
+Where do I go to see the source of my available money?	verify_source_of_funds
+Is there an alternative to a physical card?	getting_virtual_card
+I need a transaction reversed from my account.	transaction_charged_twice
+why hasn't my virtual card arrived?	getting_virtual_card
+I was only given $20 when I tried to get $100	wrong_amount_of_cash_received
+I need your help immediately.  There is a 500£ cash withdrawal that I know nothing about.  I think my account has become compromised.	cash_withdrawal_not_recognised
+I noticed a charge for a transfer.  Why is this?  Is it because I did an international transfer?	transfer_fee_charged
+There was an extra fee when I paid with my card, why was i charged this extra fee?	card_payment_fee_charged
+how do i get a pin for my card	get_physical_card
+I was checking my statement and my top-up has been reverted	top_up_reverted
+There is a fee from a transfer that is not supposed to be here!	transfer_fee_charged
+I was checking the app for my account and I noticed there is an authorized charge on it.  I haven't used my card today, so I know I didn't do it.  I would like the transaction reversed and the money replaced.	direct_debit_payment_not_recognised
+There is a direct debit transaction that I did not authorize	direct_debit_payment_not_recognised
+Hi, I was just checking over my card transactions and notice I was charged twice for a restaurant visit this week. Can you please remove one of these false charges from my account?	transaction_charged_twice
+What's the limit for auto top-up?	automatic_top_up
+How do I get a refund for something I bought?	request_refund
+Am I able to do a SWIFT transfer with you?	top_up_by_bank_transfer_charge
+How will I know when I can use the auto top-up feature?	automatic_top_up
+I need to reverse a transaction	cancel_transfer
+Is my card actually working? It's the first time I tried using it	declined_cash_withdrawal
+I'm checking my statements and I saw a purchase from a couple weeks ago that I don't recall at all. Not only that, but I don't even know the business that charged it. I'm not sure if I did it or not, but can we check it and see if we can trace it back to make certain?	direct_debit_payment_not_recognised
+google pay top up did not work	apple_pay_or_google_pay
+Can I top up using my cheque?	top_up_by_cash_or_cheque
+What is the number of transactions for one disposable card?	disposable_card_limits
+Why did the cash deposit not show up in my account?	balance_not_updated_after_cheque_or_cash_deposit
+I'd like to know what you guys look at to decide an exchange rate	exchange_rate
+I don't know what my passcode is, can you help?	passcode_forgotten
+What is the procedure for activating this card?	activate_my_card
+Can I order a new card?  Mine is about to expire.	card_about_to_expire
+Is there a way to get a virtual card?	getting_virtual_card
+Why does my transfer still say it is pending?	pending_transfer
+I need to get hold of a physical card?	order_physical_card
+I was denied cash at an ATM	declined_cash_withdrawal
+Is there a tracking number for the card?	card_arrival
+I tried to buy something online yesterday and was told declined. I got the same error today after trying again. What is the problem?	declined_transfer
+I'm supposed to have a refund but it isn't there	Refund_not_showing_up
+I think my card payment has been refunded	reverted_card_payment?
+How long will a transfer from the US take?	transfer_timing
+A payment is showing twice. Why is that?	transaction_charged_twice
+Is there an extra fee for using an ATM?	cash_withdrawal_charge
+Is my PIN sent separably?	get_physical_card
+Can I use your app to exchange USD for GBP?	exchange_via_app
+Why is the exchange rate so high? This should have been a much higher amount of cash for that to apply.	wrong_exchange_rate_for_cash_withdrawal
+How can I transfer some money into a foreign account?	transfer_into_account
+Everything was stolen from me.  I can't use the app.  Please advise.	lost_or_stolen_phone
+I noticed my transfer was declined.	declined_transfer
+How do I delete my account now	terminate_account
+what do I need to do if the ATM kept my card?	card_swallowed
+How simple is it to transfer money into my account?	transfer_into_account
+What do I do if my Google pay top up isn't working?	apple_pay_or_google_pay
+I think I was charged by mistake on my debit statement.	direct_debit_payment_not_recognised
+While abroad, I got cash, and the exchange rate applied is wrong.	wrong_exchange_rate_for_cash_withdrawal
+How long do bank transfers usually take?	balance_not_updated_after_bank_transfer
+Please verify my top-up card	verify_top_up
+I've got a problem - I purchased an item maybe a week or two ago, but to the payment came back to my account! I've already got what I ordered, so why are the payment funds showing up in my account?	reverted_card_payment?
+Can I track my card while it is in the process of delivery?	card_arrival
+I'm confused as to why my transfer is still listed as pending on the website.	pending_transfer
+I have my card now how do I activate it?	activate_my_card
+Why was I unable to complete my transfer?	failed_transfer
+I attempted to get cash in the ATM but I was denied	declined_cash_withdrawal
+Hello, I wanted to get some more information about ATM Withdraw Fee. I have been withdrawing money for years and notice this charge on my account. I always thought it was no fee since its you guys ATM company that I am withdrawing. Can you please explain?	cash_withdrawal_charge
+When I bought something abroad, the exchange rate was wrong.	card_payment_wrong_exchange_rate
+would I need to pay extra for exchanging currencies?	exchange_charge
+My new card just arrived, how can I activate it?	activate_my_card
+Why can't I complete my transfer?	failed_transfer
+is my pin the same as my passcode	get_physical_card
+Why can't I make multiple payments with my virtual card?	virtual_card_not_working
+I withdrew cash and was charged	cash_withdrawal_charge
+I could not get the top-up to go through	top_up_failed
+Will my card work at all merchant locations?	card_acceptance
+Is there a reason my transfer failed?	failed_transfer
+Where in the app do I go to verify my identity?	verify_my_identity
+I am seeing in the App a diret debit that its not mine	direct_debit_payment_not_recognised
+How do I authenticate my top-up?	verify_top_up
+Where can I use this card?	card_acceptance
+You've incorrectly calculated the exchange rate for my cash withdrawal	wrong_exchange_rate_for_cash_withdrawal
+I would like to change my name after marriage.	edit_personal_details
+I paid with card and I was charged a fee	card_payment_fee_charged
+Is it possible to transfer money to my credit card?	topping_up_by_card
+I've just been married and need to update my name	edit_personal_details
+How does someone else top up my account	topping_up_by_card
+Why is the payment from my card still pending?	pending_card_payment
+I wanted to take out $100 but I only got $20.	wrong_amount_of_cash_received

TEXTOIR-main/data/snips/dev.tsv

@@ -0,0 +1,701 @@
+text	label
+i d like to have this track onto my classical relaxations playlist 	AddToPlaylist
+add the album to my flow español playlist 	AddToPlaylist
+add digging now to my young at heart playlist	AddToPlaylist
+add this song by too poetic to my piano ballads playlist	AddToPlaylist
+add this album to old school death metal	AddToPlaylist
+i need to add baro ferret to the urban hits under my name	AddToPlaylist
+add the album to the might and myth power metal playlist 	AddToPlaylist
+to the travelling playlist  please add this david gahan song 	AddToPlaylist
+please add some pete townshend to my playlist fiesta hits con lali	AddToPlaylist
+i d like for kasey chambers s tune to be an addition to my chips and salsa playlist 	AddToPlaylist
+add recalled to life to this is alejandro fernández	AddToPlaylist
+add nuba to my metal party playlist	AddToPlaylist
+add jo stafford music to the workout twerkout playlist	AddToPlaylist
+put jean philippe goncalves onto my running to rock 170 to 190 bpm 	AddToPlaylist
+add the song virales de siempre by the cary brothers to my gym playlist 	AddToPlaylist
+onto jerry s classical moments in movies  please add the album 	AddToPlaylist
+add beyond the valley of 1984 in playlist folk music at the gaslight café	AddToPlaylist
+add jerry calliste  jr to my te quiero playlist 	AddToPlaylist
+add porter wagoner to the the sleep machine waterscapes playlist 	AddToPlaylist
+add the artist mike to the sexy as folk playlist 	AddToPlaylist
+add brazilian flag anthem to top 100 alternative tracks on spotify	AddToPlaylist
+add andy hunter to my evening commute playlist 	AddToPlaylist
+put petar georgiev kalica onto the old school hip hop playlist 	AddToPlaylist
+can you add larry heard to my laundry playlist	AddToPlaylist
+put vandemataram srinivas s track onto hiphop hot 50 	AddToPlaylist
+add millie corretjer to the rhythm playlist	AddToPlaylist
+add give us rest to my 70s smash hits playlist 	AddToPlaylist
+add this track to my hands up playlist	AddToPlaylist
+i d like for you to add bobby brown to my enamorándose playlist 	AddToPlaylist
+add jonathan sprout album to my this is miranda lambert playlist	AddToPlaylist
+add ireland in the junior eurovision song contest 2015 to my jazzy dinner playlist	AddToPlaylist
+add the album to the the sweet suite playlist 	AddToPlaylist
+add sarah slean to my playlist mellowed out gaming	AddToPlaylist
+add this album to the spanish beat playlist	AddToPlaylist
+add lofty fake anagram to the la mejor música de bso playlist 	AddToPlaylist
+add the track to the work playlist 	AddToPlaylist
+add a song to this is racionais mc s	AddToPlaylist
+add track in my playlist called hands up	AddToPlaylist
+can you put this song from yutaka ozaki onto my this is miles davis playlist	AddToPlaylist
+add a track to playlist cena con amigos	AddToPlaylist
+add the famous flower of serving-men to my evening acoustic playlist 	AddToPlaylist
+add a song to indie hipster	AddToPlaylist
+add the 40 cal tune to the laundry playlist 	AddToPlaylist
+add the album to my perfect concentration playlist 	AddToPlaylist
+add the matt murphy tune to the flow español playlist 	AddToPlaylist
+add a very cellular song to masters of metal playlist	AddToPlaylist
+can i put this tune onto my sin estrés playlist	AddToPlaylist
+i d like to add jordan rudess onto the divertido para niños playlist 	AddToPlaylist
+add kent james to the disney soundtrack 	AddToPlaylist
+add the artist adam deibert to my perfect concentration playlist 	AddToPlaylist
+can you put the artist giovanni giacomo gastoldi onto the chill out music playlist	AddToPlaylist
+add the album to the hot 50 playlist 	AddToPlaylist
+add the artist pete murray to my relaxing playlist	AddToPlaylist
+add the track to the drum & breaks playlist 	AddToPlaylist
+for my fantastic workout can you add sara bareilles	AddToPlaylist
+add the boy george track to the emo forever playlist 	AddToPlaylist
+add ted heath to the road trip playlist 	AddToPlaylist
+can you add last of the ghetto astronauts to the playlist called black sabbath the dio years	AddToPlaylist
+add this artist to showstopper being mary jane	AddToPlaylist
+put the artist onto top latin alternative 	AddToPlaylist
+add michael wittig music to country icon playlist	AddToPlaylist
+add highway patrolman in my playlist this is al green	AddToPlaylist
+add richard mcnamara newest song to the just smile playlist	AddToPlaylist
+add annesley malewana album to playlist indietronic	AddToPlaylist
+add the artist to my dishwashing playlist 	AddToPlaylist
+add this artist to fairy tales playlist	AddToPlaylist
+add muzika za decu to my crash course playlist	AddToPlaylist
+add a derek watkins tune to this is johnny cash	AddToPlaylist
+add our little corner of the world music from gilmore girls to my the funny thing about football is playlist 	AddToPlaylist
+add the current track to my this is tchaikovsky playlist	AddToPlaylist
+put abe laboriel onto the escapada playlist 	AddToPlaylist
+add abacab to beryl s party on fridays playlist	AddToPlaylist
+please add this track by paul mcguigan to the deep house playlist 	AddToPlaylist
+can you add the current tune to my calm before the storm playlist	AddToPlaylist
+please add the image of you to my playlist crate diggers anonymous	AddToPlaylist
+add a track to jazzy dinner	AddToPlaylist
+add the album to the hipster soul playlist 	AddToPlaylist
+add this tune to my sleepify playlist	AddToPlaylist
+add jack white to my playlist this is shakira	AddToPlaylist
+add tommy johnson to the metalsucks playlist	AddToPlaylist
+add the chris clark tune to my women of the blues playlist 	AddToPlaylist
+add an artist to jukebox boogie rhythm & blues	AddToPlaylist
+add this artist to my electronic bliss playlist	AddToPlaylist
+i need to add to my infinite indie folk list the works of rahim shah	AddToPlaylist
+add martin barre to my punk unplugged playlist 	AddToPlaylist
+add tierney sutton to my novedades viernes sudamérica playlist	AddToPlaylist
+add this tune to dorthy s 80 s party playlist	AddToPlaylist
+a very cellular song needs to be added to my masters of metal playlist	AddToPlaylist
+add toyan to my epic gaming playlist 	AddToPlaylist
+add the song to the mac  n cheese playlist 	AddToPlaylist
+add this artist to my spotlight on country 2016 playlist	AddToPlaylist
+add a song to my playlist madden nfl 16	AddToPlaylist
+add emilie autumn to my nação reggae playlist 	AddToPlaylist
+add farhad darya songs in virales de siempre	AddToPlaylist
+add a song in my all out 60s	AddToPlaylist
+add we have a theme song to my house afterwork playlist	AddToPlaylist
+add the song to my we everywhere playlist	AddToPlaylist
+add roel van velzen to my party of the century playlist 	AddToPlaylist
+add the artist to the political punks playlist 	AddToPlaylist
+add the album to my club hits playlist 	AddToPlaylist
+book a reservation for my babies and i	BookRestaurant
+book a reservation for a restaurant not far from ma	BookRestaurant
+i would like to book a restaurant in tanzania that is within walking distance for my mom and i	BookRestaurant
+book a reservation for an oyster bar	BookRestaurant
+book a reservation for 6 people for a creole tavern in montenegro	BookRestaurant
+i need a table in sacaton at a gluten free restaurant	BookRestaurant
+book sot for me and my grandfather nearby west reading	BookRestaurant
+book me and my nieces a reservation for a seafood restaurant in cle elum  ne on ascension day	BookRestaurant
+book spot for two at city tavern	BookRestaurant
+i want to book a brasserie for 3 people in netherlands antilles 	BookRestaurant
+book me a reservation for the best bistro	BookRestaurant
+book the best table in tanzania for 5 people at a diner	BookRestaurant
+i want to book a joint in a  spa 	BookRestaurant
+book a gastropub that serves turkish food for 4 people	BookRestaurant
+book spot for 7 at an indoor restaurant in mp now	BookRestaurant
+book a table in fiji for zero a m	BookRestaurant
+i want to book a restaurant for five people in sri lanka 	BookRestaurant
+i need a table for 5 at a highly rated gastropub in concord mn	BookRestaurant
+i want to book oregon electric station in north city 	BookRestaurant
+i need a table for 4 please confirm the reservation 	BookRestaurant
+book a popular restaurant for 5 people	BookRestaurant
+i want to book a joint close by the naomi s hostel for a meal for 8 people 	BookRestaurant
+i want to eat a delicatessen in thirteen hours that serves eastern european food	BookRestaurant
+book a reservation for nine people at a bakery in nunez	BookRestaurant
+book a reservation at tavern for noodle	BookRestaurant
+book spot for 4 in somalia	BookRestaurant
+i want to book albany pump station in buckholts washington now for a party of 9 	BookRestaurant
+i want to book a taverna in archer city for this spring for nine people 	BookRestaurant
+i want to book a top-rated brasserie for 7 people 	BookRestaurant
+book a reservation for 8 people in wardville  kansas	BookRestaurant
+table for breadline cafe in minnesota next friday	BookRestaurant
+i want to book a restaurant in niger for seven people 	BookRestaurant
+book spot for 9	BookRestaurant
+book me a reservation for a pub in cormorant for a party of nine	BookRestaurant
+book spot for my nieces and i at a tea house	BookRestaurant
+i want to book a jewish restaurant in gambia 	BookRestaurant
+book a reservation for the dome  edinburgh close to brooklawn	BookRestaurant
+book spot for 1 at town of ramsgate in merit	BookRestaurant
+book a spot for me and kathrine  at smithville	BookRestaurant
+i want to book a restaurant for my father in law and i in buckner a year from now 	BookRestaurant
+book a restaurant reservation in 6 weeks	BookRestaurant
+book a reservation for a bar with a spa nearby id	BookRestaurant
+book spot for four at cliff house  san francisco in martinique	BookRestaurant
+i need a table for 4 in saint helena at settha palace hotel	BookRestaurant
+i want to book a restaurant in frenier 12 years from now for 4 people 	BookRestaurant
+book seven in neighboring moorpark	BookRestaurant
+i want to eat by five pm in ne for a six people	BookRestaurant
+i want to book tupelo honey cafe in new jersey for five people 	BookRestaurant
+book a reservation for two at mickies dairy bar in weedsport	BookRestaurant
+book a table at a fried chicken restaurant	BookRestaurant
+book spot for mavis  sheila and i in syria at elevenses	BookRestaurant
+can you book me a table at windows on the world in cokeville   mi	BookRestaurant
+book me a table for 5 this year at cherwell boathouse	BookRestaurant
+book spot for six  at 8 pm at a coffeehouse in ne that serves hog fry	BookRestaurant
+i want to book a restaurant close-by in inman for five people 	BookRestaurant
+i need a table at eddie s attic in nevada for one	BookRestaurant
+book a reservation for an osteria restaurant for 4 people on november 4	BookRestaurant
+i want to book a top-rated restaurant close by in la for me  rebecca and loraine on 2/6/2020 	BookRestaurant
+book a reservation for 1 at a diner in wi	BookRestaurant
+book a reservation for 5 people at the top-rated brasserie restaurant	BookRestaurant
+book a table on 1/20/2023 for 5 people in mh	BookRestaurant
+book a table near pat s college	BookRestaurant
+i want to book a steakhouse in vimy ridge 	BookRestaurant
+i want a table at james d  conrey house in urbank california	BookRestaurant
+like to book a seat in monaco for the yankee doodle coffee shop	BookRestaurant
+i want to book a table in a restaurant in bouvet island 	BookRestaurant
+i would like to book a restaurant for souvlaki cuisine in the state of ne	BookRestaurant
+book a reservation for 10 people at an oyster bar with a pool within the same area of cowansburg for 10 pm	BookRestaurant
+book a reservation for velma  ana and rebecca for an american pizzeria at 5 am in ma	BookRestaurant
+book a spot for 4 in oklahoma at south street diner	BookRestaurant
+book a reservation for my mommy and i at a restaurant in central african republic	BookRestaurant
+book a reservation for five people for a tatar taverna in sargents	BookRestaurant
+phyllis ward and veronica need a table at a restaurant in 152 days	BookRestaurant
+book a reservation for ten at a restaurant in ohio	BookRestaurant
+i want to book a tea house that serves salade far from here at midnight in panama for two people 	BookRestaurant
+i want to book a food truck for seven people in the republic of the congo 	BookRestaurant
+i want to book a restaurant for ten people 	BookRestaurant
+lets eat near oakfield 17 seconds from now at ted peters famous smoked fish	BookRestaurant
+book sot for 7 at a restaurant that serves european in stringtown on feb  the 28th  2034	BookRestaurant
+book a restaurant for six at an outdoor cafe in åland	BookRestaurant
+book a table for 12 am  at our step mother s secondary residence within walking distance for one	BookRestaurant
+please book me a table at a pizzeria with a parking facility in ghana 	BookRestaurant
+book spot for four at a indoor pub within the same area of louisiana in one minute 	BookRestaurant
+please book me a restaurant	BookRestaurant
+book a reservation for me and my step brother at amt coffee in lakemoor	BookRestaurant
+i want to book a churrascaria in romeoville at ten a m for four people 	BookRestaurant
+table for 5 a m  at baker s keyboard lounge	BookRestaurant
+please book me a table at a bistro which serves lorna doone 	BookRestaurant
+i want to book a restaurant for six people in wagstaff ak 	BookRestaurant
+i would like to book a highly rated restaurant for a party of ten 	BookRestaurant
+i want to book a sundanese gastropub nearby in texas for 3 people on 5/20/2025 	BookRestaurant
+book a party of five at seagoville for 06:42	BookRestaurant
+book spot for 9 at thurmont	BookRestaurant
+i want to book a restaurant in sixteen seconds for 5 people in gold point montana 	BookRestaurant
+i want to eat in ramona	BookRestaurant
+book a party at their campus within the same area for churrascaria	BookRestaurant
+book me a reservation for a party of 3 at a pub in northern mariana islands	BookRestaurant
+i want to book a bougatsa restaurant in next year nearby penn for three people 	BookRestaurant
+book a reservation for nine people at the best pub nearby tangier in six months	BookRestaurant
+need a table somewhere in quarryville 14 hours from now	BookRestaurant
+what will the weather be faraway from here	GetWeather
+will there be fog in tahquamenon falls state park	GetWeather
+tell me the weather forecast for gibsland	GetWeather
+is there a storm now in nc	GetWeather
+what will the weather be in monument of lihula on december the 5th	GetWeather
+weather next year in dominica	GetWeather
+when will it be hot here	GetWeather
+what will the weather be in 1 day in kuwait	GetWeather
+what kind of weather will be in ukraine one minute from now	GetWeather
+humidity in olvey new hampshire	GetWeather
+what s the weather going to be in ut	GetWeather
+humidity not far from colorado city on november the 7th  2024	GetWeather
+what is the forecast for wyoming at stanardsville during the storm	GetWeather
+what will the weather be in north carolina	GetWeather
+what is the forecast starting 11 weeks from now nearby the state of wisconsin	GetWeather
+will it be rainy at sunrise in ramey saudi arabia	GetWeather
+check the forecast for nebraska 	GetWeather
+will it be warmer in north korea at nineteen o clock	GetWeather
+let me know the weather forecast around ten pm faraway from here in park narodowy brimstone hill fortress 	GetWeather
+will it be stormy in the ouachita national forest	GetWeather
+tell me if it will be snowy 8 hours from now in mount airy  vi	GetWeather
+what will the weather be nineteen hours from now neighboring saint kitts and nevis	GetWeather
+will there be hail on 11/12/2036 in singapore	GetWeather
+will it be colder here in 48 and a half weeks	GetWeather
+what s the weather going to be in knobel	GetWeather
+what will the weather be in dane on sep  the fifth  2030	GetWeather
+what will the weather be in ohio	GetWeather
+i need to know the weather for jan  the 3rd in mexico when i go to port vue	GetWeather
+what is the forecast for ōtone prefectural natural park in 1 hour and within the same area	GetWeather
+what kind of weather is forecast around one pm near vatican	GetWeather
+will it be chilly in weldona	GetWeather
+will it be colder in virgin islands national park	GetWeather
+will it be hot at 13:19 in de funiak springs serbia and montenegro	GetWeather
+what is the weather going to be like in virginia on st  patrick s day	GetWeather
+weather in kaneville maryland	GetWeather
+when is sunrise for ar	GetWeather
+what will the weather be not far from here on october the nineteenth  2026	GetWeather
+what is the forecast for waurika in samoa	GetWeather
+tell me the weather forecast here	GetWeather
+what is the weather forecast nearby nicodemus	GetWeather
+what will the weather be in nov  in brookneal	GetWeather
+will it be colder four months from now in suwanee ak	GetWeather
+what is the weather forecast for burundi	GetWeather
+what s the weather in benton city	GetWeather
+what will the weather be in ky on oct  16  2036	GetWeather
+will the sun be out in 1 minute in searcy  uganda	GetWeather
+what is the weather here	GetWeather
+what will the weather be one second from now in chad	GetWeather
+what kind of weather is forecast in ms now	GetWeather
+what is the forecast for la for freezing 	GetWeather
+how cold will it be here in 1 second 	GetWeather
+what is the forecast for hotter weather at southford falls state park	GetWeather
+what is the overcast forecast for the current position starting on jul  19  2030	GetWeather
+what is the forecast for morocco  at lake ozark on december seventeenth  2022	GetWeather
+what will the humidity be in the current spot at 15:19:29	GetWeather
+what is the forecast in nicodemus and nearby	GetWeather
+what is the weather going to be like in benton colorado in 2 and a half months	GetWeather
+what s the weather forecast for bothe-napa valley state park close by february 20	GetWeather
+what is the forecast for beginning on nov  17 for franklinville	GetWeather
+what s the forecast for sep  26 in emerado saint pierre and miquelon	GetWeather
+will there be a blizzard next winter in visalia  idaho	GetWeather
+will it be warmer in the district of columbia on may 25  2033	GetWeather
+what will the weather be here on dec  7th	GetWeather
+what is the forecast for colder temps beginning on law day here	GetWeather
+what s the weather like in tyonek  new jersey	GetWeather
+what is the forecast for here for blizzard conditions at five pm	GetWeather
+will there be a storm in gibsonia at 8 p m	GetWeather
+what is the cold condition of our current position for tomorrow	GetWeather
+what will the weather be in hialeah gardens on october the 24th	GetWeather
+will it be freezing today in delaware and lehigh national heritage corridor	GetWeather
+what is the forecast in admire in tx starting at seventeen 	GetWeather
+what is the forecast in north carolina for edgemoor	GetWeather
+what is the forecast for costa rica	GetWeather
+need weather for parc national tolhuaca to see if it will be fog today	GetWeather
+weather in walden russia on 12/26/2018	GetWeather
+what s the humidity here right now	GetWeather
+how s the weather at petit manan national wildlife refuge and nearby right now	GetWeather
+what is the forecast for lansford for temperate weather	GetWeather
+overcast on state holiday in pawling nature reserve  and neighboring places	GetWeather
+i need the weather in wakarusa	GetWeather
+tell me the forecast for 6 am in tatra-nationalpark	GetWeather
+tell me the weather forecast for ut on thursday	GetWeather
+what is the forecast for turtle islands national park	GetWeather
+will it be hotter in pr at 23 o clock	GetWeather
+weather in two hours  in uzbekistan	GetWeather
+what is the forecast for this afternoon for blizzard conditions in dieterich chad	GetWeather
+how s the weather here at two am	GetWeather
+will custer national forest be chillier at seven pm	GetWeather
+what is the forecast for starting at three a m in two buttes for warm weather	GetWeather
+what s the weather in fox chapel	GetWeather
+what is the rain forecast for one hour from now in south korea	GetWeather
+tell me the weather forecast here	GetWeather
+will there be a cloud in vi in 14 minutes 	GetWeather
+how much colder will it be not far from utah around 3 am	GetWeather
+will it be chilly midday in cresbard afghanistan	GetWeather
+what will the weather be in sarygamyş sanctuary on august 21  2035	GetWeather
+will it be rainy in tenino	GetWeather
+will it be hot in the netherlands on february 16th	GetWeather
+where is belgium located	GetWeather
+what will the weather be in milleville beach	GetWeather
+can you put on like a hurricane by paul landers	PlayMusic
+play the happy blues by ronnie wood 	PlayMusic
+play the newest melody on last fm by eddie vinson 	PlayMusic
+use groove shark to play music	PlayMusic
+please play something good from u-roy   any song from 1975 on zvooq will do 	PlayMusic
+play a symphony from 2013 	PlayMusic
+let me hear the good songs from james iha	PlayMusic
+play my inventive playlist	PlayMusic
+i want to play music from iheart	PlayMusic
+play subconscious lobotomy from jennifer paull 	PlayMusic
+i want to hear a seventies sound track 	PlayMusic
+play a john maher track	PlayMusic
+please play something from dihan slabbert that s on the top fifty	PlayMusic
+please play something catchy on youtube	PlayMusic
+play something from 2004 by imogen heap on spotify	PlayMusic
+play seventies music please 	PlayMusic
+play music from the artist sean yseult and sort it through top-50	PlayMusic
+play anything jd natasha did in the thirties	PlayMusic
+play music off netflix 	PlayMusic
+nineties songs on zvooq	PlayMusic
+open itunes and play ben burnley ready to die	PlayMusic
+play an ep by zak starkey 	PlayMusic
+play an album from nithyasree mahadevan 	PlayMusic
+i want to listen to something on youtube	PlayMusic
+start playing something from iheart	PlayMusic
+play trance life on zvooq 	PlayMusic
+find and play a concerto on zvooq from 1978 by ginger pooley	PlayMusic
+play all things must pass 	PlayMusic
+i want to hear music from allen toussaint from the fifties	PlayMusic
+turn on last fm	PlayMusic
+play a song by rahsaan patterson 	PlayMusic
+play femme fatale by bonobo	PlayMusic
+play some anneliese van der pol from the thirties on groove shark	PlayMusic
+i want to listen to an ep from 1998 	PlayMusic
+play paul mccartney	PlayMusic
+play jill sobule album	PlayMusic
+play chant s from 1973 	PlayMusic
+play something from 90s pop rock essentials	PlayMusic
+play have you met miss jones by nicole from google music 	PlayMusic
+play chant by nigger kojak on itunes	PlayMusic
+play some sixties songs on google music	PlayMusic
+play a fifties album from dj yoda on last fm 	PlayMusic
+please play my ecstatic playlist 	PlayMusic
+open deezer and play curtain call: the hits by junichi okada 	PlayMusic
+let s play jamie robertson s handover on vimeo 	PlayMusic
+play a sixties soundtrack	PlayMusic
+play this is: miles davis on lastfm 	PlayMusic
+live in l a joseph meyer please	PlayMusic
+play the top twenty hisham abbas on youtube	PlayMusic
+play some seventies filipp kirkorow	PlayMusic
+play the most popular puretone	PlayMusic
+play music from e-type 	PlayMusic
+can you play a j  pero on groove shark	PlayMusic
+play a bob burns song	PlayMusic
+i want to hear leroi moore on vimeo   play the song chance of a lifetime 	PlayMusic
+play some symphony music from david lindley 	PlayMusic
+please play something on iheart from artist ari gold last album	PlayMusic
+i want to hear them from the artist murcof	PlayMusic
+play sound track music from the twenties	PlayMusic
+play dance with the devil by mr  lordi	PlayMusic
+play music from 1996 	PlayMusic
+go to itunes and play dr  lecter by david hodges	PlayMusic
+play s t r e e t  d a d  from hiromitsu agatsuma through pandora 	PlayMusic
+play some movement from the fourties	PlayMusic
+please tune into chieko ochi s good music	PlayMusic
+play the greatest music from bryan maclean 	PlayMusic
+play something on last fm	PlayMusic
+play music by joy nilo 	PlayMusic
+play some gary lee conner	PlayMusic
+play music by brian chase 	PlayMusic
+can you play top zvooq by fink	PlayMusic
+play the top-20 nawang khechog soundtrack	PlayMusic
+let s hear stuff from andrew hewitt 	PlayMusic
+play a good ep from the eighties by peter murphy 	PlayMusic
+play another passenger from louis nelson delisle 	PlayMusic
+play the top music from the railway children off last fm 	PlayMusic
+play the best becca	PlayMusic
+play something by duke ellington from the seventies	PlayMusic
+use the last fm service to play a mis niños de 30	PlayMusic
+play my black sabbath: the dio years playlist 	PlayMusic
+play an ep from mike harding 	PlayMusic
+i want to hear anything from the rock symphonique genre please 	PlayMusic
+please play a 1997 record 	PlayMusic
+put what color is your sky by alana davis on the stereo 	PlayMusic
+please play a movement from george formby jr 	PlayMusic
+play some new les vandyke on slacker	PlayMusic
+please open zvooq	PlayMusic
+play progressive metal 	PlayMusic
+i want to hear soundtrack music on youtube from helena iren michaelsen	PlayMusic
+play a song by ramesh narayan from 1960	PlayMusic
+play some blues britânico 	PlayMusic
+proceed with hitomi nabatame music from 2003 	PlayMusic
+play something on zvooq	PlayMusic
+play music from lynn & wade llp 	PlayMusic
+let me hear chris knight music	PlayMusic
+let s hear good mohammad mamle on vimeo 	PlayMusic
+please play a sound track from the fifties that s on iheart	PlayMusic
+play music from van-pires by dmitry malikov 	PlayMusic
+play rich sex on iheart	PlayMusic
+play modern psychedelia 	PlayMusic
+rate this album four out of 6 stars	RateBook
+give this textbook four stars 	RateBook
+rate a twist in the tale zero out of 6 points	RateBook
+rate the children of niobe 1 out of 6 points 	RateBook
+give zero stars to halo: ghosts of onyx	RateBook
+give this novel a score of 5	RateBook
+give the current series four of 6 points 	RateBook
+give 4 out of 6 points to the spirit ring chronicle	RateBook
+give two stars out of 6 to 36 children	RateBook
+rate the sneetches and other stories a three	RateBook
+rate the current series four stars 	RateBook
+rate this book a 4 out of 6	RateBook
+rate the current novel 5 of 6 stars	RateBook
+rate this book a 1 	RateBook
+give zero out of 6 to the current album	RateBook
+give this album 5 points 	RateBook
+rate the mystery of the tolling bell series 4 stars 	RateBook
+give the current novel two stars 	RateBook
+give the current book 4 stars	RateBook
+give joe magarac and his usa citizen papers 5 points	RateBook
+rate the guilty 0 of 6 points	RateBook
+rate this textbook four out of 6	RateBook
+give the catedral series four stars 	RateBook
+reminiscences of the anti-japanese guerillas chronicle deserves zero points out of 6 for a rating 	RateBook
+give small screen  big picture a 0 out of 6 rating 	RateBook
+gods and pawns should get a three	RateBook
+give zero stars to this textbook	RateBook
+rate the current novel a 4 out of 6 stars 	RateBook
+rate the book the atmospheric railway 5 out of 6	RateBook
+rate black boy 4 out of 6	RateBook
+rate the chronicle current 1 star	RateBook
+mark this album a score of 5	RateBook
+rate the current novel zero out of 6	RateBook
+rate the current novel a 2	RateBook
+give the giant devil dingo 4 points 	RateBook
+rate this current novel two out of 6 	RateBook
+give monthly index of medical specialities a two out of 6 rating 	RateBook
+rate this novel 2 out of 6 points	RateBook
+rate the current novel 3 stars	RateBook
+rate the current essay zero out of 6 stars	RateBook
+rate this current album 0 stars	RateBook
+give a brief stop on the road from auschwitz 1 out of 6 stars	RateBook
+rate this album 4 out of 6 stars	RateBook
+rate hate that cat 1 out of 6 stars 	RateBook
+give my current book one of 6 stars	RateBook
+rate current novel one stars 	RateBook
+give five out of 6 points to this album	RateBook
+give a rating of 2 to juneteenth 	RateBook
+rate ruth five out of 6 points 	RateBook
+rate the sea of trolls 1 stars out of 6	RateBook
+give the zenith angle one out of 6 points	RateBook
+give zero stars to rhialto the marvellous	RateBook
+give the current book a zero of 6	RateBook
+rate personal demons 0 out of 6 points	RateBook
+rate the current series a 4	RateBook
+give one of 6 points to who will cry when you die	RateBook
+give zero out of 6 stars to this album	RateBook
+give this novel 2 stars	RateBook
+rate the 8-week cholesterol cure three out of 6 	RateBook
+rate this novel 3 out of 6 points	RateBook
+rate the lives of john lennon five points	RateBook
+give the american scene 2 of 6 stars	RateBook
+rate this textbook a one 	RateBook
+give summer of the swans 1 points 	RateBook
+give the current textbook a rating of five 	RateBook
+give 4 points to the person and the common good	RateBook
+give a four rating to a world apart 	RateBook
+rate this chronicle 0 points	RateBook
+give wilco: learning how to die a rating of four points 	RateBook
+rate this saga two out of 6 	RateBook
+rate the gift: imagination and the erotic life of property five stars	RateBook
+rate neverwhere four out of 6	RateBook
+rate in the company of cheerful ladies a zero out of 6	RateBook
+give one start to the current book	RateBook
+give this chronicle a 2 rating 	RateBook
+rate this essay a 1	RateBook
+out of 6  give rivers of babylon a 1	RateBook
+give 5 of 6 stars to expressive processing	RateBook
+rate the ghost house series a one	RateBook
+rate know ye not agincourt 2 out of 6 stars	RateBook
+i would rate theft: a love story four out of 6 stars 	RateBook
+rate the further adventures of the joker four stars	RateBook
+give 0 rating to in the heart of the country	RateBook
+give 1 out of 6 rating to the current textbook	RateBook
+give the current chronicle five of 6 points	RateBook
+rate cotton comes to harlem a 2	RateBook
+give this album one stars	RateBook
+rate the adventures of augie march one points	RateBook
+rate soul music a 0 	RateBook
+give hindu temples: what happened to them a 5 out of 6 stars 	RateBook
+give this novel a 1 	RateBook
+rate the current textbook 1 out of 6	RateBook
+give this textbook 0 out of 6 stars	RateBook
+give the crystal snare 5 stars	RateBook
+rate this saga two out of 6 	RateBook
+give wilco: learning how to die a rating of four points 	RateBook
+rate this book 3 stars out of 6	RateBook
+rate the three junes one out of 6	RateBook
+give four stars to the broken window	RateBook
+rate the current series 4 points	RateBook
+wish to find the movie the heart beat	SearchCreativeWork
+please look up the tv show  vanity 	SearchCreativeWork
+get me the elvis  christmas album tv show 	SearchCreativeWork
+please find me the saga  the deep six 	SearchCreativeWork
+wish to see the photograph with the name live: right here	SearchCreativeWork
+looking for a novel called death march	SearchCreativeWork
+can you find me the work  the curse of oak island	SearchCreativeWork
+please get me the sacred and profane love machine game 	SearchCreativeWork
+need a creative work called hit by love	SearchCreativeWork
+search for the trailer for the office	SearchCreativeWork
+looking for a creative work called plant ecology	SearchCreativeWork
+find the television show to me	SearchCreativeWork
+can you please find me the saga chump change	SearchCreativeWork
+can you find me the ridiculous 6 book	SearchCreativeWork
+please fine me the tv series  now we are married 	SearchCreativeWork
+please look up the work  bachelor pad 	SearchCreativeWork
+please help me find the late night heartbroken blues television show 	SearchCreativeWork
+please help me find  bend it like beckham the musical 	SearchCreativeWork
+please look up the tv series parables for wooden ears 	SearchCreativeWork
+can you find me  hey man	SearchCreativeWork
+please search for switched 	SearchCreativeWork
+can you get me the controlled conversations tv series	SearchCreativeWork
+please look up the song the mad magician 	SearchCreativeWork
+please search for the tv show  the best of white lion 	SearchCreativeWork
+please find me phineas redux 	SearchCreativeWork
+get me the procession of ants tv show 	SearchCreativeWork
+looking for a game called phinally phamous	SearchCreativeWork
+can you search the daring youth saga	SearchCreativeWork
+look for the book the girl who was plugged in 	SearchCreativeWork
+find me a tv show called baby blue	SearchCreativeWork
+search for appalachian journey	SearchCreativeWork
+look for the television show meet the prince	SearchCreativeWork
+can you find me cracks the safe	SearchCreativeWork
+please help me search the hell money saga 	SearchCreativeWork
+get me the secret south song 	SearchCreativeWork
+can you find me the work titled music for millions	SearchCreativeWork
+please search for the painting titled this is the night 	SearchCreativeWork
+could you locate the epic conditions picture	SearchCreativeWork
+get me the trailer of good morning sunshine	SearchCreativeWork
+please search the an introduction to karl marx painting 	SearchCreativeWork
+can you find me the blue spring trailer	SearchCreativeWork
+could you find the tv series the approach	SearchCreativeWork
+search for the tv show  a lawless street 	SearchCreativeWork
+please look up three essays on the theory of sexuality show 	SearchCreativeWork
+please get me the compulsive disclosure song 	SearchCreativeWork
+can you look up the molecular oncology saga	SearchCreativeWork
+search for the sound of one hand clapping	SearchCreativeWork
+find the creative work deadly weapons	SearchCreativeWork
+need the creative work called the logic of scientific discovery	SearchCreativeWork
+can you find me the national anthem of the ancient britons television show	SearchCreativeWork
+can you please find me the harry hood saga	SearchCreativeWork
+can you find me the work  bible translations into hawaii pidgin	SearchCreativeWork
+please look up and find me monty python live at the hollywood bowl 	SearchCreativeWork
+please search for mary 	SearchCreativeWork
+please search the game atla: all this life allows 	SearchCreativeWork
+find me the novel with the name to lose my life …	SearchCreativeWork
+looking for a song with the title of live at the kings center	SearchCreativeWork
+can you find the american bison photograph	SearchCreativeWork
+can you find me the free for all show	SearchCreativeWork
+please find me the olympia 74 soundtrack 	SearchCreativeWork
+look for the album slave to the grind	SearchCreativeWork
+please find me the projekt: the new face of goth 	SearchCreativeWork
+can you get me the message from god saga	SearchCreativeWork
+find me the soundtrack a honeymoon adventure	SearchCreativeWork
+please get me the henderson kids saga 	SearchCreativeWork
+find the movie splendor in the grass	SearchCreativeWork
+am looking for a book with the title free to play	SearchCreativeWork
+look for the tv series jersey boys	SearchCreativeWork
+can you search the book  paris - when it sizzles	SearchCreativeWork
+looking for a painting with the title with you	SearchCreativeWork
+please find me the classified book 	SearchCreativeWork
+look for the show v-the new mythology suite	SearchCreativeWork
+find the creative work face down	SearchCreativeWork
+find four songs	SearchCreativeWork
+find me the soundtrack live at the greek theatre	SearchCreativeWork
+please search for the television show  episodi di the blacklist 	SearchCreativeWork
+find a creative work called fire in the hole	SearchCreativeWork
+looking for the picture with the name of who made stevie crye	SearchCreativeWork
+look for the album wolves within	SearchCreativeWork
+find the album orphan girl at the cemetery	SearchCreativeWork
+please find me the journal of the british astronomical association movie 	SearchCreativeWork
+find the tv show the daydreamer	SearchCreativeWork
+can you please get me the book dracula 5: the blood legacy	SearchCreativeWork
+please look up the novel  live to dance 	SearchCreativeWork
+please find me the video game titled 20 hours in america 	SearchCreativeWork
+find the creative work the devil in stitches	SearchCreativeWork
+please look up the work  prophets 	SearchCreativeWork
+i m looking for welcome to the canteen	SearchCreativeWork
+please search for the journal of official statistics show 	SearchCreativeWork
+please look up show-biz blues photograph 	SearchCreativeWork
+please search the woodsmen of the west 	SearchCreativeWork
+can you find the creative works associated with caryl & marilyn: real friends	SearchCreativeWork
+please get me the dead soul saga 	SearchCreativeWork
+please search the live from leeds album 	SearchCreativeWork
+please look up the johnny english - la rinascita painting 	SearchCreativeWork
+can you find me the sword with no name trailer	SearchCreativeWork
+i wish to watch the fold trailer  please search 	SearchCreativeWork
+can you find me the almost human painting	SearchCreativeWork
+please find me the work  serious awesomeness 	SearchCreativeWork
+search for the game difficult loves	SearchCreativeWork
+is babar: king of the elephants playing 	SearchScreeningEvent
+is the ghost playing	SearchScreeningEvent
+is bartok the magnificent playing at seven am	SearchScreeningEvent
+what s the movie schedule	SearchScreeningEvent
+i want to see jla adventures: trapped in time	SearchScreeningEvent
+when is the fox and the child playing in this cinema	SearchScreeningEvent
+show me the schedule for rat rod rockers	SearchScreeningEvent
+is any which way you can playing in 15 seconds	SearchScreeningEvent
+i want to see the portrait of a lady at the nearest cinema 	SearchScreeningEvent
+where can i see the prime ministers: the pioneers	SearchScreeningEvent
+i need to find the movie theatre showing the crooked web closest to me	SearchScreeningEvent
+i want to see while the sun shines at the closest movie house 	SearchScreeningEvent
+i want to see those kids from town  when will it be showing	SearchScreeningEvent
+find the schedule for the comedian at santikos theatres 	SearchScreeningEvent
+what are the movie schedules for my favorite theaters 	SearchScreeningEvent
+what are the movies showing in the neighbourhood 	SearchScreeningEvent
+is without witness playing twenty two hours from now	SearchScreeningEvent
+i need animated movies in the area for dinner time	SearchScreeningEvent
+i want to see i dream of jeanie in a movie theatre	SearchScreeningEvent
+can i see ellis island revisited in 1 minute	SearchScreeningEvent
+i want animated movies at mjr theatres	SearchScreeningEvent
+show me the schedule for the oblong box	SearchScreeningEvent
+i want to know if there are any movies playing in the area 	SearchScreeningEvent
+is what a wonderful place showing at cinemark theatres	SearchScreeningEvent
+show the closest movie theatre that shows boycott	SearchScreeningEvent
+i want to see doa: dead or alive at loews cineplex entertainment 	SearchScreeningEvent
+is the nightmare showing six hours from now at the nearest cinema 	SearchScreeningEvent
+what is the nearest movie house with window connection playing at lunch	SearchScreeningEvent
+is patrick still lives showing at amc theaters	SearchScreeningEvent
+fine the movie schedules for the wanda group 	SearchScreeningEvent
+give me the movie schedule nearby	SearchScreeningEvent
+find the schedule at the douglas theatre company 	SearchScreeningEvent
+show me the movies at harkins theatres 	SearchScreeningEvent
+what movies at star theatres 	SearchScreeningEvent
+i want a movie schedule 	SearchScreeningEvent
+can i get the movie times	SearchScreeningEvent
+i want to see medal for the general	SearchScreeningEvent
+can i get the times for movies in the neighbourhood	SearchScreeningEvent
+may i have the movie schedules for speakeasy theaters	SearchScreeningEvent
+find animated movies close by	SearchScreeningEvent
+is american primitive showing in santikos theatres	SearchScreeningEvent
+what are the movie schedules in the neighborhood	SearchScreeningEvent
+check the schedule for bow tie cinemas 	SearchScreeningEvent
+check the timings for snowbound at the closest movie theatre 	SearchScreeningEvent
+what are the movie times at caribbean cinemas	SearchScreeningEvent
+i need films in the neighborhood	SearchScreeningEvent
+show the movie schedules in the neighborhood	SearchScreeningEvent
+where s the nearest movie house showing foreign films	SearchScreeningEvent
+what movies are showing now at the closest cinema	SearchScreeningEvent
+is rumor has it playing	SearchScreeningEvent
+i need a list of speakeasy theaters movie times	SearchScreeningEvent
+when is the outer space connection playing at the nearest cinema 	SearchScreeningEvent
+find the movie times at harkins theatres 	SearchScreeningEvent
+find the films at century theatres 	SearchScreeningEvent
+show the animated movies playing in the neighbourhood	SearchScreeningEvent
+i want to see fear chamber 	SearchScreeningEvent
+show me southern theatres movie times	SearchScreeningEvent
+is the unnaturals showing at 13 	SearchScreeningEvent
+is no time to be young showing at amc theaters	SearchScreeningEvent
+find the movie schedules for regal entertainment group 	SearchScreeningEvent
+i want to see shattered image 	SearchScreeningEvent
+find the schedule at star theatres 	SearchScreeningEvent
+will i think i do be playing at 7 pm	SearchScreeningEvent
+show me the schedule for arclight hollywood for only animated movies	SearchScreeningEvent
+find the schedule for great mail robbery 	SearchScreeningEvent
+give me the movies in the neighborhood 	SearchScreeningEvent
+what movies are playing close by	SearchScreeningEvent
+is the two gladiators playing	SearchScreeningEvent
+what s the movie schedule for great escape theatres	SearchScreeningEvent
+find the movie schedule close by	SearchScreeningEvent
+i want to see outcast 	SearchScreeningEvent
+show me the schedule of movie the great gildersleeve near movie house	SearchScreeningEvent
+i need times for a yiddish world remembered at dipson theatres	SearchScreeningEvent
+find the movie schedules at goodrich quality theaters 	SearchScreeningEvent
+show me the movie schedule in the neighbourhood 	SearchScreeningEvent
+show me the movie times for films  nearby	SearchScreeningEvent
+show the movie times for animated movies in the neighbourhood	SearchScreeningEvent
+is the eye – infinity playing at general cinema corporation	SearchScreeningEvent
+can you check the timings for super sweet 16: the movie	SearchScreeningEvent
+is we are northern lights playing in any movie theatre	SearchScreeningEvent
+what times will the young swordsman be showing at my cinema	SearchScreeningEvent
+show the sexy dance 2 times at the  closest movie house	SearchScreeningEvent
+what are some close by animated movies showing	SearchScreeningEvent
+movie schedules close by for animated movies	SearchScreeningEvent
+what films are playing close by	SearchScreeningEvent
+find the movie schedule in the area 	SearchScreeningEvent
+is cowboy canteen playing	SearchScreeningEvent
+is rare birds showing at the nearest movie theatre at noon 	SearchScreeningEvent
+what are the movie times	SearchScreeningEvent
+where can i find the movie schedules	SearchScreeningEvent
+find the movie schedule for north american cinemas in eleven seconds 	SearchScreeningEvent
+find the nearest cinema with movies playing	SearchScreeningEvent
+what are the movie times	SearchScreeningEvent
+what are the times for the gingerbread man	SearchScreeningEvent
+what films are playing close by	SearchScreeningEvent
+is any cinema playing the spirit of youth	SearchScreeningEvent
+what are the movie times for animated movies in the neighbourhood	SearchScreeningEvent
+what s the movie schedule at great escape theatres	SearchScreeningEvent
+show the times for cheers for miss bishop at dipson theatres 	SearchScreeningEvent
+i want to see married to the enemy 2 at a cinema 	SearchScreeningEvent

TEXTOIR-main/data/snips/test.tsv

@@ -0,0 +1,701 @@
+text	label
+add sabrina salerno to the grime instrumentals playlist	AddToPlaylist
+i want to bring four people to a place that s close to downtown that serves churrascaria cuisine  	BookRestaurant
+put lindsey cardinale into my hillary clinton s women s history month playlist  	AddToPlaylist
+will it snow in mt on june 13  2038	GetWeather
+play signe anderson chant music that is newest	PlayMusic
+can you let me know what animated movies are playing close by	SearchScreeningEvent
+can you get me reservations for a highly rated restaurant in seychelles	BookRestaurant
+what s the weather here on 2/7/2021	GetWeather
+find worldly goods starting now at a movie house	SearchScreeningEvent
+on june 27  2026 i d like to go to a delaware gastropub	BookRestaurant
+what movies are playing at mann theatres	SearchScreeningEvent
+find a movie called living in america 	SearchCreativeWork
+find on dress parade	SearchScreeningEvent
+make a reservation at a bakery that has acquacotta in central african republic for five 	BookRestaurant
+where can i purchase the tv show time for heroes	SearchCreativeWork
+will the wind die down at my current location by supper time	GetWeather
+please search the young warriors game 	SearchCreativeWork
+make me a reservation in south carolina	BookRestaurant
+what movie theatre is showing if the huns came to melbourne	SearchScreeningEvent
+restaurant in bulgaria this week party for 9 numbers	BookRestaurant
+rate the current novel four of 6 stars	RateBook
+add the song don t drink the water to my playlist	AddToPlaylist
+add this tune by rod argent to propuesta alternativa playlist	AddToPlaylist
+show the movie times	SearchScreeningEvent
+will it snow in amy	GetWeather
+what will the weather be at nine am in hi	GetWeather
+in one hour find king of hearts	SearchScreeningEvent
+book a spot for ten at a top-rated  caucasian restaurant not far  from selmer	BookRestaurant
+play music from clark kent in the year 1987	PlayMusic
+add to the rock games	AddToPlaylist
+add this artist to pop 2017 picks	AddToPlaylist
+i rate shadow of suribachi at five stars 	RateBook
+play some sixties music 	PlayMusic
+what film is playing nearby	SearchScreeningEvent
+add nothing fancy to meditate to sounds of nature playlist	AddToPlaylist
+get the video game of the chipmunk song	SearchCreativeWork
+rate lamy of santa fe 5 of 6 stars	RateBook
+show me movie schedules	SearchScreeningEvent
+what will the weather be in lago vista on october fourteenth  2022	GetWeather
+weather next year in canada	GetWeather
+play a new symphony by perfecto de castro on lastfm	PlayMusic
+rate cuisines of the axis of evil and other irritating states one out of 6	RateBook
+play arif music from the fourties 	PlayMusic
+what is the weather of east portal ks	GetWeather
+play a melody from elmer bernstein	PlayMusic
+what is the weather going to be like in klondike gold rush national historical park on february the 28th  2034	GetWeather
+play songs by sarah harding	PlayMusic
+rate the chronicle ten from tomorrow a 2	RateBook
+book a table for 2 at a restaurant in follett	BookRestaurant
+book a brasserie in samoa for four people 	BookRestaurant
+play the new noise theology e p 	SearchCreativeWork
+find a reservation at a restaurant that serves gougère in laneville with a party of nine	BookRestaurant
+find the cold dead hand video game for me	SearchCreativeWork
+book a bakery for lebanese on january 11th  2032	BookRestaurant
+rate the book an appeal from the new to the old whigs a 0	RateBook
+book a table for 8 at a restaurant that serves far breton 	BookRestaurant
+rate this current novel 1 stars	RateBook
+i rate secret water as a 4 	RateBook
+is unbeatable harold at century theatres	SearchScreeningEvent
+please find me asking alexandria discography	SearchCreativeWork
+what will the weather be in berville ak on feb  6  2017	GetWeather
+is it warm in botna	GetWeather
+please add a track to my playlist called this is coti	AddToPlaylist
+find the via dolorosa: songs of redemption saga	SearchCreativeWork
+can you add confessions to my playlist called clásica	AddToPlaylist
+find the schedule for nearby animated movies 	SearchScreeningEvent
+book a table today at a steakhouse for eight that serves sashimi	BookRestaurant
+play the last sound track by soko from around 1975	PlayMusic
+add this song to blues roots	AddToPlaylist
+coon chicken inn restaurant for 1 am for me  clarice and debbie	BookRestaurant
+add karusellen to jazz brasileiro	AddToPlaylist
+play some steve boyett chant music	PlayMusic
+give 1 out of 6 points to this novel	RateBook
+show the movie schedule of animated movies close by	SearchScreeningEvent
+please play the newest music by evil jared hasselhoff	PlayMusic
+add tune to my mellow bars playlist	AddToPlaylist
+put coming back to life onto winter music	AddToPlaylist
+rate this textbook a zero	RateBook
+i want to hear any tune from the twenties	PlayMusic
+play me a top-ten song by phil ochs on groove shark	PlayMusic
+find a video game called family dog 	SearchCreativeWork
+rate awaiting strange gods: weird and lovecraftian fictions a 1	RateBook
+add lisa m to my guitar hero live playlist	AddToPlaylist
+what is the weather forecast for my current place	GetWeather
+add strong to the metal monday playlist	AddToPlaylist
+where can i find conduct unbecoming	SearchCreativeWork
+will it be freezing in the current position	GetWeather
+add the da brat track to the soak up the sun playlist 	AddToPlaylist
+add a track to the another glass playlist 	AddToPlaylist
+find now and forever	SearchScreeningEvent
+the workout playlist needs more chris cross	AddToPlaylist
+play some jungle music on iheart	PlayMusic
+give 1 point to current textbook	RateBook
+put no mystery into my punk essentials playlist  	AddToPlaylist
+i want to put look to you on the playlist named 80s classic hits	AddToPlaylist
+what time is beat the devil coming on at mann theatres	SearchScreeningEvent
+rate the current chronicle a zero	RateBook
+add garry shider album to my classical essentials	AddToPlaylist
+add the artist cho kyu hyun to funky jams	AddToPlaylist
+find the work i looked up	SearchCreativeWork
+play this is colour by panda bear	PlayMusic
+play the god that failed on vimeo 	PlayMusic
+can i get the butterfly crush showings	SearchScreeningEvent
+add hanging on to my just dance by aftercluv playlist 	AddToPlaylist
+show me when scandalous john is playing 	SearchScreeningEvent
+a day no pigs would die deserves a best rating of 6 and a value of 4	RateBook
+for my crossfit playlist add the soul sessions volume 2	AddToPlaylist
+play some james cleveland 	PlayMusic
+put this tune on dancepop	AddToPlaylist
+what time will paris by night aired	SearchScreeningEvent
+play music on spotify 	PlayMusic
+i want a matt garrison tune in my fresh finds fire emoji playlist  	AddToPlaylist
+will there be snowfall at six pm in leisure knoll california	GetWeather
+search for the television show me and my guitar	SearchCreativeWork
+tell me when it will be chilly in chicken  united kingdom	GetWeather
+is it windy in telogia	GetWeather
+find a tv show called revenge of the nerds 	SearchCreativeWork
+find the video game called turnin me on	SearchCreativeWork
+play the song i get ideas as performed by richard kruspe	PlayMusic
+add turk to the deep house playlist	AddToPlaylist
+find a reservation at fish express	BookRestaurant
+check the forecast for the current spot in the future oct  19  2037 	GetWeather
+how can i view the show corpus: a home movie about selena	SearchCreativeWork
+i would rate that old ace in the hole one stars and a best rating of 6	RateBook
+add the rating for this current series a four out of 6 points	RateBook
+add justin mcroberts to this is chopin	AddToPlaylist
+book a bar that serves italian-american cuisine neighboring wilson av for one person 	BookRestaurant
+is fog forecast close-by to pakistan	GetWeather
+book a restaurant for 3 people at eighteen oclock in saint vincent and the grenadines 	BookRestaurant
+find the schedule for films at night at great escape theatres 	SearchScreeningEvent
+is there snow in the district of columbia	GetWeather
+find a movie schedule	SearchScreeningEvent
+rate the beggar of volubilis 1 out of 6	RateBook
+what is the forecast in heber	GetWeather
+please play an album from 1987	PlayMusic
+show me the courts of chaos	SearchCreativeWork
+give the current book five stars out of 6 	RateBook
+when is fine  totally fine playing	SearchScreeningEvent
+add a tune to clásicos del hip hop español	AddToPlaylist
+play jawad ahmad	PlayMusic
+what is the forecast for in 1 second at monte sereno for freezing temps	GetWeather
+i would like to eat fast food and have a party of two in kentucky 	BookRestaurant
+play music from itunes for ric grech	PlayMusic
+add jennie  jennie to my metal playlist	AddToPlaylist
+show the tv show the last samurai 	SearchCreativeWork
+add rob tyner to betsy s we everywhere	AddToPlaylist
+show me the weather forecast for the city of spencer	GetWeather
+how is the weather in getzville  minnesota	GetWeather
+what is dear old girl cooper foundation	SearchScreeningEvent
+i need a weather forecast for são tomé and príncipe on december 8th  2026	GetWeather
+what animated movies are showing in the area 	SearchScreeningEvent
+tell me the weather forecast for april 15  2019 here	GetWeather
+play the track asleep in the deep	PlayMusic
+play kurt cobain ballad tunes	PlayMusic
+can you add a track to my spain top 50 playlist	AddToPlaylist
+at meal time while i m here will it be hot	GetWeather
+can you find me the magic hour song	SearchCreativeWork
+add mary wells sings my guy to the electro sur playlist 	AddToPlaylist
+play some kyle ward from the seventies	PlayMusic
+book a table around london borough of ealing that is highly rated in a gluten free bar	BookRestaurant
+when is crime and punishment u s a  showing	SearchScreeningEvent
+will it snowstorm in long lake national wildlife refuge	GetWeather
+rate current essay a zero	RateBook
+book me a reservation at a bar around juliff for three people that serves bucatini for now	BookRestaurant
+book a highly rated place in in in seven years  at a pub	BookRestaurant
+what time is southern theatres showing ukraine is not a brothel	SearchScreeningEvent
+add this album ny bill callahan to my mi casa es la tuya playlist oficial list 	AddToPlaylist
+find a soundtrack called pax warrior	SearchCreativeWork
+book a table for ten for breakfast in minnesota 	BookRestaurant
+what is the local movie schedule	SearchScreeningEvent
+book a restaurant for three on feb  18	BookRestaurant
+i d like to know what movies are on the movie schedules nearby 	SearchScreeningEvent
+please make me reservations somewhere for eight people in foley  nv	BookRestaurant
+she me movie times at mann theatres	SearchScreeningEvent
+find the picture ultima vi: the false prophet	SearchCreativeWork
+play the best album from the seventies	PlayMusic
+add kylie minogue to my novedades viernes sudamérica playlist  	AddToPlaylist
+is it freezing in colorado	GetWeather
+the last hawk gets a total of 3 out of 6 stars from me	RateBook
+will it be stormy in ma	GetWeather
+play pop 2017 picks	PlayMusic
+play some theme songs from 1974	PlayMusic
+what will the weather be in la at 9 o clock	GetWeather
+can you add xanadu to latin alternative music	AddToPlaylist
+can you find me the naked city – justice with a bullet album	SearchCreativeWork
+please search the work  eve-olution 	SearchCreativeWork
+add i dreamt of a dragon to my futuros hits playlist  	AddToPlaylist
+add this artist to the laugh list	AddToPlaylist
+i d like to eat at a restaurant around china with a party of 7 anywhere that serves ouzeri	BookRestaurant
+the sleep machine waterscapes playlist needs some kris chetan ramlu in it  	AddToPlaylist
+rate the current chronicle five stars	RateBook
+rate this novel five of 6	RateBook
+my rating for the eiffel tower and other mythologies is 0 out of 6 stars	RateBook
+i d like a table for midday at the unseen bean	BookRestaurant
+where can i see the movie across the line: the exodus of charlie wright	SearchCreativeWork
+turn on spotify to tiny tim ep	PlayMusic
+what are the movie schedules	SearchScreeningEvent
+i want a table for me and my kids in turkey at a neighboring restaurant	BookRestaurant
+play a top 5 song from wally bastian on google music	PlayMusic
+please search the ironbound picture 	SearchCreativeWork
+put a gary clark song into the soul bpm playlist  	AddToPlaylist
+will it be hot on orthodox good friday in michigan and close-by	GetWeather
+i want to see the television show called cuts both ways	SearchCreativeWork
+i d like to reserve a table at a pub that serves andouillettes within the same area in san marino 	BookRestaurant
+what is the weather like in hurstville	GetWeather
+put this album on my wild country playlist	AddToPlaylist
+rate this textbook 2 out of 6	RateBook
+search for the complots	SearchCreativeWork
+find the schedule for the band of honest men at the nearest movie theatre 	SearchScreeningEvent
+what will the weather be in waverly city brazil on purple heart day	GetWeather
+what is the weather forecast in delaware	GetWeather
+play a top-50 tune from 1982 	PlayMusic
+play shinji miyazaki s music on netflix	PlayMusic
+can i get the game list of mew singles	SearchCreativeWork
+what s the forecast for belize around meal time	GetWeather
+add gary lachman track to jazz for loving couples playlist	AddToPlaylist
+find the path to power 	SearchCreativeWork
+put artist paulinho da costa on my very nearly nashville playlist	AddToPlaylist
+i am looking for the work: nikki	SearchCreativeWork
+what s the weather in low moor	GetWeather
+play some nineties music	PlayMusic
+find a television show called swing high 	SearchCreativeWork
+use netflix to play bizzy bone kiss me goodnight  sergeant major	PlayMusic
+i d like to see movie schedules for kerasotes theatres	SearchScreeningEvent
+i want these are the days added to my spotlight spain 2016 playlist  	AddToPlaylist
+play the greatest soundtrack by nhat son on last fm 	PlayMusic
+what is the tv series in app store	SearchCreativeWork
+book the space aliens grill & bar in hord wy for feb  the twenty-seventh 	BookRestaurant
+find a saga called set sail the prairie	SearchCreativeWork
+can jovino santos neto s album get added to my confidence boost playlist	AddToPlaylist
+show animated movies in nearest movie theatre	SearchScreeningEvent
+find the game company of heroes	SearchCreativeWork
+where can i find paranormal activity 3 playing near me 1 hour from now	SearchScreeningEvent
+book a table this evening in saint vincent and the grenadines at a gastropub	BookRestaurant
+can i listen to dj vibe s top 10	PlayMusic
+what films are at the nearest cinema	SearchScreeningEvent
+what is the weather like in north salt lake and afghanistan	GetWeather
+can you tell me the actors of the saga awards/	SearchCreativeWork
+go to my all out 00s and add brian wilson	AddToPlaylist
+food truck in panama for five	BookRestaurant
+look up the movie schedule	SearchScreeningEvent
+book a table for chasity ruiz and mary at the fat duck in puerto rico	BookRestaurant
+find the gill deacon show 	SearchCreativeWork
+find the movie schedule for films in the area 	SearchScreeningEvent
+will i be able to watch camping-car at movie house at 6 pm	SearchScreeningEvent
+play how does it work by helen carter 	PlayMusic
+what s the weather like in schenectady ma	GetWeather
+play some folk-rock music	PlayMusic
+give this current book zero out of 6	RateBook
+rate this album 5 points	RateBook
+how is the weather right now at my current place	GetWeather
+play sixties music by giovanni battista guadagnini 	PlayMusic
+tell me the weather forecast close by brown county state park for meal time	GetWeather
+play the last wellman braud album relaesd 	PlayMusic
+play sugar baby by frank beard	PlayMusic
+find the schedule for the solitude of prime numbers at the nearest cinema in 1 hour  	SearchScreeningEvent
+play the discografia de the pretty reckless saga	SearchCreativeWork
+i want to give the current textbook 0 out of 6 stars	RateBook
+show me movie times for animated movies playing three hours from now in the neighbourhood 	SearchScreeningEvent
+find the game just dance greatest hits	SearchCreativeWork
+add this track to the sin ti playlist	AddToPlaylist
+show me the closest movie house playing an unfinished life at eight pm 	SearchScreeningEvent
+what s it like in bahrain right now	GetWeather
+can you add blood on the face to the playlist called heartland country	AddToPlaylist
+on jan  the twentieth what will it feel like in ct or the area not far from it	GetWeather
+i need a table in uruguay in 213 days when it s chillier	GetWeather
+add this track by horace andy to acoustic soul	AddToPlaylist
+plan an album by roni duani 	PlayMusic
+add song to siesta	AddToPlaylist
+can you tell me the weather forecast for samoa	GetWeather
+play music on youtube	PlayMusic
+add spirit touches ground to my leche con chocolate list	AddToPlaylist
+i need a table for 1 minute from now at any pub for five around in that also serves fisn n chips 	BookRestaurant
+book a spot at the food truck in ma	BookRestaurant
+21 weeks from now elinor  crystal turner and nita want to eat german food at a bar in distant california	BookRestaurant
+find a tv show called ruthless	SearchCreativeWork
+find animated movies close by with a movie schedule	SearchScreeningEvent
+book a spot for 7 at an outdoor food court in denmark	BookRestaurant
+i would rate the persistence of vision 1 stars and a best rating of 6	RateBook
+i need a reservation for february 27  2020 at a bar that serves paté	BookRestaurant
+find the ghost of tom joad	SearchCreativeWork
+i need a reservation for ten at a tavern in west virginia	BookRestaurant
+what time is children of divorce playing	SearchScreeningEvent
+will there be a blizzard in white house curacao	GetWeather
+play the top melody from artist maakii	PlayMusic
+are any animated movies playing at magic johnson theatres	SearchScreeningEvent
+give the current album a five 	RateBook
+i want to add digital line to my playlist called infantil	AddToPlaylist
+the current essay gets four points	RateBook
+what will the weather be in grand coteau ut at six pm	GetWeather
+can you find me a trailer for phineas redux	SearchCreativeWork
+add the singer ivan roudyk to my fairy tales playlists 	AddToPlaylist
+add song in my playlist dance workout	AddToPlaylist
+what movies can i see in the area	SearchScreeningEvent
+tell me what films are playing at plitt theatres	SearchScreeningEvent
+add in the heart of the world to the epic gaming playlist	AddToPlaylist
+find movie times	SearchScreeningEvent
+rate the book english grammar in use a five	RateBook
+play tujiko noriko s ten years and running 	PlayMusic
+add the song to the soundscapes for gaming playlist 	AddToPlaylist
+can you put a song by jessica mauboy on my playlist entitled a sudden rainstorm	AddToPlaylist
+show movie schedule	SearchScreeningEvent
+show me movie schedules for today 	SearchScreeningEvent
+add cecil womack to my 50 great female voices playlist 	AddToPlaylist
+will it be freezing here in 9 seconds	GetWeather
+forecast for serbia	GetWeather
+i want to give a mortal flower a two	RateBook
+where can i view the picture reaching horizons	SearchCreativeWork
+in hawaii will it be warmer at 3 am	GetWeather
+rate the little book four stars	RateBook
+rate the current textbook one of 6 stars	RateBook
+i want a table for five at a restaurant with latin food in arkansas for 1 hour from now	BookRestaurant
+find love will tear us apart  a photograph	SearchCreativeWork
+please play me a popular track from 1984 	PlayMusic
+book a mediterranean restaurant for my sister and i	BookRestaurant
+how will the weather be different 5 years from now in waconia	GetWeather
+search for teenage mutant hero turtles: fall of the foot clan photograph	SearchCreativeWork
+play party anthems	PlayMusic
+what is the niceville forecast in fm	GetWeather
+find heat wave	SearchScreeningEvent
+which is the nearest movie house playing the diary of anne frank	SearchScreeningEvent
+can i have the movie schedule for imax corporation	SearchScreeningEvent
+book me a reservation for eight for the top-rated bakery eleven hours from now in mango	BookRestaurant
+play yung joc on slacker	PlayMusic
+show 50 words for snow creative picture	SearchCreativeWork
+play the electrochemical and solid state letters song	SearchCreativeWork
+table for 8 at a popular food court	BookRestaurant
+find me a table for 8 people at a nearby al restaurant one minute from now 	BookRestaurant
+is there rain now in maine	GetWeather
+show me the photograph johnny cash: the complete columbia album collection	SearchCreativeWork
+find movie schedules 	SearchScreeningEvent
+find movie schedules for  united paramount theatres 	SearchScreeningEvent
+what is the forecast for montana at dinner	GetWeather
+please add this track to my de camino playlist	AddToPlaylist
+book me a restaurant please 	BookRestaurant
+find drumline: a new beat  a picture	SearchCreativeWork
+play the red room sessions from chris cunningham	PlayMusic
+play the great adventures of slick rick game	SearchCreativeWork
+list movie schedules for movies playing close by	SearchScreeningEvent
+i am looking for the tv show called the flight of the lost balloon	SearchCreativeWork
+add david axelrod to my futuros hits list	AddToPlaylist
+play me sun ra songs from the fifties	PlayMusic
+add this track to my dinnertime acoustics playist	AddToPlaylist
+add tune to atmospheric black metal playlist	AddToPlaylist
+need to see mother joan of the angels in one second 	SearchScreeningEvent
+give 2 out of 6 points to the following textbook 	RateBook
+i would like to book a restaurant for two in 42 weeks from now in wagram	BookRestaurant
+play some last fm music like the 1992 ep from peaches	PlayMusic
+where is the closest cinema playing a drink in the passage	SearchScreeningEvent
+i m hoping you can find a photograph from live at the isle of wight 1970 	SearchCreativeWork
+what movies are around here	SearchScreeningEvent
+book a restaurant distant from downtown 	BookRestaurant
+find doggy day school  an album	SearchCreativeWork
+please play bitch please ii	PlayMusic
+find a video game called young 	SearchCreativeWork
+is strauss is playing today at the cineplex odeon corporation	SearchScreeningEvent
+award this current novel 0 points	RateBook
+weather for this winter here	GetWeather
+what animated movies are playing at the closest movie theatre	SearchScreeningEvent
+rate this book four of 6 points	RateBook
+i want to go see the trouble with girls	SearchScreeningEvent
+cock-a-doodle-doo was awful  i m giving it a 0 out of 6 	RateBook
+show me the schedule of films in the neighbourhood	SearchScreeningEvent
+book a table for nine people in svalbard and jan mayen 	BookRestaurant
+i would give french poets and novelists a best rating of 6 and a value of three	RateBook
+what animated movies are playing nearby	SearchScreeningEvent
+will there be a cloud here at 06:50:20	GetWeather
+i want to give the chronicle zombie bums from uranus 3 points	RateBook
+i d like to know when i can see the taking of flight 847: the uli derickson story at amco entertainment 	SearchScreeningEvent
+play is this my world by leo arnaud	PlayMusic
+book a reservation for clinton street baking company & restaurant distant from downtown	BookRestaurant
+add nyoil to my this is prince playlist	AddToPlaylist
+show me the everybody wants you picture	SearchCreativeWork
+find a restaurant in fm that servec quiche 	BookRestaurant
+i would give this current novel 2 stars with a best rating of 6	RateBook
+i want to book a pastelaria cafe in alabama for me and my great grandfather 	BookRestaurant
+is hail in the weather forecast for monterey bay national marine sanctuary	GetWeather
+add tune to sxsw fresh playlist	AddToPlaylist
+make a reservation in a popular sicilian bar place nearby  for me only  tomorrow	BookRestaurant
+i need a table for 9	BookRestaurant
+add this artist to my post-grunge playlist	AddToPlaylist
+rate this album a 2	RateBook
+what will the weather be like this tuesday in the area neighboring rendezvous mountain educational state forest	GetWeather
+i need a table in ottoville on feb  15th  2029 at gus stevens seafood restaurant & buccaneer lounge	BookRestaurant
+i need a table for five at childs restaurants in brunei 	BookRestaurant
+how do i get the game still on it	SearchCreativeWork
+i would like to make a reservation for 2 for brunch 	BookRestaurant
+need a table  for party of five for december 26  2040 in the state of mt	BookRestaurant
+book me a restaurant for nine in statham	BookRestaurant
+i d like a table for ten in 2 minutes at french horn  sonning eye	BookRestaurant
+find a movie house for 07:52 showing ganges: river to heaven	SearchScreeningEvent
+what is the michael moore is a big fat stupid white man video game	SearchCreativeWork
+i want to eat close to bowlegs seven years from now	BookRestaurant
+for my playlist chill add the name cater fe she	AddToPlaylist
+search for the halfway home tv show	SearchCreativeWork
+find movie times	SearchScreeningEvent
+play journey list	PlayMusic
+tell me what animated movies i can see at the closest movie theatre	SearchScreeningEvent
+i d like to see the trailer tony parker	SearchCreativeWork
+what time is holiday heart showing at the movie house	SearchScreeningEvent
+play the movie white christmas	SearchCreativeWork
+is it forecast to be warm in doi inthanon national park	GetWeather
+add this tune to cristina s endorphin rush playlist	AddToPlaylist
+play a song by nash the slash	PlayMusic
+i rate doom 3: worlds on fire a 1 of 6	RateBook
+what time is phil ochs: there but for fortune playing at the movie house	SearchScreeningEvent
+add andreas johnson to my rock save the queen playlist	AddToPlaylist
+i d like to watch take this waltz	SearchScreeningEvent
+what are the mann theatres showtimes for secret sunshine	SearchScreeningEvent
+will there be snowfall in kitlope heritage conservancy	GetWeather
+play geddy lee music on spotify sort by top	PlayMusic
+rate in the eyes of mr  fury zero of 6	RateBook
+look up the tv series operace silver a	SearchCreativeWork
+i m looking for the tv series  called unborn	SearchCreativeWork
+play the song memories are my only witness	SearchCreativeWork
+i give the phishing manual four stars out of 6 	RateBook
+play clásicos del hip hop español	PlayMusic
+add rupee to my ultra metal playlist	AddToPlaylist
+add shi xin hui to my piano chill playlist 	AddToPlaylist
+what time is the clutching hand playing at amco entertainment	SearchScreeningEvent
+add circus to my post garage wave revival list	AddToPlaylist
+the chronicle charlie peace earns 4 stars from me	RateBook
+find conker: live and reloaded	SearchCreativeWork
+show me the nearest movie house showing the luckiest girl in the world	SearchScreeningEvent
+play track music from peter finestone on netflix sort by newest	PlayMusic
+play the song shine a light	PlayMusic
+book a popular restaurant of thai cuisine	BookRestaurant
+which animated movies are playing in the neighbourhood and when	SearchScreeningEvent
+i want to listen to the song only the greatest	SearchCreativeWork
+i d like to eat at the best restaurant 	BookRestaurant
+is it going to be chilly in western sahara in 13 hours	GetWeather
+i want to book a restaurant for four around zapata	BookRestaurant
+rate if tomorrow comes 2 of 6 stars	RateBook
+the book history by contract is rated five stars in my opinion 	RateBook
+i want to book a bar in bonaparte palau 	BookRestaurant
+i m looking for dead at 21  the tv series	SearchCreativeWork
+can you make reservations at a tea house that serves fettucine	BookRestaurant
+put a track by lil mama into my guest list sneaky zebra playlist  	AddToPlaylist
+put some frank ferrer into my edna st vincent millay playlist  	AddToPlaylist
+what is the forecast for niger	GetWeather
+rate this novel a 3	RateBook
+add this ruth crawford seeger song to my playlist called the soundtrack 007	AddToPlaylist
+is it going to snow next year in wv	GetWeather
+is romulus and the sabines playing at the nearest cinema at ten	SearchScreeningEvent
+show me the new showings for animated movies in the neighborhood	SearchScreeningEvent
+play the video game the genesis machine	SearchCreativeWork
+i want to go to 88th st-boyd av or close by and book seats for 10	BookRestaurant
+i need to add to the funk soul disco playlist my favorite artist	AddToPlaylist
+i want to book a cafe for 3 in fargo	BookRestaurant
+where can i watch tv series shopping spree	SearchCreativeWork
+play an andy silvester sound track from the thirties on spotify	PlayMusic
+i d like to eat at a popular brasserie in chile with a party of 5	BookRestaurant
+what s the forecast for my current place at five pm	GetWeather
+give private games 3 stars out of 6	RateBook
+in 17 minutes  will it be foggy in songimvelo game reserve	GetWeather
+how hot will it be in wisconsin on august fourth	GetWeather
+i d like to put qriii onto songs to sing in the car 	AddToPlaylist
+will it be chilly in oakdale ok	GetWeather
+add dwele to marguerite s eurovision 2016 playlist	AddToPlaylist
+what s the weather forecast for croatia on jul  25th	GetWeather
+find tv series titled a life in the death of joe meek	SearchCreativeWork
+open fadl shaker on spotify and play a melody starting with the newest	PlayMusic
+please add jency anthony to my playlist this is mozart	AddToPlaylist
+whats the weather in ga	GetWeather
+i rate the chronicle  son of the tree with four of 6 points 	RateBook
+add git to domingo indie	AddToPlaylist
+will there be cloud coverage in verdery  myanmar	GetWeather
+rate maps for lost lovers 1 of 6	RateBook
+will it snow in granbury	GetWeather
+play me a cinder block movement	PlayMusic
+find the tv series shaun the sheep	SearchCreativeWork
+i want to hear the jody williams sound track	PlayMusic
+what is the forecast for foggy conditions here in twenty one minutes	GetWeather
+book a table at grecian coffee house for 7 on apr  7th  2024	BookRestaurant
+show creative photograph of icewind dale: heart of winter	SearchCreativeWork
+rate the manxman 5 out of 6	RateBook
+add this song to my lo que suena new york playlist	AddToPlaylist
+find reproductions: songs of the human league	SearchCreativeWork
+play a 2001 sound track on deezer 	PlayMusic
+weather for ma in the morning	GetWeather
+play a ballad by bob johnston	PlayMusic
+is there a snowstorm in russia	GetWeather
+will it be nice on aug  the nineteenth in beda bulgaria	GetWeather
+i d like for you to put this artist to my evening commute playlist 	AddToPlaylist
+play the caps lock trailer	SearchCreativeWork
+give me the movie schedules for warren theatres	SearchScreeningEvent
+i need current movie schedules 	SearchScreeningEvent
+add even serpents shine to dorothea s indie hipster playlist	AddToPlaylist
+play ep by arjen anthony lucassen	PlayMusic
+give 4 points to this novel	RateBook
+add star light  star bright to my jazz classics playlist  	AddToPlaylist
+put nothing remains the same on my summer music playlist	AddToPlaylist
+weather for the night time in new mexico	GetWeather
+add pangaea to my gold edition playlist	AddToPlaylist
+find me a movie with the name oshin	SearchCreativeWork
+add ian stuart donaldson to canadian country	AddToPlaylist
+show me movie time for i am sorry  at my movie house	SearchScreeningEvent
+please add ruud jolie to my playlist guest list polygon	AddToPlaylist
+add patti page album to i love my neo soul	AddToPlaylist
+add an album by twink to my classic country playlist	AddToPlaylist
+will it be a snowy day in dalcour	GetWeather
+rate this essay a two out of 6	RateBook
+find the movie schedules for animated movies nearby at 09:44 am 	SearchScreeningEvent
+add armand van helden to my black sabbath the ozzy years playlist	AddToPlaylist
+give this chronicle a 4	RateBook
+i m looking for a churrascaria place with wifi that can serve a party of five 	BookRestaurant
+what time is goodbye mothers playing	SearchScreeningEvent
+book the city tavern in holiday ks 	BookRestaurant
+what movies are playing dickinson theatres	SearchScreeningEvent
+rate the key word and other mysteries 4 of 6	RateBook
+i d like to watch may blossom	SearchScreeningEvent
+play some music on slacker	PlayMusic
+i want to rate the ingenuity gap 3 out of 6	RateBook
+add song to my wild country playlist	AddToPlaylist
+what is the weather forecast for close-by burkina	GetWeather
+i want to watch supernatural: the unseen powers of animals	SearchCreativeWork
+listen to dragon ball: music collection	SearchCreativeWork
+add troy van leeuwen to my nu metal list	AddToPlaylist
+add born free to fresh r&b	AddToPlaylist
+book at table at forest av restaurant close-by for 2 1 second from now	BookRestaurant
+can you get me the trailer of the multiversity	SearchCreativeWork
+are there movies at malco theatres	SearchScreeningEvent
+rate the current chronicle series 3 out of 6 points	RateBook
+can i get the movie times	SearchScreeningEvent
+i want to add hind etin to my la mejor música dance 2017 playlist	AddToPlaylist
+play some latin on zvooq	PlayMusic
+what is the freezing forecast for british virgin islands	GetWeather
+pull up sweeney todd - il diabolico barbiere di fleet street	SearchCreativeWork
+put four rating on the raging quiet	RateBook
+show me the tv show limit of love: umizaru	SearchCreativeWork
+which movies are playing at the closest cinema	SearchScreeningEvent
+add this album by karl davydov to reyna s this is luis fonsi playlist	AddToPlaylist
+where can i see the television show falling away from me	SearchCreativeWork
+book me a table for 5 at a best rated restaurant in italy	BookRestaurant
+will there be a snowstorm in taberville	GetWeather
+add this song to this is no te va gustar playlist	AddToPlaylist
+can i get the movies  showtimes for the closest movie house 	SearchScreeningEvent
+do you have something like impossible is nothing by abderrahmane abdelli	PlayMusic
+what is the weather forecast for cistern	GetWeather
+please make reservations in yeager for seven am at a highly rated indian brasserie	BookRestaurant
+play me a nineties sound track	PlayMusic
+where can i find thor meets captain america	SearchCreativeWork
+i need to have pat alger s album placed onto the spotlight spain 2016 playlist 	AddToPlaylist
+can i get the movie times for fox theatres	SearchScreeningEvent
+i d like to watch wish you were dead	SearchScreeningEvent
+i d like to watch apocalypse 2024	SearchScreeningEvent
+show creativity of song a discord electric	SearchCreativeWork
+is love and other troubles playing	SearchScreeningEvent
+show me the current movie times	SearchScreeningEvent
+rate the lie tree five	RateBook
+i want to add another album to the wine & dine playlist  	AddToPlaylist
+add another tune to my pumping iron playlist  	AddToPlaylist
+play a track by mila islam from deezer	PlayMusic
+is it rainy season in manitou springs	GetWeather
+give 2 stars to the doom brigade	RateBook
+add this tune to my dinnertime acoustics list	AddToPlaylist
+what are the current movie schedules	SearchScreeningEvent
+what is the showtime for arsho	SearchScreeningEvent
+list movie times at harkins theatres	SearchScreeningEvent
+what movies are showing in the neighborhood 	SearchScreeningEvent
+play my playlist tgif on itunes	PlayMusic
+what will the weather be like on january 2nd  2025 in ga	GetWeather
+what animated movies are playing in the neighbourhood and when	SearchScreeningEvent
+book a spot at savoy hotel and grill that is neighboring wisconsin	BookRestaurant
+can you find me the back when i knew it all album	SearchCreativeWork
+add george thorogood to el mejor rock en espa��ol	AddToPlaylist
+play the album how insensitive	PlayMusic
+i m looking for the pokémon: the movie 2000 tv show	SearchCreativeWork
+place this tune onto my dinner for 2 playlist 	AddToPlaylist
+where can i see the trailer for love on the beat	SearchCreativeWork
+list movie times at megaplex theatres	SearchScreeningEvent
+will it be chillier at 06:05:48 in wagener réunion	GetWeather
+what is the weather in south bradenton	GetWeather
+get jump down painting	SearchCreativeWork
+please book a room in spaghetti warehouse for catalina  delores and brandie mendoza at 12 am	BookRestaurant
+what is the nh forecast for mexican hat	GetWeather
+i need to book a top-rated steakhouse this autumn  for 1 around azerbaijan	BookRestaurant
+will it be chillier at my current location in one minute	GetWeather
+show me heavenly sword	SearchCreativeWork
+what is the weather forecast for close-by gu 3 years from now	GetWeather
+will it be freezing on 4/20/2038 in american beach nc	GetWeather
+i need the wather for next week in the philippines	GetWeather
+add tune to my metal crash course playlist	AddToPlaylist
+i would like to book the best food court with persian food within the same area as ok for my ex husband and i 	BookRestaurant
+i d like to see the picture the principle of hope	SearchCreativeWork
+rate this series 2 out of 6	RateBook
+find a man needs a maid	SearchCreativeWork
+book a restaurant close by my daughters s work location with burrito three years from now	BookRestaurant
+add this tune to the refugee playlist	AddToPlaylist
+find time for movie times now	SearchScreeningEvent
+i would like to book a highly rated brasserie with souvlaki neighboring la next week 	BookRestaurant
+find the panic in needle park	SearchScreeningEvent
+is it freezing on jun  the 21st in apshawa south africa	GetWeather
+i need to take three people to eat	BookRestaurant
+play a 2006 chant	PlayMusic
+show me the schedule of the loves of letty in cinema closest	SearchScreeningEvent
+play the top 20 ep from the fifties by john bundrick 	PlayMusic
+show creativity of photograph of my wonderful day	SearchCreativeWork
+book a table in the united states for 10 at the berghoff	BookRestaurant
+i d like to book a brasserie in virginia city  ga 	BookRestaurant
+will it be temperate in the same area in vi	GetWeather
+rate the current novel four out of 6 points	RateBook
+is it going to get chillier near hocking state forest	GetWeather
+for the current saga i rate 2 of 6 stars	RateBook
+i want to play the video game espn major league soccer	SearchCreativeWork
+rate the current book a three	RateBook
+rate this novel 0 of 6 stars	RateBook
+is it going to be chillier at 10 pm in texas	GetWeather
+what s the weather in timbo	GetWeather
+add the blurred crusade to crate diggers anonymous	AddToPlaylist
+tell me the weather forecast for sugarloaf provincial park ten weeks from now	GetWeather
+add a gackt camui track to the white noise playlist  	AddToPlaylist
+rate canto for a gypsy two of 6 stars	RateBook
+i m looking for circus world 	SearchCreativeWork
+this textbook gets a two	RateBook
+show me the movie times 	SearchScreeningEvent
+add song to my underground hits	AddToPlaylist
+play the album journeyman	SearchCreativeWork
+find the family jams saga	SearchCreativeWork
+play rob mills album the golden archipelago 	PlayMusic
+book a spot at a restaurant within walking distance of palau	BookRestaurant
+find me the balance and timing book	SearchCreativeWork
+find movie schedules for bow tie cinemas 	SearchScreeningEvent
+add get happy to cherry s las canciones más lindas del mundo	AddToPlaylist
+rate this textbook a 1	RateBook
+shw the picture twin husbands	SearchCreativeWork
+rate a taste of blackberries  a three	RateBook
+play the 1991 soundtrack from ian mcdonald	PlayMusic
+find an album called just call me stupid 	SearchCreativeWork
+play the insoc ep	PlayMusic
+i want to hear major harris s songs from the fifties	PlayMusic
+book a restaurant in donnelly	BookRestaurant
+rate the saint in trouble 1 of 6	RateBook
+play punk rock music	PlayMusic
+look for a photograph of i wanna sex you up	SearchCreativeWork
+what is the humidity like in faraway on ak	GetWeather
+i d like to eat at an internet restaurant with a party of four	BookRestaurant
+when is just before nightfall playing	SearchScreeningEvent
+play moondog s chupacabra 	PlayMusic
+add album to pop rising	AddToPlaylist
+rate this book three points	RateBook
+i am giving this current book album 0 out of 6 stars	RateBook
+play artist vlada divljan from something he did that is good	PlayMusic
+what will the humidity be in varnado georgia at one am	GetWeather
+add no prejudice to 90s indie	AddToPlaylist
+what are the movies movie times nearby	SearchScreeningEvent
+i want to hear some songs from the twenties	PlayMusic
+please make reservations for nine at 3 am 	BookRestaurant
+can you pull up queen of the organ	SearchCreativeWork
+lets hear some dawood sarkhosh from their the power of your love album from groove shark	PlayMusic
+will it get overcast in la dolores	GetWeather
+book a spot for kelli  jean and i at a pub at elevenses	BookRestaurant
+add this candi staton artist to my dancefloor hits	AddToPlaylist
+i want to add a song by jazz brasileiro	AddToPlaylist
+rate wielding a red sword 0 stars	RateBook
+book a taverna that serves bengali for six at five	BookRestaurant
+play the tv series heart of gold	SearchCreativeWork
+show crafty hands saga	SearchCreativeWork
+will it be hotter in wyomissing hills	GetWeather
+show weather while sunset in the same area in south carolina	GetWeather
+table for one somewhere in palco	BookRestaurant
+i would like to add something by kuk harrell to my hip hop 2017 new school playlist	AddToPlaylist
+add list of rush instrumentals to this is lady antebellum	AddToPlaylist
+where can i see a slice of life	SearchScreeningEvent
+the current textbook gets a 2 rating  	RateBook
+add wing track to all a cappella	AddToPlaylist
+show me dangers of the canadian mounted	SearchCreativeWork
+please add this this tune to the playlist this is selena	AddToPlaylist
+what will the weather be in stelvio national park 1 hour and 1 minute from now	GetWeather
+can you put musiri subramania iyer s song onto the lo-fi love soundtrack	AddToPlaylist
+i want to add michelle heaton to this is chopin	AddToPlaylist
+show me the movie operetta for the theatre organ	SearchCreativeWork
+where s the nearest movie house playing no trains no planes	SearchScreeningEvent
+put a xiang xiang track onto women of the blues	AddToPlaylist
+can you add a track by david wolfenberger to janell s all funked up playlist	AddToPlaylist
+play the album vibrations by marion elise raven	PlayMusic
+add fabri fibra to evening acoustic	AddToPlaylist
+can you play any chant from the fourties	PlayMusic
+show the night riders	SearchCreativeWork
+i m looking for a movie called salvage mice 	SearchCreativeWork
+find your personal touch 	SearchCreativeWork
+add this tune to my weekend playlist	AddToPlaylist
+is it going to storm in black rock alaska	GetWeather
+show the movie schedules at united paramount theatres	SearchScreeningEvent
+i want to read the saga michael clayton	SearchCreativeWork
+book me a table for 3 at tkk fried chicken in sri lanka	BookRestaurant
+rate this book titled the improvisatore five stars	RateBook
+book a restaurant for one person at 7 am	BookRestaurant
+weather for beauregard il	GetWeather
+will there be alot of wind on march 13th in lost creek bahrain	GetWeather
+i d like a reservation at a place in iran for neva  alice and maggie parker 	BookRestaurant
+show me movie schedule for animated movie around here at eleven a m 	SearchScreeningEvent
+i give this book dictionary of the english language a 4 rating	RateBook
+play some symphonic rock 	PlayMusic
+add to my playlist all funked up this track	AddToPlaylist
+find a tv series called armageddon summer	SearchCreativeWork
+find politicsnation with al sharpton 	SearchCreativeWork
+rate this album 0 points out of 6	RateBook
+add leah kauffman to my uncharted 4 nathan drake playlist	AddToPlaylist
+rate this album two out of 6	RateBook

TEXTOIR-main/open_intent_detection/README.md

@@ -0,0 +1,260 @@
+# Open Intent Detection
+
+This package provides the toolkit for open intent detection implemented with PyTorch.
+
+## Introduction
+
+Open intent detection aims to identify n-class known intents, and detect the one-class open intent, which is regarded as an open classification problem. The following is an example:
+
+<img src="figs/open_intent_detection.png" width="360" height = "200">
+
+We collect benchmark intent datasets, and reproduce related methods to our best. For the convenience of users, we provide flexible and extensible interfaces to add new methods. Welcome to contact us (zhang-hl20@mails.tsinghua.edu.cn) to add your methods!
+
+## Basic Information
+
+### Benchmark Datasets
+| Datasets | Source |
+| :---: | :---: |
+| [BANKING](../data/banking) | [Paper](https://aclanthology.org/2020.nlp4convai-1.5/) |
+| [OOS](../data/oos) | [Paper](https://aclanthology.org/D19-1131/) |
+| [StackOverflow](../data/stackoverflow) | [Paper](https://aclanthology.org/W15-1509.pdf) |
+
+### Integrated Models
+
+| Model Name | Source | Published |
+| :---: | :---: | :---: |
+| [OpenMax*](./examples/run_OpenMax.sh) | [Paper](https://openaccess.thecvf.com/content_cvpr_2016/papers/Bendale_Towards_Open_Set_CVPR_2016_paper.pdf) [Code](https://github.com/abhijitbendale/OSDN) | CVPR 2016 |
+| [MSP](./examples/run_MSP.sh) | [Paper](https://arxiv.org/pdf/1610.02136.pdf) [Code](https://github.com/hendrycks/error-detection) | ICLR 2017 |
+| [DOC](./examples/run_DOC.sh) | [Paper](https://aclanthology.org/D17-1314.pdf) [Code](https://github.com/leishu02/EMNLP2017_DOC) | EMNLP 2017 |
+| [DeepUnk](./examples/run_DeepUnk.sh) | [Paper](https://aclanthology.org/P19-1548.pdf) [Code](https://github.com/thuiar/DeepUnkID) | ACL 2019 |
+| [SEG](./examples/run_SEG.sh) | [Paper](https://aclanthology.org/2020.acl-main.99) [Code](https://github.com/fanolabs/0shot-classification) | ACL 2020 |
+| [ADB](./examples/run_ADB.sh) | [Paper](https://ojs.aaai.org/index.php/AAAI/article/view/17690) [Code](https://github.com/thuiar/Adaptive-Decision-Boundary) | AAAI 2021 |
+| [(K+1)-way](./examples/run_K+1-way.sh) | [Paper](https://aclanthology.org/2021.acl-long.273) [Code](https://github.com/fanolabs/out-of-scope-intent-detection) | ACL 2021 |
+| [MDF](./examples/run_MDF.sh) | [Paper](https://aclanthology.org/2021.acl-long.85.pdf) [Code](https://github.com/rivercold/BERT-unsupervised-OOD) | ACL 2021 |
+| [ARPL*](./examples/run_ARPL.sh) | [Paper](https://ieeexplore.ieee.org/document/9521769) [Code](https://github.com/iCGY96/ARPL) | IEEE TPAMI 2022 |
+| [KNNCL](./examples/run_KNNCL.sh) | [Paper](https://aclanthology.org/2022.acl-long.352/) [Code](https://github.com/zyh190507/KnnContrastiveForOOD) | ACL 2022 |
+| [DA-ADB](./examples/run_DA-ADB.sh) | [Paper](https://ieeexplore.ieee.org/document/10097558) [Code](https://github.com/thuiar/TEXTOIR) | IEEE/ACM TASLP 2023 |
+
+We welcome any issues and requests for model implementation and bug fix. 
+
+### Data Settings
+
+Each dataset is split to training, development, and testing sets. We select partial intents as known (the labeled ratio can be changed) for training, and use all intents for testing. All the unknown intents are regarded as one open class (with token \<UNK> or \<OOS> in our codes). More detailed information can be seen in the [paper](https://ojs.aaai.org/index.php/AAAI/article/view/17690).
+
+### Parameter Configurations
+
+The basic parameters include parsing parameters about selected dataset, method, setting, etc. More details can be seen in [run.py](./run.py). For specific parameters of each method, we support add configuration files with different hyper-parameters in the [configs](./configs) directory. 
+
+An example can be seen in [ADB.py](./configs/ADB.py). Notice that the config file name is corresponding to the parsing parameter.
+
+Normally, the input commands are as follows:
+```
+python run.py --dataset xxx --known_cls_ratio xxx --labeled_ratio xxx --config_file_name xxx
+```
+
+Notice that if you want to train the model, save the model, or save the testing results, you need to add related parameters (--train, --save_model, --save_results)
+
+### Results
+The detailed results can be seen in [results.md](results/results.md).
+#### Overall Performance
+
+* KIR means "Known Intent Ratio". "Open" and "Known" denote the macro f1-score over open class and known classes respectively.    
+* KNNCL (All) utilizes all 12 transformer layers for fine-tuning. KNNCL (last) utilizes only the last transformer layer for fine-tuning as the other baselines.
+* We also test the performance of the DA-ADB method on the backbone of llama, corresponding to the methond DA-ADB_llama.
+
+| | | BANKING     |  | OOS      |  |  StackOverflow     |  |  
+|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|
+| KIR | Methods | Accuracy | F1-score | Accuracy  |F1-score  | Accuracy | F1-score | 
+|0.25|MSP|42.19|49.92|53.38|51.23|27.91|37.49|
+|0.25|SEG|48.73|51.49|52.18|47.0|23.33|34.4|
+|0.25|OpenMax|47.76|53.18|70.27|63.03|38.97|45.35|
+|0.25|LOF|66.73|63.38|87.77|78.13|25.02|35.29|
+|0.25|DOC|70.31|65.74|86.08|75.86|57.75|57.34|
+|0.25|DeepUnk|70.68|65.57|87.18|77.32|40.03|45.64|
+|0.25|(K+1)-way|76.66|68.44|85.36|74.43|49.75|50.82|
+|0.25|MDF|77.17|46.85|76.56|50.34|74.1|53.95|
+|0.25|ARPL|76.8|64.01|84.51|73.44|66.76|62.62|
+|0.25|KNNCL (Last)|73.01|66.23|89.87|79.23|28.65|37.37|
+|0.25|ADB|79.33|71.63|88.3|78.23|86.75|79.85|
+|0.25|KNNCL (All)|**86.14**|**77.01**|**93.07**|82.45|85.04|79.06|
+|0.25|DA-ADB|81.19|73.73|89.48|79.92|**89.07**|**82.83**|
+|0.25|DA-ADB_llama|83.29|76.78|91.53|**84.54**|85.54|77.81|
+|||||||||
+|0.5|MSP|61.67|72.51|66.68|72.7|53.23|62.7|
+|0.5|SEG|55.11|63.32|60.67|62.55|43.04|55.1|
+|0.5|OpenMax|65.53|74.64|80.22|79.86|60.27|67.72|
+|0.5|LOF|71.13|76.26|85.22|83.86|44.56|56.57|
+|0.5|DOC|74.6|78.24|85.19|83.89|73.88|76.8|
+|0.5|DeepUnk|71.01|75.41|84.95|83.35|55.46|64.78|
+|0.5|(K+1)-way|74.65|77.83|82.19|81.56|62.57|68.81|
+|0.5|MDF|60.18|64.1|60.72|61.61|56.46|61.47|
+|0.5|ARPL|74.11|77.77|80.36|80.88|75.65|77.87|
+|0.5|KNNCL (Last)|70.41|74.96|85.32|83.31|45.38|56.69|
+|0.5|ADB|79.61|81.34|86.54|85.16|86.49|85.54|
+|0.5|KNNCL (All)|**82.76**|81.31|88.66|83.99|86.69|86.15|
+|0.5|DA-ADB|81.51|82.53|87.93|85.64|**87.78**|**86.91**|
+|0.5|DA-ADB_llama|82.66|**83.67**|**90.29**|**88.86**|86.42|86.09|
+|||||||||
+|0.75|MSP|77.08|84.33|76.19|83.48|73.2|78.7|
+|0.75|SEG|64.65|69.54|42.78|42.7|62.72|69.97|
+|0.75|OpenMax|78.32|84.95|75.36|71.17|75.78|80.9|
+|0.75|LOF|77.21|83.64|85.07|87.2|65.05|71.87|
+|0.75|DOC|78.94|83.79|85.93|87.87|80.55|84.37|
+|0.75|DeepUnk|74.73|81.12|84.61|86.53|71.56|77.63|
+|0.75|(K+1)-way|79.18|84.71|83.51|86.66|74.0|78.95|
+|0.75|MDF|64.59|74.76|63.98|72.02|62.98|71.12|
+|0.75|ARPL|79.6|85.16|81.29|86.0|79.64|83.85|
+|0.75|KNNCL (Last)|74.78|81.25|84.12|86.1|65.01|71.85|
+|0.75|ADB|81.39|86.11|86.99|88.94|82.89|86.11|
+|0.75|KNNCL (All)|77.50|82.30|85.07|85.11|83.15|86.73|
+|0.75|DA-ADB|81.12|85.65|87.39|88.41|**83.56**|**86.84**|
+|0.75|DA-ADB_llama|**82.19**|**86.52**|**89.23**|**90.48**|82.69|86.28|
+
+#### Fine-grained Performance
+
+|  | | BANKING     |  | OOS      |  |  StackOverflow     |  |  
+|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|
+| KIR | Methods | Open | Known | Open | Known | Open | Known | 
+|0.25|MSP|39.42|50.47|59.26|51.02|11.66|42.66|
+|0.25|SEG|51.58|51.48|59.22|46.67|4.19|40.44|
+|0.25|OpenMax|48.52|53.42|77.51|62.65|34.52|47.51|
+|0.25|LOF|72.64|62.89|91.96|77.77|7.14|40.92|
+|0.25|DOC|76.64|65.16|90.78|75.46|62.5|56.3|
+|0.25|DeepUnk|76.98|64.97|91.61|76.95|36.87|47.39|
+|0.25|(K+1)-way|82.66|67.7|90.27|74.02|52.23|50.54|
+|0.25|MDF|85.7|44.8|84.89|49.43|83.03|48.13|
+|0.25|ARPL|83.39|62.99|89.63|73.01|72.95|60.55|
+|0.25|KNNCL (Last)|79.34|65.54|93.56|78.85|15.26|41.79|
+|0.25|ADB|85.05|70.92|92.36|77.85|90.96|77.62|
+|0.25|KNNCL (All)|**90.55**|**76.30**|**95.73**|82.10|89.59|76.96|
+|0.25|DA-ADB|86.57|73.05|93.2|79.57|**92.65**|**80.87**|
+|0.25|DA-ADB_llama|88.19|76.18|94.55|**84.27**|90.04|75.36|
+|||||||||
+|0.5|MSP|46.29|73.2|63.71|72.82|26.94|66.28|
+|0.5|SEG|43.03|63.85|61.34|62.57|4.72|60.14|
+|0.5|OpenMax|55.03|75.16|82.15|79.83|46.11|69.88|
+|0.5|LOF|66.81|76.51|87.57|83.81|5.18|61.71|
+|0.5|DOC|72.66|78.38|87.45|83.84|71.18|77.37|
+|0.5|DeepUnk|67.8|75.61|87.48|83.3|35.8|67.67|
+|0.5|(K+1)-way|72.58|77.97|84.25|81.52|51.69|70.53|
+|0.5|MDF|57.72|64.27|62.31|61.6|50.19|62.6|
+|0.5|ARPL|71.79|77.93|81.81|80.87|73.97|78.26|
+|0.5|KNNCL (Last)|67.21|75.16|87.85|83.25|8.5|61.5|
+|0.5|ADB|79.43|81.39|88.6|85.12|87.7|85.32|
+|0.5|KNNCL (All)|**84.28**|81.23|91.17|83.89|87.59|86.01|
+|0.5|DA-ADB|81.93|82.54|90.1|85.58|**88.86**|**86.71**|
+|0.5|DA-ADB_llama|83.23|**83.68**|**92.04**|**88.82**|87.43|85.95|
+|||||||||
+|0.75|MSP|46.05|84.99|63.86|83.65|37.86|81.42|
+|0.75|SEG|37.22|70.1|40.74|42.72|6.0|74.24|
+|0.75|OpenMax|53.02|85.5|75.18|71.14|49.69|82.98|
+|0.75|LOF|54.19|84.15|82.81|87.24|5.22|76.31|
+|0.75|DOC|63.51|84.14|83.87|87.91|65.32|85.64|
+|0.75|DeepUnk|50.57|81.65|82.67|86.57|34.38|80.51|
+|0.75|(K+1)-way|59.89|85.14|79.59|86.72|45.22|81.2|
+|0.75|MDF|33.43|75.47|51.33|72.21|28.52|73.96|
+|0.75|ARPL|61.26|85.58|74.67|86.1|62.99|85.24|
+|0.75|KNNCL (Last)|51.42|81.76|82.05|86.14|7.19|76.16|
+|0.75|ADB|67.34|86.44|84.85|88.97|74.1|86.91|
+|0.75|KNNCL (All)|67.01|82.56|84.31|85.11|72.81|**87.66**|
+|0.75|DA-ADB|69.37|85.93|86.0|88.43|**74.55**|**87.66**|
+|0.75|DA-ADB_llama|**70.94**|**86.78**|**87.93**|**90.51**|72.84|87.17|
+
+## Tutorials
+### a. How to add a new dataset? 
+1. Prepare Data  
+Create a new directory to store your dataset in the [data](../data) directory. You should provide the train.tsv, dev.tsv, and test.tsv, with the same formats as in the provided [datasets](../data/banking).
+
+2. Dataloader Setting  
+Calculate the maximum sentence length (token unit) and count the labels of the dataset. Add them in the [file](./dataloaders/init.py) as follows:  
+```
+max_seq_lengths = {
+    'new_dataset': max_length
+}
+benchmark_labels = {
+    'new_dataset': label_list
+}
+```
+
+### b. How to add a new backbone?
+
+1. Add a new backbone in the [backbones](./backbones) directory. For example, we provide some bert-based backbones in the [file](./backbones/bert.py). And we also provide llama-based backbones in the [file](./backbones/llama.py). You can add a new backbone in the same way.
+
+2. Add the new backbone mapping in the [file](./backbones/__init__.py) as follows:
+```
+from .bert import new_backbone_class # from .llama import new_backbone_class
+backbones_map = {
+    'new_backbone': new_backbone_class
+}
+```
+Add a new loss in the [losses](./losses) directory is almost the same as adding a new backbone.
+
+### c. How to add a new method?
+
+1. Configuration Setting   
+Create a new file, named "method_name.py" in the [configs](./configs) directory, and set the hyper-parameters for the method (an example can be seen in [MSP.py](./configs/MSP.py)). 
+
+2. Dataloader Setting  
+Add the dataloader mapping if you use new backbone for the method. For example, the bert-based model corresponds to the bert dataloader as follows.
+```
+from .bert_loader import BERT_Loader
+backbone_loader_map = {
+    'bert': BERT_Loader,
+    'bert_xxx': BERT_Loader,
+}
+```
+
+The llama-based model corresponds to the llama dataloader as follows.
+```
+from .llama_loader import LLAMA_Loader
+backbone_loader_map = {
+    'llama_disaware': LLAMA_Loader,
+}
+```
+
+3. Add Methods  (Take MSP as an example)
+- Create a new directory, named "MSP" in the [methods](./methods) directory. 
+
+- Add the manager file for MSP. The file should include the method manager class (e.g., MSPManager), which includes training, evalutation, and testing modules for the method. An example can be seen in [manager.py](./methods/MSP/manager.py).  
+
+- Add the related method dependency in [__init__.py](./methods/__init__.py) as below:
+```
+from .MSP.manager import xxxManager
+method_map = {
+    'MSP': MSPManager
+}
+```
+(The key corresponds to the input parameter "method")
+
+4. Run Examples
+Add a script in the [examples](./examples) directory, and configure the parsing parameters in the [run.py](./run.py). You can also run the programs serially by setting the combination of different parameters. A running example is shown in [run_MSP.sh](./examples/run_MSP.sh).
+
+## Citations
+
+If this work is helpful, or you want to use the codes and results in this repo, please cite the following papers:
+
+* [TEXTOIR: An Integrated and Visualized Platform for Text Open Intent Recognition](https://aclanthology.org/2021.acl-demo.20/)
+* [Learning Discriminative Representations and Decision Boundaries for Open Intent Detection](https://ieeexplore.ieee.org/document/10097558)
+
+```
+@inproceedings{zhang-etal-2021-textoir,
+    title = "{TEXTOIR}: An Integrated and Visualized Platform for Text Open Intent Recognition",
+    author = "Zhang, Hanlei  and Li, Xiaoteng  and Xu, Hua  and Zhang, Panpan and Zhao, Kang  and Gao, Kai",
+    booktitle = "Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations",
+    pages = "167--174",
+    year = "2021",
+    url = "https://aclanthology.org/2021.acl-demo.20",
+    doi = "10.18653/v1/2021.acl-demo.20",
+}
+```
+```
+@article{DA-ADB, 
+    title = {Learning Discriminative Representations and Decision Boundaries for Open Intent Detection},  
+    author = {Zhang, Hanlei and Xu, Hua and Zhao, Shaojie and Zhou, Qianrui}, 
+    journal = {IEEE/ACM Transactions on Audio, Speech, and Language Processing},  
+    volume = {31},
+    pages = {1611-1623},
+    year = {2023}, 
+    doi = {10.1109/TASLP.2023.3265203} 
+} 
+```

TEXTOIR-main/open_intent_detection/__init__.py

@@ -0,0 +1,28 @@
+'''
+The open intent detection module of the TEXTOIR toolkit @THUIAR. 
+
+If you use this code, please cite the following papers:
+@inproceedings{zhang-etal-2021-textoir,
+    title = "{TEXTOIR}: An Integrated and Visualized Platform for Text Open Intent Recognition",
+    author = "Zhang, Hanlei  and
+      Li, Xiaoteng  and
+      Xu, Hua  and
+      Zhang, Panpan  and
+      Zhao, Kang  and
+      Gao, Kai",
+    booktitle = "Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations",
+    year = "2021",
+    pages = "167--174",
+}
+@article{Zhang_Xu_Lin_2021, 
+      title={Deep Open Intent Classification with Adaptive Decision Boundary}, 
+      volume={35}, 
+      number={16}, 
+      journal={Proceedings of the AAAI Conference on Artificial Intelligence}, 
+      author={Zhang, Hanlei and Xu, Hua and Lin, Ting-En}, 
+      year={2021}, 
+      month={May}, 
+      pages={14374-14382} 
+}
+'''
+

TEXTOIR-main/open_intent_detection/backbones/__init__.py

@@ -0,0 +1,15 @@
+from .bert import BERT, BERT_Norm, BERT_K_1_way, BERT_SEG, BERT_Disaware, BERT_DOC, BERT_MDF, BERT_MDF_Pretrain, BERT_KNNCL
+from .llama import LLAMA_lora_Disaware
+
+backbones_map = {
+                    'bert': BERT, 
+                    'bert_norm': BERT_Norm,
+                    'bert_K+1-way': BERT_K_1_way,
+                    'bert_seg': BERT_SEG,
+                    'bert_disaware': BERT_Disaware,
+                    'bert_doc': BERT_DOC,
+                    'bert_mdf': BERT_MDF,
+                    'bert_mdf_pretrain': BERT_MDF_Pretrain,
+                    'bert_knncl': BERT_KNNCL,
+                    'llama_disaware': LLAMA_lora_Disaware,
+                }

TEXTOIR-main/open_intent_detection/backbones/base.py

@@ -0,0 +1,51 @@
+import torch
+import logging
+from transformers import AdamW, get_linear_schedule_with_warmup
+from .utils import freeze_bert_parameters, freeze_bert_parameters_KCL
+from .__init__ import backbones_map
+
+class ModelManager:
+
+    def __init__(self, args, data, logger_name = 'Detection'):
+        
+        self.logger = logging.getLogger(logger_name)
+
+    def set_optimizer(self, model, num_train_examples, train_batch_size, num_train_epochs, lr, warmup_proportion):
+        num_train_optimization_steps = int(num_train_examples / train_batch_size) * num_train_epochs
+
+        param_optimizer = list(model.named_parameters())
+        no_decay = ['bias', 'LayerNorm.bias', 'LayerNorm.weight']
+        optimizer_grouped_parameters = [
+            {'params': [p for n, p in param_optimizer if not any(nd in n for nd in no_decay)], 'weight_decay': 0.01},
+            {'params': [p for n, p in param_optimizer if any(nd in n for nd in no_decay)], 'weight_decay': 0.0}
+        ]
+        
+        optimizer = AdamW(optimizer_grouped_parameters, lr = lr, correct_bias=False)
+        num_warmup_steps= int(num_train_examples * num_train_epochs * warmup_proportion / train_batch_size)
+        
+        scheduler = get_linear_schedule_with_warmup(optimizer,
+                                                    num_warmup_steps=num_warmup_steps,
+                                                    num_training_steps=num_train_optimization_steps)
+        
+        return optimizer, scheduler
+    
+    def set_model(self, args, pattern):
+        backbone = backbones_map[args.backbone]
+        args.device = self.device = torch.device('cuda:%d' % int(args.gpu_id) if torch.cuda.is_available() else 'cpu')
+
+        if pattern == 'bert' or pattern == 'llama':
+            if hasattr(backbone, 'from_pretrained'):
+                model = backbone.from_pretrained('bert-base-uncased', args = args)  
+            else:
+                model = backbone(args)
+                
+            if args.freeze_backbone_parameters:
+                self.logger.info('Freeze all parameters but the last layer for efficiency')
+                if args.method == 'KCL':
+                    model = freeze_bert_parameters_KCL(model)
+                else: 
+                    model = freeze_bert_parameters(model)
+        model.to(self.device)
+        
+        return model
+

TEXTOIR-main/open_intent_detection/backbones/bert.py

@@ -0,0 +1,624 @@
+import torch
+import math
+import torch.nn.functional as F
+import numpy as np
+
+from torch import nn
+from torch.nn import CrossEntropyLoss, MSELoss
+from torch.nn.parameter import Parameter
+from transformers import BertPreTrainedModel, BertModel, BertForMaskedLM, AutoConfig
+from transformers.modeling_outputs import SequenceClassifierOutput
+
+from .utils import ConvexSampler
+
+activation_map = {'relu': nn.ReLU(), 'tanh': nn.Tanh()}
+
+class BERT_DOC(BertPreTrainedModel):
+    def __init__(self, config, args):
+        super(BERT_DOC, self).__init__(config)
+        self.num_labels = args.num_labels
+        self.bert = BertModel(config)
+        self.dense = nn.Linear(config.hidden_size, config.hidden_size)
+        self.activation = activation_map[args.activation]
+        self.dropout = nn.Dropout(config.hidden_dropout_prob)
+        self.classifier = nn.Linear(config.hidden_size, args.num_labels)
+        self.init_weights()
+
+    def forward(self, input_ids=None, token_type_ids=None, attention_mask=None, labels=None,
+                feature_ext=False, mode=None, loss_fct=None, centroids = None):
+        outputs = self.bert(
+            input_ids, token_type_ids=token_type_ids, attention_mask=attention_mask, output_hidden_states=True)
+        encoded_layer_12 = outputs.hidden_states
+        
+        pooled_output = self.dense(encoded_layer_12[-1].mean(dim=1))
+        pooled_output = self.dropout(pooled_output)
+        pooled_output = self.activation(pooled_output)
+        
+        logits = self.classifier(pooled_output)
+        logits = self.dropout(logits)
+        sigmoid = nn.Sigmoid()
+        logits = sigmoid(logits)
+
+        if feature_ext:
+            return pooled_output
+        else:
+            if mode == 'train':
+                target = F.one_hot(labels, num_classes = self.num_labels)
+                loss_bce = loss_fct(logits, target.float())
+                return loss_bce
+            else:
+                return pooled_output, logits
+
+class BERT(BertPreTrainedModel):
+    def __init__(self, config, args):
+        super(BERT, self).__init__(config)
+        self.num_labels = args.num_labels
+        self.bert = BertModel(config)
+        self.dense = nn.Linear(config.hidden_size, config.hidden_size)
+        self.activation = activation_map[args.activation]
+        self.dropout = nn.Dropout(config.hidden_dropout_prob)
+        self.classifier = nn.Linear(config.hidden_size, args.num_labels)
+        self.init_weights()
+
+    def forward(self, input_ids=None, token_type_ids=None, attention_mask=None, labels=None,
+                feature_ext=False, mode=None, loss_fct=None, centroids = None):
+        outputs = self.bert(
+            input_ids, token_type_ids=token_type_ids, attention_mask=attention_mask, output_hidden_states=True)
+        encoded_layer_12 = outputs.hidden_states
+        
+        pooled_output = self.dense(encoded_layer_12[-1].mean(dim=1))
+        pooled_output = self.activation(pooled_output)
+        pooled_output = self.dropout(pooled_output)
+
+        logits = self.classifier(pooled_output)
+
+        if feature_ext:
+            return pooled_output
+        else:
+            if mode == 'train':
+                loss_ce = loss_fct(logits, labels)
+                return loss_ce
+            else:
+                return pooled_output, logits
+
+class BERT_Norm(BertPreTrainedModel):
+    def __init__(self, config, args):
+
+        super(BERT_Norm, self).__init__(config)
+        self.num_labels = args.num_labels
+        self.bert = BertModel(config)
+        self.dense = nn.Linear(config.hidden_size, config.hidden_size)
+        self.dropout = nn.Dropout(config.hidden_dropout_prob)
+        self.init_weights()
+        self.weight = Parameter(torch.FloatTensor(args.num_labels, args.feat_dim).to(args.device))
+        nn.init.xavier_uniform_(self.weight)
+
+    def forward(self, input_ids=None, token_type_ids=None, attention_mask=None, labels=None,
+                feature_ext=False, mode=None, loss_fct=None, device = None, head = None):
+        outputs = self.bert(
+            input_ids, token_type_ids=token_type_ids, attention_mask=attention_mask, output_hidden_states=True)
+        encoded_layer_12 = outputs.hidden_states
+        pooled_output = encoded_layer_12[-1].mean(dim=1)
+        pooled_output = self.dropout(pooled_output)
+        pooled_output = F.normalize(pooled_output)
+
+        logits = F.linear(pooled_output, F.normalize(self.weight))
+        logits = F.softmax(logits, dim = 1)
+        
+        if feature_ext:
+            return pooled_output
+        else:
+            if mode == 'train':
+                loss = loss_fct(logits, labels)
+                return loss
+            else:
+                return pooled_output, logits
+
+class BERT_K_1_way(BertPreTrainedModel):
+    def __init__(self, config, args):
+        super(BERT_K_1_way, self).__init__(config)
+        self.num_labels = args.num_labels
+        self.bert = BertModel(config)
+        self.dense = nn.Linear(config.hidden_size, config.hidden_size)
+        self.activation = activation_map[args.activation]
+        self.dropout = nn.Dropout(config.hidden_dropout_prob)
+        self.sampler = ConvexSampler(args)
+        self.classifier = nn.Linear(config.hidden_size, self.num_labels + 1)
+        self.t = args.temp
+        self.init_weights()
+
+    def forward(self, input_ids = None, token_type_ids = None, attention_mask=None , labels = None,
+                feature_ext = False, mode = None, loss_fct = None):
+        outputs = self.bert(
+            input_ids, token_type_ids=token_type_ids, attention_mask=attention_mask, output_hidden_states=True)
+        encoded_layer_12 = outputs.hidden_states
+        pooled_output = self.dense(encoded_layer_12[-1].mean(dim=1))
+        
+        if mode is not 'test':
+            pooled_output, labels = self.sampler(pooled_output, labels, mode=mode)
+            
+        pooled_output = self.activation(pooled_output)
+        pooled_output = self.dropout(pooled_output)
+        logits = self.classifier(pooled_output)
+        
+        if feature_ext:
+            return pooled_output
+        else:
+            if mode == 'train':
+                loss = loss_fct(torch.div(logits, self.t), labels)
+                return loss
+            else:
+                return pooled_output, logits, labels
+
+class BERT_SEG(BertPreTrainedModel):
+    def __init__(self, config, args):
+        
+        super(BERT_SEG, self).__init__(config)
+        self.num_labels = args.num_labels
+        self.bert = BertModel(config)
+        self.dense = nn.Linear(config.hidden_size, config.hidden_size)
+        self.activation = activation_map[args.activation]
+        self.dropout = nn.Dropout(config.hidden_dropout_prob)
+        self.init_weights()
+
+        self.alpha = args.alpha
+        self.lambda_ = args.lambda_
+        self.means = nn.Parameter(torch.randn(self.num_labels, args.feat_dim).cuda())
+        nn.init.xavier_uniform_(self.means, gain=math.sqrt(2.0))
+
+
+    def forward(self, input_ids = None, token_type_ids = None, attention_mask=None , labels = None,
+                feature_ext = False, mode = None, device=None, p_y = None, class_emb=None, loss_fct=None):
+
+        outputs = self.bert(
+            input_ids, token_type_ids=token_type_ids, attention_mask=attention_mask, output_hidden_states=True)
+        encoded_layer_12 = outputs.hidden_states
+        pooled_output = self.dense(encoded_layer_12[-1].mean(dim=1))
+        pooled_output = self.activation(pooled_output)
+        pooled_output = self.dropout(pooled_output)
+
+        if feature_ext:
+            return pooled_output
+        else:
+            
+            batch_size = pooled_output.shape[0]
+
+            XY = torch.matmul(pooled_output, torch.transpose(self.means, 0, 1))
+            XX = torch.sum(pooled_output ** 2, dim=1, keepdim=True)
+            YY = torch.sum(torch.transpose(self.means, 0, 1)**2, dim=0, keepdim=True)
+            neg_sqr_dist = - 0.5 * (XX - 2.0 * XY + YY)
+            
+            # with p_y
+            ########################################
+            p_y = p_y.expand_as(neg_sqr_dist).to(device)  # [bsz, n_c_seen]
+            dist_exp = torch.exp(neg_sqr_dist)
+            dist_exp_py = p_y.mul(dist_exp)
+            dist_exp_sum = torch.sum(dist_exp_py, dim=1, keepdim=True)  # [bsz, n_c_seen] -> [bsz, 1]
+            logits = dist_exp_py / dist_exp_sum  # [bsz, n_c, seen]
+
+            if mode == 'train':
+                
+                labels_reshped = labels.view(labels.size()[0], -1)  # [bsz] -> [bsz, 1]
+                ALPHA = torch.zeros(batch_size, self.num_labels).to(device).scatter_(1, labels_reshped, self.alpha)  # margin
+                K = ALPHA + torch.ones([batch_size, self.num_labels]).to(device)
+
+                #######################################
+                dist_margin = torch.mul(neg_sqr_dist, K)
+                dist_margin_exp = torch.exp(dist_margin)
+                dist_margin_exp_py = p_y.mul(dist_margin_exp)
+                dist_exp_sum_margin = torch.sum(dist_margin_exp_py, dim=1, keepdim=True)
+                likelihood = dist_margin_exp_py / dist_exp_sum_margin
+                loss_ce = - likelihood.log().sum() / batch_size
+                
+                #######################################
+                means = self.means if class_emb is None else class_emb
+                means_batch = torch.index_select(means, dim=0, index=labels)
+                loss_gen = (torch.sum((pooled_output - means_batch)**2) / 2) * (1. / batch_size)
+                ########################################
+                loss = loss_ce + self.lambda_ * loss_gen
+                return loss
+
+            else:
+                return pooled_output, logits
+
+class CosNorm_Classifier(nn.Module):
+
+    def __init__(self, in_dims, out_dims, scale=64, device = None):
+
+        super(CosNorm_Classifier, self).__init__()
+        self.in_dims = in_dims
+        self.out_dims = out_dims
+        self.scale = scale
+        self.weight = Parameter(torch.Tensor(out_dims, in_dims).to(device))
+        self.reset_parameters()
+
+    def reset_parameters(self):
+        stdv = 1. / math.sqrt(self.weight.size(1))
+        self.weight.data.uniform_(-stdv, stdv)
+
+    def forward(self, input, *args):
+        norm_x = torch.norm(input, 2, 1, keepdim=True)
+        ex = (norm_x / (1 + norm_x)) * (input / norm_x)
+        ew = self.weight / torch.norm(self.weight, 2, 1, keepdim=True)
+        return torch.mm(self.scale * ex, ew.t())
+
+class BERT_Disaware(BertPreTrainedModel):
+
+    def __init__(self, config, args):
+
+        super(BERT_Disaware, self).__init__(config)
+        self.num_labels = args.num_labels
+        self.bert = BertModel(config)
+
+        self.dense = nn.Linear(config.hidden_size, config.hidden_size)
+        self.activation = nn.ReLU()
+        self.dropout = nn.Dropout(config.hidden_dropout_prob)
+        self.init_weights()
+
+        self.cosnorm_classifier = CosNorm_Classifier(
+            config.hidden_size, args.num_labels, args.scale, args.device)
+
+    def forward(self, input_ids=None, token_type_ids=None, attention_mask=None, labels=None,
+                feature_ext=False, mode=None, loss_fct=None, centroids=None, dist_infos = None):
+
+        outputs = self.bert(
+            input_ids, token_type_ids, attention_mask,  output_hidden_states=True)
+        encoded_layer_12 = outputs.hidden_states
+        pooled_output = self.dense(encoded_layer_12[-1].mean(dim=1))
+        pooled_output = self.activation(pooled_output)
+        pooled_output = self.dropout(pooled_output)
+        x = pooled_output
+
+        if feature_ext:
+            return pooled_output
+
+        else:
+
+            feat_size = x.shape[1]
+            batch_size = x.shape[0]
+
+            f_expand = x.unsqueeze(1).expand(-1, self.num_labels, -1)
+            centroids_expand = centroids.unsqueeze(0).expand(batch_size, -1, -1)        
+            dist_cur = torch.norm(f_expand - centroids_expand, 2, 2)
+            values_nn, labels_nn = torch.sort(dist_cur, 1)        
+
+            nearest_centers = centroids[labels_nn[:, 0]]
+            dist_denominator = torch.norm(x - nearest_centers, 2, 1)
+            second_nearest_centers = centroids[labels_nn[:, 1]]
+            dist_numerator = torch.norm(x - second_nearest_centers, 2, 1)
+            
+            dist_info = dist_numerator - dist_denominator
+            dist_info = torch.exp(dist_info)
+            scalar = dist_info
+
+            reachability = scalar.unsqueeze(1).expand(-1, feat_size)
+            x = reachability * pooled_output
+
+            logits = self.cosnorm_classifier(x)
+
+            if mode == 'train':
+                loss = loss_fct(logits, labels)
+                return loss
+
+            elif mode == 'eval':
+                return pooled_output, logits
+
+class BERT_MDF_Pretrain(nn.Module):
+    
+    def __init__(self, args):
+
+        super(BERT_MDF_Pretrain, self).__init__()
+        self.num_labels = args.num_labels
+        self.bert = BertForMaskedLM.from_pretrained(args.pretrained_bert_model)
+        self.dropout = nn.Dropout(0.1) #0.1
+        self.classifier = nn.Linear(args.feat_dim, args.num_labels)  
+        
+    
+    def forward(self, X):
+
+        outputs = self.bert(**X,  output_hidden_states=True)
+        
+        CLSEmbedding = outputs.hidden_states[-1][:,0]
+        CLSEmbedding = self.dropout(CLSEmbedding)
+        logits = self.classifier(CLSEmbedding)
+        output_dir = {"logits": logits}
+        output_dir["hidden_states"] = outputs.hidden_states[-1][:, 0]
+        
+        return output_dir
+    
+    def mlmForward(self, X, Y = None):
+        outputs = self.bert(**X,  labels = Y)
+        return outputs.loss
+        
+    def loss_ce(self, logits, Y):
+        loss = nn.CrossEntropyLoss()
+        output = loss(logits, Y)
+        return output
+
+
+
+class BERT_MDF(BertPreTrainedModel):
+    def __init__(self, config, args):
+        super(BERT_MDF, self).__init__(config)
+        self.num_labels = args.num_labels
+        self.bert = BertModel(config)
+        self.dropout = nn.Dropout(0.1) #0.1
+        self.classifier = nn.Linear(args.feat_dim, 2)  
+        self.init_weights() 
+
+    def forward(
+        self,
+        input_ids=None,
+        attention_mask=None,
+        token_type_ids=None,
+        position_ids=None,
+        head_mask=None,
+        inputs_embeds=None,
+        labels=None,
+    ):
+
+        outputs = self.bert(
+            input_ids=input_ids,
+            attention_mask=attention_mask,
+            token_type_ids=token_type_ids,
+            position_ids=position_ids,
+            head_mask=head_mask,
+            output_hidden_states=True
+        )
+        # Complains if input_embeds is kept
+
+        pooled_output = outputs[1]
+        
+        pooled_output = self.dropout(pooled_output)
+        logits = self.classifier(pooled_output)
+
+        outputs = (logits,) + outputs[
+            2:
+        ]  # add hidden states and attention if they are here
+
+        return outputs  # (loss), logits, (hidden_states), (attentions)
+
+
+class BertClassificationHead(nn.Module):
+    def __init__(self, config):
+        super(BertClassificationHead, self).__init__()
+        self.dense = nn.Linear(config.hidden_size, config.hidden_size)
+        self.dropout = nn.Dropout(config.hidden_dropout_prob)
+        self.out_proj = nn.Linear(config.hidden_size, config.num_labels-1)
+
+    def forward(self, feature):
+        x = self.dropout(feature)
+        x = self.dense(x)
+        x = torch.tanh(x)
+        x = self.dropout(x)
+        x = self.out_proj(x)
+        return x
+
+class BertContrastiveHead(nn.Module):
+    def __init__(self, config):
+        super(BertContrastiveHead, self).__init__()
+        self.dense = nn.Linear(config.hidden_size, config.hidden_size)
+        self.dropout = nn.Dropout(config.hidden_dropout_prob)
+        self.out_proj = nn.Linear(config.hidden_size, config.hidden_size)
+
+    def forward(self, feature):
+        x = self.dropout(feature)
+        x = self.dense(x)
+        x = torch.tanh(x)
+        x = self.dropout(x)
+        x = self.out_proj(x)
+        return x
+
+
+class BERT_KNNCL(nn.Module):
+
+    def __init__(self, args):
+        super(BERT_KNNCL, self).__init__()
+
+        self.number_labels = args.anum_labels
+
+        config = AutoConfig.from_pretrained(
+        args.bert_model ,
+        num_labels=self.number_labels,
+        )
+        
+        self.encoder_q = BertModel.from_pretrained(args.bert_model, config=config)
+        self.encoder_k = BertModel.from_pretrained(args.bert_model, config=config)
+
+        self.classifier_liner = BertClassificationHead(config)
+
+        self.contrastive_liner_q = BertContrastiveHead(config)
+        self.contrastive_liner_k = BertContrastiveHead(config)
+
+        self.m = 0.999
+        self.T = args.temperature
+        self.init_weights()  # Exec
+        self.contrastive_rate_in_training = args.contrastive_rate_in_training
+
+        # create the label_queue and feature_queue
+        self.K = args.queue_size  # 7500
+
+        self.register_buffer("label_queue", torch.randint(0, self.number_labels, [self.K]))  # Tensor:(7500,)
+        self.register_buffer("feature_queue", torch.randn(self.K, config.hidden_size))  # Tensor:(7500, 768)
+        self.feature_queue = torch.nn.functional.normalize(self.feature_queue, dim=0)
+
+        self.register_buffer("queue_ptr", torch.zeros(1, dtype=torch.long))  # Tensor(1,)
+        self.top_k = args.top_k  # 25
+        self.update_num = args.positive_num  # 3
+
+        # optional and delete can improve the performance indicated
+        # by some experiment
+        # params_to_train = ["layer." + str(i) for i in range(0, 12)]
+        # for name, param in self.encoder_q.named_parameters():
+        #     param.requires_grad_(False)
+        #     for term in params_to_train:
+        #         if term in name:
+        #             param.requires_grad_(True)
+
+    def _dequeue_and_enqueue(self, keys, label):
+        batch_size = keys.shape[0]
+
+        ptr = int(self.queue_ptr)
+
+        if ptr + batch_size > self.K:
+            batch_size = self.K - ptr
+            keys = keys[: batch_size]
+            label = label[: batch_size]
+
+        # replace the keys at ptr (dequeue ans enqueue)
+        self.feature_queue[ptr: ptr + batch_size, :] = keys
+        self.label_queue[ptr: ptr + batch_size] = label
+
+        ptr = (ptr + batch_size) % self.K
+
+        self.queue_ptr[0] = ptr
+
+    def select_pos_neg_sample(self, liner_q, label_q):
+        label_queue = self.label_queue.clone().detach()  # K
+        feature_queue = self.feature_queue.clone().detach()  # K * hidden_size
+
+        # 1. expand label_queue and feature_queue to batch_size * K
+        batch_size = label_q.shape[0]
+        tmp_label_queue = label_queue.repeat([batch_size, 1])
+        tmp_feature_queue = feature_queue.unsqueeze(0)
+        tmp_feature_queue = tmp_feature_queue.repeat([batch_size, 1, 1])  # batch_size * K * hidden_size
+
+        # 2.caluate sim
+        cos_sim = torch.einsum('nc,nkc->nk', [liner_q, tmp_feature_queue])
+
+        # 3. get index of postive and neigative 
+        tmp_label = label_q.unsqueeze(1)
+        tmp_label = tmp_label.repeat([1, self.K])
+
+        pos_mask_index = torch.eq(tmp_label_queue, tmp_label)
+        neg_mask_index = ~ pos_mask_index
+
+        # 4.another option 
+        feature_value = cos_sim.masked_select(neg_mask_index)
+        neg_sample = torch.full_like(cos_sim, -np.inf).cuda()
+        neg_sample = neg_sample.masked_scatter(neg_mask_index, feature_value)
+
+        # 5.topk
+        pos_mask_index = pos_mask_index.int()
+        pos_number = pos_mask_index.sum(dim=-1)
+        pos_min = pos_number.min()
+        if pos_min == 0:
+            return None
+        pos_sample, _ = cos_sim.topk(pos_min, dim=-1)
+        pos_sample_top_k = pos_sample[:, 0:self.top_k]  # self.topk = 25
+        pos_sample = pos_sample_top_k
+        pos_sample = pos_sample.contiguous().view([-1, 1])
+
+        neg_mask_index = neg_mask_index.int()
+        neg_number = neg_mask_index.sum(dim=-1)
+        neg_min = neg_number.min()
+        if neg_min == 0:
+            return None
+        neg_sample, _ = neg_sample.topk(neg_min, dim=-1)
+        neg_topk = min(pos_min, self.top_k)
+        neg_sample = neg_sample.repeat([1, neg_topk])
+        neg_sample = neg_sample.view([-1, neg_min])
+        logits_con = torch.cat([pos_sample, neg_sample], dim=-1)
+        logits_con /= self.T
+        return logits_con
+
+    def init_weights(self):
+        for param_q, param_k in zip(self.contrastive_liner_q.parameters(), self.contrastive_liner_k.parameters()):
+            param_k.data = param_q.data
+
+    def update_encoder_k(self):
+        for param_q, param_k in zip(self.encoder_q.parameters(), self.encoder_k.parameters()):
+            param_k.data = param_k.data * self.m + param_q.data * (1. - self.m)
+        for param_q, param_k in zip(self.contrastive_liner_q.parameters(), self.contrastive_liner_k.parameters()):
+            param_k.data = param_k.data * self.m + param_q.data * (1. - self.m)
+
+    def reshape_dict(self, batch):
+        for k, v in batch.items():
+            shape = v.shape
+            batch[k] = v.view([-1, shape[-1]])
+        return batch
+
+    def l2norm(self, x: torch.Tensor):
+        norm = torch.pow(x, 2).sum(dim=-1, keepdim=True).sqrt()
+        x = torch.div(x, norm)
+        return x
+
+    def forward_no_multi_v2(self,
+                            query,
+                            positive_sample=None,
+                            negative_sample=None,
+                            ):
+        labels = query["labels"]
+        labels = labels.view(-1)
+
+        with torch.no_grad():
+            self.update_encoder_k()
+            update_sample = self.reshape_dict(positive_sample)
+            bert_output_p = self.encoder_k(**update_sample)
+            update_keys = bert_output_p[1]
+            update_keys = self.contrastive_liner_k(update_keys)
+            update_keys = self.l2norm(update_keys)
+            tmp_labels = labels.unsqueeze(-1)
+            tmp_labels = tmp_labels.repeat([1, self.update_num])
+            tmp_labels = tmp_labels.view(-1)
+            self._dequeue_and_enqueue(update_keys, tmp_labels)
+
+        query.pop('labels')
+
+        bert_output_q = self.encoder_q(**query)
+        q = bert_output_q[1]
+        liner_q = self.contrastive_liner_q(q)
+        liner_q = self.l2norm(liner_q)
+        logits_cls = self.classifier_liner(q)
+
+        if self.number_labels == 1:
+            loss_fct = MSELoss()
+            loss_cls = loss_fct(logits_cls.view(-1), labels)
+        else:
+            loss_fct = CrossEntropyLoss()
+            loss_cls = loss_fct(logits_cls.view(-1, self.number_labels - 1), labels)
+
+        logits_con = self.select_pos_neg_sample(liner_q, labels)
+
+        if logits_con is not None:
+            labels_con = torch.zeros(logits_con.shape[0], dtype=torch.long).cuda()
+            loss_fct = CrossEntropyLoss()
+            loss_con = loss_fct(logits_con, labels_con)
+
+            loss = loss_con * self.contrastive_rate_in_training + \
+                   loss_cls * (1 - self.contrastive_rate_in_training)
+        else:
+            loss = loss_cls
+
+        return SequenceClassifierOutput(
+            loss=loss,
+        )
+
+    def forward(self,
+                query,  # batch_size * max_length
+                mode,
+                positive_sample=None,  # batch_size * max_length
+                negative_sample=None,  # batch_size * sample_num * max_length
+                ):
+        if mode == 'train':
+            return self.forward_no_multi_v2(query=query, positive_sample=positive_sample,
+                                            negative_sample=negative_sample)
+        elif mode == 'validation':
+            labels = query['labels']
+            query.pop('labels')
+            seq_embed = self.encoder_q(**query)[1]
+
+            logits_cls = self.classifier_liner(seq_embed)
+            probs = torch.softmax(logits_cls, dim=1)
+            return torch.argmax(probs, dim=1).tolist(), labels.cpu().numpy().tolist()
+        elif mode == 'test':
+
+            query.pop('labels')
+            seq_embed = self.encoder_q(**query)[1]
+            logits_cls = self.classifier_liner(seq_embed)
+
+            probs = torch.softmax(logits_cls, dim=1)
+            return probs, seq_embed
+        else:
+            raise ValueError("undefined mode")
+

TEXTOIR-main/open_intent_detection/backbones/llama.py

@@ -0,0 +1,100 @@
+import torch
+
+from peft import (  
+    LoraConfig,
+    get_peft_model,
+)
+
+from torch import nn
+from transformers import AutoModelForCausalLM
+
+from .bert import CosNorm_Classifier
+
+activation_map = {'relu': nn.ReLU(), 'tanh': nn.Tanh()}
+
+class LLAMA_lora_Disaware(nn.Module):
+    def __init__(self, args):
+        super().__init__()
+        self.num_labels = args.num_labels
+        self.llama = AutoModelForCausalLM.from_pretrained(
+            args.llama_model,
+            return_dict=True,
+            load_in_8bit=False,
+            device_map=args.device,
+            low_cpu_mem_usage=True,
+            torch_dtype=torch.float16,
+        )
+        self.llama.config.pad_token_id  = 0  # unk
+        self.llama.config.bos_token_id = 1
+        self.llama.config.eos_token_id = 2
+        #self.llama.eval()
+        target_modules=[ "q_proj", "v_proj"]
+        config = LoraConfig(
+                r=4,
+                lora_alpha=8,
+                target_modules=target_modules,
+                lora_dropout=0.05,
+                bias="none",
+                task_type="CAUSAL_LM",
+            )
+        print("lora", config)
+        self.llama = get_peft_model(self.llama, config)
+        hidden_dropout_prob = 0.1
+        hidden_size = self.llama.config.hidden_size
+        hidden_size_2 = hidden_size // 2
+        self.dense = nn.Linear(hidden_size, hidden_size).half()
+        self.activation = activation_map[args.activation]
+        self.dropout = nn.Dropout(hidden_dropout_prob).half()
+        self.dense = self.dense.to(args.device)
+        self.activation = self.activation.to(args.device)
+        self.dropout = self.dropout.to(args.device)
+        #self.init_weights()
+        self.cosnorm_classifier = CosNorm_Classifier(
+            hidden_size, args.num_labels, args.scale, args.device)
+
+
+    def forward(self, input_ids=None, token_type_ids=None, attention_mask=None, labels=None,
+                feature_ext=False, mode=None, loss_fct=None, centroids=None, dist_infos = None):
+        outputs = self.llama(input_ids=input_ids, attention_mask=attention_mask, output_hidden_states=True )
+        encoded_layer_ = outputs.hidden_states[-1].mean(dim=1)
+        
+        #input_data = input_data.float()
+        pooled_output = self.dense(encoded_layer_)
+        pooled_output = self.activation(pooled_output)
+        pooled_output = self.dropout(pooled_output)
+        
+        x = pooled_output
+
+        if feature_ext:
+            return pooled_output
+
+        else:
+
+            feat_size = x.shape[1]
+            batch_size = x.shape[0]
+
+            f_expand = x.unsqueeze(1).expand(-1, self.num_labels, -1)
+            centroids_expand = centroids.unsqueeze(0).expand(batch_size, -1, -1)        
+            dist_cur = torch.norm(f_expand - centroids_expand, 2, 2)
+            values_nn, labels_nn = torch.sort(dist_cur, 1)        
+
+            nearest_centers = centroids[labels_nn[:, 0]]
+            dist_denominator = torch.norm(x - nearest_centers, 2, 1)
+            second_nearest_centers = centroids[labels_nn[:, 1]]
+            dist_numerator = torch.norm(x - second_nearest_centers, 2, 1)
+            
+            dist_info = dist_numerator - dist_denominator
+            dist_info = torch.exp(dist_info)
+            scalar = dist_info
+
+            reachability = scalar.unsqueeze(1).expand(-1, feat_size)
+            x = reachability * pooled_output
+
+            logits = self.cosnorm_classifier(x)
+
+            if mode == 'train':
+                loss = loss_fct(logits, labels)
+                return loss
+
+            elif mode == 'eval':
+                return pooled_output, logits

TEXTOIR-main/open_intent_detection/backbones/utils.py

@@ -0,0 +1,90 @@
+import torch
+from torch import nn
+import numpy as np
+
+def l2_norm(input,axis=1):
+    norm = torch.norm(input, 2, axis, True)
+    output = torch.div(input, norm)
+    return output
+
+class L2_normalization(nn.Module):
+    def forward(self, input):
+        return l2_norm(input)   
+
+def freeze_bert_parameters(model):
+    for name, param in model.bert.named_parameters():  
+        param.requires_grad = False
+        if "encoder.layer.11" in name or "pooler" in name:
+            param.requires_grad = True
+    return model
+
+def freeze_bert_parameters_KCL(model):
+    for name, param in model.encoder_q.named_parameters():  
+        param.requires_grad = False
+        if "encoder.layer.11" in name or "pooler" in name:
+            param.requires_grad = True
+    for name, param in model.encoder_k.named_parameters():  
+        param.requires_grad = False
+        if "encoder.layer.11" in name or "pooler" in name:
+            param.requires_grad = True
+    return model
+
+class ConvexSampler(nn.Module):
+    def __init__(self, args):
+        super(ConvexSampler, self).__init__()
+        self.multiple_convex = args.multiple_convex
+        self.multiple_convex_eval = args.multiple_convex_eval
+        self.unseen_label_id = args.unseen_label_id
+        self.device = args.device
+        self.batch_size = args.train_batch_size
+        self.oos_num = args.train_batch_size
+        self.feat_dim = args.feat_dim
+
+    def forward(self, z, label_ids, mode=None):
+        num_convex = self.batch_size * self.multiple_convex
+        num_convex_eval = self.batch_size * self.multiple_convex_eval
+        convex_list = []
+        if mode =='train':
+            if label_ids.size(0)>2:
+                while len(convex_list) < num_convex:
+                    cdt = np.random.choice(label_ids.size(0), 2, replace=False)
+                    # cdt = np.random.choice(label_ids.size(0) - self.oos_num, 2, replace=False)
+                    if label_ids[cdt[0]] != label_ids[cdt[1]]:
+                        s = np.random.uniform(0, 1, 1)
+                        convex_list.append(s[0] * z[cdt[0]] + (1 - s[0]) * z[cdt[1]])
+                convex_samples = torch.cat(convex_list, dim=0).view(num_convex, -1)
+                z = torch.cat((z, convex_samples), dim=0)
+                label_ids = torch.cat((label_ids, torch.tensor([self.unseen_label_id] * num_convex).to(self.device)), dim=0)
+        elif mode == 'eval':
+            if label_ids.size(0) > 2:
+                val_num = num_convex_eval
+                while len(convex_list) < val_num:
+                    cdt = np.random.choice(label_ids.size(0), 2, replace=False)
+                    if label_ids[cdt[0]] != label_ids[cdt[1]]:
+                        s = np.random.uniform(0, 1, 1)
+                        convex_list.append(s[0] * z[cdt[0]] + (1 - s[0]) * z[cdt[1]])
+                convex_samples = torch.cat(convex_list, dim=0).view(val_num, -1)
+                z = torch.cat((z, convex_samples), dim=0)
+                label_ids = torch.cat((label_ids, torch.tensor([self.unseen_label_id] * val_num).to(self.device)), dim=0)
+        return z, label_ids
+
+def pair_cosine_similarity(x, x_adv, eps=1e-8):
+    n = x.norm(p=2, dim=1, keepdim=True)
+    n_adv = x_adv.norm(p=2, dim=1, keepdim=True)
+    return (x @ x.t()) / (n * n.t()).clamp(min=eps), (x_adv @ x_adv.t()) / (n_adv * n_adv.t()).clamp(min=eps), (x @ x_adv.t()) / (n * n_adv.t()).clamp(min=eps)
+
+def nt_xent(x, x_adv, mask, cuda=True, t=0.1):
+    x, x_adv, x_c = pair_cosine_similarity(x, x_adv)
+    x = torch.exp(x / t)
+    x_adv = torch.exp(x_adv / t)
+    x_c = torch.exp(x_c / t)
+    mask_count = mask.sum(1)
+    mask_reverse = (~(mask.bool())).long()
+    if cuda:
+        dis = (x * (mask - torch.eye(x.size(0)).long().cuda()) + x_c * mask) / (x.sum(1) + x_c.sum(1) - torch.exp(torch.tensor(1 / t))) + mask_reverse
+        dis_adv = (x_adv * (mask - torch.eye(x.size(0)).long().cuda()) + x_c.T * mask) / (x_adv.sum(1) + x_c.sum(0) - torch.exp(torch.tensor(1 / t))) + mask_reverse
+    else:
+        dis = (x * (mask - torch.eye(x.size(0)).long()) + x_c * mask) / (x.sum(1) + x_c.sum(1) - torch.exp(torch.tensor(1 / t))) + mask_reverse
+        dis_adv = (x_adv * (mask - torch.eye(x.size(0)).long()) + x_c.T * mask) / (x_adv.sum(1) + x_c.sum(0) - torch.exp(torch.tensor(1 / t))) + mask_reverse
+    loss = (torch.log(dis).sum(1) + torch.log(dis_adv).sum(1)) / mask_count
+    return -loss.mean()

TEXTOIR-main/open_intent_detection/configs/ADB.py

@@ -0,0 +1,43 @@
+class Param():
+    
+    def __init__(self, args):
+        
+        self.hyper_param = self.get_hyper_parameters(args)
+
+    def get_hyper_parameters(self, args):
+        """
+        Args:
+            bert_model (directory): The path for the pre-trained bert model.
+            num_train_epochs (int): The number of training epochs.
+            num_labels (autofill): The output dimension.
+            max_seq_length (autofill): The maximum total input sequence length after tokenization. Sequences longer than this will be truncated, sequences shorter will be padded.
+            freeze_backbone_parameters (binary): Whether to freeze all parameters but the last layer.
+            feat_dim (int): The feature dimension.
+            warmup_proportion (float): The warmup ratio for learning rate.
+            lr_boundary (float): The learning rate of the decision boundary.
+            lr (float): The learning rate of backbone.
+            activation (str): The activation function of the hidden layer (support 'relu' and 'tanh').
+            train_batch_size (int): The batch size for training.
+            eval_batch_size (int): The batch size for evaluation. 
+            test_batch_size (int): The batch size for testing.
+            wait_patient (int): Patient steps for Early Stop.
+        """
+        hyper_parameters = {
+
+            'bert_model': "/home/sharing/disk1/pretrained_embedding/bert/uncased_L-12_H-768_A-12/",
+            'num_train_epochs': 100,
+            'num_labels': None,
+            'max_seq_length': None, 
+            'freeze_backbone_parameters': True,
+            'feat_dim': 768,
+            'warmup_proportion': 0.1,
+            'lr_boundary': 0.05,
+            'lr': 2e-5, 
+            'activation': 'relu',
+            'train_batch_size': 128,
+            'eval_batch_size': 64,
+            'test_batch_size': 64,
+            'wait_patient': 10
+
+        }
+        return hyper_parameters

TEXTOIR-main/open_intent_detection/configs/ARPL.py

@@ -0,0 +1,46 @@
+class Param():
+    
+    def __init__(self, args):
+        
+        self.hyper_param = self.get_hyper_parameters(args)
+
+    def get_hyper_parameters(self, args):
+        """
+        Args:
+            bert_model (directory): The path for the pre-trained bert model.
+            num_train_epochs (int): The number of training epochs.
+            num_labels (autofill): The output dimension.
+            max_seq_length (autofill): The maximum total input sequence length after tokenization. Sequences longer than this will be truncated, sequences shorter will be padded.
+            freeze_backbone_parameters (binary): Whether to freeze all parameters but the last layer.
+            feat_dim (int): The feature dimension.
+            warmup_proportion (float): The warmup ratio for learning rate.
+            lr (float): The learning rate of backbone.
+            activation (str): The activation function of the hidden layer (support 'relu' and 'tanh').
+            threshold (float): The probability threshold for detecting the open samples.
+            train_batch_size (int): The batch size for training.
+            eval_batch_size (int): The batch size for evaluation. 
+            test_batch_size (int): The batch size for testing.
+            wait_patient (int): Patient steps for Early Stop.
+        """
+        hyper_parameters = {
+
+            'bert_model': "/home/sharing/disk1/pretrained_embedding/bert/uncased_L-12_H-768_A-12/",
+            'num_train_epochs': 100,
+            'num_labels': None,
+            'max_seq_length': None, 
+            'freeze_backbone_parameters': True,
+            'feat_dim': 768,
+            'warmup_proportion': 0.1,
+            'lr': 2e-5, 
+            'activation': 'tanh',
+            'threshold': 0.5,
+            'train_batch_size': 128,
+            'eval_batch_size': 64,
+            'test_batch_size': 64,
+            'wait_patient': 10,
+            'lr_2': 2e-5,
+            'temp': 1.0,
+            'weight_pl': 0.1
+
+        }
+        return hyper_parameters

TEXTOIR-main/open_intent_detection/configs/DA-ADB.py

@@ -0,0 +1,45 @@
+class Param():
+    
+    def __init__(self, args):
+        
+        self.hyper_param = self.get_hyper_parameters(args)
+
+    def get_hyper_parameters(self, args):
+        """
+        Args:
+            bert_model (directory): The path for the pre-trained bert model.
+            num_train_epochs (int): The number of training epochs.
+            num_labels (autofill): The output dimension.
+            max_seq_length (autofill): The maximum total input sequence length after tokenization. Sequences longer than this will be truncated, sequences shorter will be padded.
+            freeze_backbone_parameters (binary): Whether to freeze all parameters but the last layer.
+            feat_dim (int): The feature dimension.
+            warmup_proportion (float): The warmup ratio for learning rate.
+            scale (float): The scale factor of the cosine classifier.
+            lr_boundary (float): The learning rate of the decision boundary.
+            lr (float): The learning rate of backbone.
+            activation (str): The activation function of the hidden layer (support 'relu' and 'tanh').
+            train_batch_size (int): The batch size for training.
+            eval_batch_size (int): The batch size for evaluation. 
+            test_batch_size (int): The batch size for testing.
+            wait_patient (int): Patient steps for Early Stop.
+        """
+        hyper_parameters = {
+
+            'bert_model': "/home/sharing/disk1/pretrained_embedding/bert/uncased_L-12_H-768_A-12/",
+            'num_train_epochs':100,
+            'num_labels': None,
+            'max_seq_length': None, 
+            'freeze_backbone_parameters': True,
+            'feat_dim': 768,
+            'warmup_proportion': 0.1,
+            'scale': 4,
+            'lr_boundary': 0.05,
+            'lr': 2e-5, 
+            'activation': 'relu',
+            'train_batch_size': 128,
+            'eval_batch_size': 64,
+            'test_batch_size': 64,
+            'wait_patient': 10,
+
+        }
+        return hyper_parameters

TEXTOIR-main/open_intent_detection/configs/DA-ADB_llama.py

@@ -0,0 +1,47 @@
+class Param():
+    
+    def __init__(self, args):
+        
+        self.hyper_param = self.get_hyper_parameters(args)
+
+    def get_hyper_parameters(self, args):
+        """
+        Args:
+            llama_model (directory): The path for the pre-trained llama model.
+            num_train_epochs (int): The number of training epochs.
+            num_labels (autofill): The output dimension.
+            max_seq_length (autofill): The maximum total input sequence length after tokenization. Sequences longer than this will be truncated, sequences shorter will be padded.
+            freeze_backbone_parameters (binary): Whether to freeze all parameters but the last layer.
+            feat_dim (int): The feature dimension.
+            warmup_proportion (float): The warmup ratio for learning rate.
+            scale (float): The scale factor of the cosine classifier.
+            lr_boundary (float): The learning rate of the decision boundary.
+            lr (float): The learning rate of backbone.
+            activation (str): The activation function of the hidden layer (support 'relu' and 'tanh').
+            train_batch_size (int): The batch size for training.
+            eval_batch_size (int): The batch size for evaluation. 
+            test_batch_size (int): The batch size for testing.
+            wait_patient (int): Patient steps for Early Stop.
+        """
+        hyper_parameters = {
+
+            'llama_model': "/home/sharing/disk1/pretrained_embedding/llama/llama",
+            'num_train_epochs':100,
+            'num_labels': None,
+            'max_seq_length': None, 
+            'freeze_backbone_parameters': False,
+            'feat_dim': 4096,
+            'warmup_proportion': 0.1,
+            'scale': 4,
+            'lr_boundary': 0.05,
+            'lr': 5e-8, 
+            'activation': 'relu',
+            'train_batch_size': 32,
+            'eval_batch_size': 8,
+            'test_batch_size': 8,
+            'wait_patient': 10,
+
+        }
+        print("Hyper-parameters: ", hyper_parameters)
+
+        return hyper_parameters

TEXTOIR-main/open_intent_detection/configs/DOC.py

@@ -0,0 +1,43 @@
+class Param():
+
+    def __init__(self, args):
+
+        self.hyper_param = self.get_hyper_parameters(args)
+
+    def get_hyper_parameters(self, args):
+        """
+        Args:
+            bert_model (directory): The path for the pre-trained bert model.
+            num_train_epochs (int): The number of training epochs.
+            num_labels (autofill): The output dimension.
+            max_seq_length (autofill): The maximum total input sequence length after tokenization. Sequences longer than this will be truncated, sequences shorter will be padded.
+            freeze_backbone_parameters (binary): Whether to freeze all parameters but the last layer.
+            feat_dim (int): The feature dimension.
+            warmup_proportion (float): The warmup ratio for learning rate.
+            lr (float): The learning rate of backbone.
+            activation (str): The activation function of the hidden layer (support 'relu' and 'tanh').
+            scale (float): The scale factor of DOC.
+            train_batch_size (int): The batch size for training.
+            eval_batch_size (int): The batch size for evaluation. 
+            test_batch_size (int): The batch size for testing.
+            wait_patient (int): Patient steps for Early Stop.
+        """
+        hyper_parameters = {
+
+            'bert_model': "/home/sharing/disk1/pretrained_embedding/bert/uncased_L-12_H-768_A-12/",
+            'num_train_epochs': 100,
+            'num_labels': None,
+            'max_seq_length': None,
+            'freeze_backbone_parameters': True,
+            'feat_dim': 768,
+            'warmup_proportion': 0.1,
+            'lr': 2e-5,
+            'activation': 'relu',
+            'scale': 3,
+            'train_batch_size': 128,
+            'eval_batch_size': 64,
+            'test_batch_size': 64,
+            'wait_patient': 10
+        }
+
+        return hyper_parameters

TEXTOIR-main/open_intent_detection/configs/DeepUnk.py

@@ -0,0 +1,46 @@
+class Param():
+    
+    def __init__(self, args):
+        
+        self.hyper_param = self.get_hyper_parameters(args)
+
+    def get_hyper_parameters(self, args):
+        """
+        Args:
+            bert_model (directory): The path for the pre-trained bert model.
+            num_train_epochs (int): The number of training epochs.
+            num_labels (autofill): The output dimension.
+            max_seq_length (autofill): The maximum total input sequence length after tokenization. Sequences longer than this will be truncated, sequences shorter will be padded.
+            freeze_backbone_parameters (binary): Whether to freeze all parameters but the last layer.
+            feat_dim (int): The feature dimension.
+            warmup_proportion (float): The warmup ratio for learning rate.
+            lr (float): The learning rate of backbone.
+            activation (str): The activation function of the hidden layer (support 'relu' and 'tanh').
+            n_neighbors (int): The number of neighbors of LOF.
+            contamination (float): The contamination factor of LOF.
+            train_batch_size (int): The batch size for training.
+            eval_batch_size (int): The batch size for evaluation. 
+            test_batch_size (int): The batch size for testing.
+            wait_patient (int): Patient steps for Early Stop.
+        """
+        hyper_parameters = {
+
+            'bert_model': "/home/sharing/disk1/pretrained_embedding/bert/uncased_L-12_H-768_A-12/",
+            'num_train_epochs': 100,
+            'num_labels': None,
+            'max_seq_length': None, 
+            'freeze_backbone_parameters': True,
+            'feat_dim': 768,
+            'warmup_proportion': 0.1,
+            'lr': 2e-5, 
+            'activation': 'relu',
+            'n_neighbors': 20,
+            'contamination': 0.05,
+            'train_batch_size': 128,
+            'eval_batch_size': 64,
+            'test_batch_size': 64,
+            'wait_patient': 10
+
+        }
+
+        return hyper_parameters

TEXTOIR-main/open_intent_detection/configs/K+1-way.py

@@ -0,0 +1,44 @@
+class Param():
+    
+    def __init__(self, args):
+        
+        self.hyper_param = self.get_hyper_parameters(args)
+
+    def get_hyper_parameters(self, args):
+        """
+        Args:
+            bert_model (directory): The path for the pre-trained bert model.
+            num_train_epochs: The training epochs.
+            max_seq_len (int): The maximum total input sequence length after tokenization. Sequences longer than this will be truncated, sequences shorter will be padded.
+            feat_dim (int): The feature dimension.
+            warmup_proportion (float): The warmup ratio for learning rate.
+            lr (float): The learning rate of backbone.
+            threshold (float): The probability threshold for detecting the open samples.
+            train_batch_size (int): The batch size for training.
+            eval_batch_size (int): The batch size for evaluation. 
+            wait_patient (int): Patient steps for Early Stop.
+            lr_mlp (float): The learning rate of mlp.
+            multiple_convex (int): The multiple of gen ood of training stage.
+            multiple_convex_eval (int): The multiple of gen ood of eval stage.
+        """
+        hyper_parameters = {
+
+            'bert_model': "/home/sharing/disk1/pretrained_embedding/bert/uncased_L-12_H-768_A-12/",
+            'num_train_epochs': 100,
+            'max_seq_length': None, 
+            'freeze_backbone_parameters': True,
+            'feat_dim': 768,
+            'warmup_proportion': 0.1,
+            'lr': 2e-5, 
+            'activation': 'relu',
+            'train_batch_size': 64,
+            'eval_batch_size': 64,
+            'test_batch_size': 64,
+            'wait_patient': 10,
+            'multiple_convex': 1,
+            'multiple_convex_eval': 1,
+            'temp': 0.1
+
+        }
+
+        return hyper_parameters

TEXTOIR-main/open_intent_detection/configs/KNNCL.py

@@ -0,0 +1,50 @@
+class Param():
+    
+    def __init__(self, args):
+        
+        self.hyper_param = self.get_hyper_parameters(args)
+
+    def get_hyper_parameters(self, args):
+
+        hyper_parameters = {
+
+            'bert_model': "/home/sharing/disk1/pretrained_embedding/bert/uncased_L-12_H-768_A-12/",
+            'num_train_epochs': 100,
+            'num_labels': None,
+            'max_seq_length': None, 
+            'freeze_backbone_parameters': True,
+            'feat_dim': 768,
+            'warmup_proportion': 0.1,
+            'lr': 1e-05, 
+            'activation': 'relu',
+            'train_batch_size': 128,
+            'eval_batch_size': 64,
+            'test_batch_size': 64,
+            'wait_patient': 100,
+            'warmup_proportion':0.1,
+
+            'n_neighbors': 20,
+            'contamination': 0.05,
+
+            "temperature": 0.5,
+            "contrastive_rate_in_training": 0.1,
+            "top_k": 25,
+            "positive_num":3,
+            "clip": 0.25,
+            "queue_size": 7500,
+
+        }
+
+        if args.dataset == "banking":
+            if args.known_cls_ratio == 0.25:
+                hyper_parameters["queue_size"]=6500
+            elif args.known_cls_ratio == 0.5:
+                hyper_parameters["queue_size"]=8000
+                hyper_parameters["top_k"]=30
+
+        if args.dataset == "oos":
+            hyper_parameters["queue_size"]=6500
+            hyper_parameters["top_k"]=15
+
+
+        return hyper_parameters

TEXTOIR-main/open_intent_detection/configs/LOF.py

@@ -0,0 +1,46 @@
+class Param():
+    
+    def __init__(self, args):
+        
+        self.hyper_param = self.get_hyper_parameters(args)
+
+    def get_hyper_parameters(self, args):
+        """
+        Args:
+            bert_model (directory): The path for the pre-trained bert model.
+            num_train_epochs (int): The number of training epochs.
+            num_labels (autofill): The output dimension.
+            max_seq_length (autofill): The maximum total input sequence length after tokenization. Sequences longer than this will be truncated, sequences shorter will be padded.
+            freeze_backbone_parameters (binary): Whether to freeze all parameters but the last layer.
+            feat_dim (int): The feature dimension.
+            warmup_proportion (float): The warmup ratio for learning rate.
+            lr (float): The learning rate of backbone.
+            activation (str): The activation function of the hidden layer (support 'relu' and 'tanh').
+            n_neighbors (int): The number of neighbors of LOF.
+            contamination (float): The contamination factor of LOF.
+            train_batch_size (int): The batch size for training.
+            eval_batch_size (int): The batch size for evaluation. 
+            test_batch_size (int): The batch size for testing.
+            wait_patient (int): Patient steps for Early Stop.
+        """
+        hyper_parameters = {
+
+            'bert_model': "/home/sharing/disk1/pretrained_embedding/bert/uncased_L-12_H-768_A-12/",
+            'num_train_epochs': 100,
+            'num_labels': None,
+            'max_seq_length': None, 
+            'freeze_backbone_parameters': True,
+            'feat_dim': 768,
+            'warmup_proportion': 0.1,
+            'lr': 2e-5, 
+            'activation': 'relu',
+            'n_neighbors': 20,
+            'contamination': 0.05,
+            'train_batch_size': 128,
+            'eval_batch_size': 64,
+            'test_batch_size': 64,
+            'wait_patient': 10
+
+        }
+
+        return hyper_parameters

TEXTOIR-main/open_intent_detection/configs/MDF.py

@@ -0,0 +1,52 @@
+class Param():
+    
+    def __init__(self, args):
+        
+        self.hyper_param = self.get_hyper_parameters(args)
+
+    def get_hyper_parameters(self, args):
+        """
+        Args:
+            bert_model (directory): The path for the pre-trained bert model.
+            max_seq_length (autofill): The maximum total input sequence length after tokenization. Sequences longer than this will be truncated, sequences shorter will be padded.
+            num_train_epochs (int): The number of training epochs.
+            num_pretrain_epochs (int): The number of pre-training epochs.
+            num_labels (autofill): The output dimension.
+            freeze_bert_parameters (binary): Whether to freeze all parameters but the last layer.
+            feat_dim (int): The feature dimension.
+            warmup_proportion (float): The warmup ratio for learning rate.
+            lr_pre (float): The learning rate for pre-training the backbone.
+            lr (float): The learning rate for training the backbone.
+            loss_fct (str): The loss function for training.
+            activation (str): The activation function of the hidden layer (support 'relu' and 'tanh').
+            train_batch_size (int): The batch size for training.
+            eval_batch_size (int): The batch size for evaluation.
+            test_batch_size (int): The batch size for testing. 
+            wait_patient (int): Patient steps for Early Stop.
+        """
+        hyper_parameters = {
+            
+            'pretrained_bert_model': '/home/sharing/disk1/pretrained_embedding/bert/uncased_L-12_H-768_A-12/',
+            'max_seq_length': None, 
+            'num_pretrain_epochs': 100,
+            'num_train_epochs': 100,
+            'update_per_epoch' : 5,
+            'num_labels': None,
+            'freeze_backbone_parameters': False,
+            'feat_dim': 768,
+            'warmup_proportion': 0.1,
+            'lr_pre': 5e-5,
+            'lr': 1e-5, 
+            'activation': 'tanh',
+            'train_batch_size': 128,
+            'pretrain_batch_size': 64,
+            'eval_batch_size': 64,
+            'test_batch_size': 64,
+            'wait_patient': 10,
+            'grad_clip': 1,
+            'nuu': 0.1,
+            'k': 'rbf',
+            'use_cls': False
+        }
+
+        return hyper_parameters

TEXTOIR-main/open_intent_detection/configs/MSP.py

@@ -0,0 +1,47 @@
+class Param():
+    
+    def __init__(self, args):
+        
+        self.hyper_param = self.get_hyper_parameters(args)
+
+    def get_hyper_parameters(self, args):
+        """
+        Args:
+            bert_model (directory): The path for the pre-trained bert model.
+            num_train_epochs (int): The number of training epochs.
+            num_labels (autofill): The output dimension.
+            max_seq_length (autofill): The maximum total input sequence length after tokenization. Sequences longer than this will be truncated, sequences shorter will be padded.
+            freeze_backbone_parameters (binary): Whether to freeze all parameters but the last layer.
+            feat_dim (int): The feature dimension.
+            warmup_proportion (float): The warmup ratio for learning rate.
+            lr (float): The learning rate of backbone.
+            activation (str): The activation function of the hidden layer (support 'relu' and 'tanh').
+            threshold (float): The probability threshold for detecting the open samples.
+            train_batch_size (int): The batch size for training.
+            eval_batch_size (int): The batch size for evaluation. 
+            test_batch_size (int): The batch size for testing.
+            wait_patient (int): Patient steps for Early Stop.
+        """
+        hyper_parameters = {
+
+            'bert_model': "/home/sharing/disk1/pretrained_embedding/bert/uncased_L-12_H-768_A-12/",
+            'num_train_epochs': 100,
+            'num_labels': None,
+            'max_seq_length': None, 
+            'freeze_backbone_parameters': True,
+            'feat_dim': 768,
+            'warmup_proportion': 0.1,
+            'lr': 2e-5, 
+            'activation': 'tanh',
+            'threshold': 0.5,
+            'train_batch_size': 128,
+            'eval_batch_size': 64,
+            'test_batch_size': 64,
+            'wait_patient': 15
+
+        }
+        if args.dataset == 'stackoverflow':
+            hyper_parameters['lr'] = 5e-5
+            hyper_parameters['wait_patient'] = 20
+
+        return hyper_parameters

TEXTOIR-main/open_intent_detection/configs/OpenMax.py

@@ -0,0 +1,50 @@
+class Param():
+    
+    def __init__(self, args):
+        
+        self.hyper_param = self.get_hyper_parameters(args)
+
+    def get_hyper_parameters(self, args):
+        """
+        Args:
+            bert_model (directory): The path for the pre-trained bert model.
+            num_train_epochs (int): The number of training epochs.
+            num_labels (autofill): The output dimension.
+            max_seq_length (autofill): The maximum total input sequence length after tokenization. Sequences longer than this will be truncated, sequences shorter will be padded.
+            freeze_backbone_parameters (binary): Whether to freeze all parameters but the last layer.
+            feat_dim (int): The feature dimension.
+            warmup_proportion (float): The warmup ratio for learning rate.
+            lr (float): The learning rate of backbone.
+            activation (str): The activation function of the hidden layer (support 'relu' and 'tanh').
+            weibull_tail_size (int): The factor of weibull model.
+            alpharank (int): The factor of alpha rank.
+            distance_type (str): The distance_type.
+            threshold (float): The probability threshold for detecting the open samples.
+            train_batch_size (int): The batch size for training.
+            eval_batch_size (int): The batch size for evaluation. 
+            test_batch_size (int): The batch size for testing.
+            wait_patient (int): Patient steps for Early Stop.
+        """
+        hyper_parameters = {
+
+            'bert_model': "/home/sharing/disk1/pretrained_embedding/bert/uncased_L-12_H-768_A-12/",
+            'num_train_epochs': 100,
+            'num_labels': None,
+            'max_seq_length': None, 
+            'freeze_backbone_parameters': True,
+            'feat_dim': 768,
+            'warmup_proportion': 0.1,
+            'lr': 2e-5, 
+            'activation': 'tanh',
+            'weibull_tail_size': 20,
+            'alpharank': 10,
+            'distance_type': 'cosine',
+            'threshold': 0.5, 
+            'train_batch_size': 128,
+            'eval_batch_size': 64,
+            'test_batch_size': 64,
+            'wait_patient': 15
+
+        }
+
+        return hyper_parameters

TEXTOIR-main/open_intent_detection/configs/SEG.py

@@ -0,0 +1,47 @@
+class Param():
+    
+    def __init__(self, args):
+        
+        self.hyper_param = self.get_hyper_parameters(args)
+
+    def get_hyper_parameters(self, args):
+        """
+        Args:
+            bert_model (directory): The path for the pre-trained bert model.
+            num_train_epochs: The training epochs.
+            max_seq_len (int): The maximum total input sequence length after tokenization. Sequences longer than this will be truncated, sequences shorter will be padded.
+            feat_dim (int): The feature dimension.
+            warmup_proportion (float): The warmup ratio for learning rate.
+            lr (float): The learning rate of backbone.
+            loss_fct (str): The loss function for training.
+            n_neighbors (int): The number of neighbors of LOF.
+            contamination (float): The contamination factor of LOF.
+            train_batch_size (int): The batch size for training.
+            eval_batch_size (int): The batch size for evaluation. 
+            wait_patient (int): Patient steps for Early Stop.
+        """
+        hyper_parameters = {
+
+            'bert_model': "/home/sharing/disk1/pretrained_embedding/bert/uncased_L-12_H-768_A-12/",
+            'num_train_epochs': 100,
+            'max_seq_length': None, 
+            'freeze_backbone_parameters': True,
+            'feat_dim': 768,
+            'warmup_proportion': 0.1,
+            'lr': 2e-5, 
+            'activation': 'relu',
+            'n_neighbors': 20,
+            'contamination': 0.05,
+            'train_batch_size': 128,
+            'eval_batch_size': 64,
+            'test_batch_size': 64,
+            'wait_patient': 10,
+            'class_emb': True,
+            'alpha': 1.0,
+            'lambda_': 0.5
+
+        }
+        # if args.dataset == 'stackoverflow':
+        #     hyper_parameters['class_emb'] = False
+
+        return hyper_parameters

TEXTOIR-main/open_intent_detection/configs/base.py

@@ -0,0 +1,60 @@
+import argparse
+import sys
+import os
+import importlib
+from easydict import EasyDict
+
+class ParamManager:
+    
+    def __init__(self, args):
+
+        output_path_param = self.add_output_path_param(args)
+
+        method_param = self.get_method_param(args)
+
+        self.args = EasyDict(
+                                dict(
+                                        vars(args),
+                                        **output_path_param,
+                                        **method_param
+                                     )
+                            )
+
+    def get_method_param(self, args):
+        
+        if args.config_file_name.endswith('.py'):
+            module_name = '.' + args.config_file_name[:-3]
+        else:
+            module_name = '.' + args.config_file_name
+
+        config = importlib.import_module(module_name, 'configs')
+
+        method_param = config.Param
+        method_args = method_param(args)
+
+        return method_args.hyper_param
+
+    def add_output_path_param(self, args):
+        
+        
+        task_output_dir = os.path.join(args.output_dir, args.type)
+        if not os.path.exists(task_output_dir):
+            os.makedirs(task_output_dir)
+        
+        concat_names = [args.method, args.dataset, args.known_cls_ratio, args.labeled_ratio, args.backbone, args.seed]
+        method_output_name = "_".join([str(x) for x in concat_names])
+
+        method_output_dir = os.path.join(task_output_dir, method_output_name)
+        if not os.path.exists(method_output_dir):
+            os.makedirs(method_output_dir)
+
+        model_output_dir = os.path.join(method_output_dir, args.model_dir)
+        if not os.path.exists(model_output_dir):
+            os.makedirs(model_output_dir)
+
+        output_path_param = {
+            'method_output_dir': method_output_dir,
+            'model_output_dir': model_output_dir
+        }
+            
+        return output_path_param

TEXTOIR-main/open_intent_detection/dataloaders/__init__.py

@@ -0,0 +1,89 @@
+from .bert_loader import BERT_Loader
+from .llama_loader import LLAMA_Loader
+
+max_seq_lengths = {
+                        'stackoverflow':45,
+                        'banking':55, 
+                        'oos':30, 
+                        'snips':35
+                    }
+
+
+backbone_loader_map = {
+                            'bert': BERT_Loader,
+                            'bert_doc': BERT_Loader,
+                            'bert_norm': BERT_Loader,
+                            'bert_K+1-way': BERT_Loader,
+                            'bert_seg': BERT_Loader,
+                            'bert_disaware': BERT_Loader,
+                            'bert_mdf': BERT_Loader,
+                            'bert_knncl': BERT_Loader,
+                            'llama_disaware': LLAMA_Loader,
+                        }
+
+benchmark_labels = {
+    'oos': 
+            ['accept_reservations', 'account_blocked', 'alarm', 'application_status', 'apr', 
+             'are_you_a_bot', 'balance', 'bill_balance', 'bill_due', 'book_flight', 
+             'book_hotel', 'calculator', 'calendar', 'calendar_update', 'calories', 
+             'cancel', 'cancel_reservation', 'car_rental', 'card_declined', 'carry_on', 
+             'change_accent', 'change_ai_name', 'change_language', 'change_speed', 'change_user_name',
+             'change_volume', 'confirm_reservation', 'cook_time', 'credit_limit', 'credit_limit_change', 
+             'credit_score', 'current_location', 'damaged_card', 'date', 'definition', 
+             'direct_deposit', 'directions', 'distance', 'do_you_have_pets', 'exchange_rate',
+             'expiration_date', 'find_phone', 'flight_status', 'flip_coin', 'food_last', 
+             'freeze_account', 'fun_fact', 'gas', 'gas_type', 'goodbye', 
+             'greeting', 'how_busy', 'how_old_are_you', 'improve_credit_score', 'income', 
+             'ingredient_substitution', 'ingredients_list', 'insurance', 'insurance_change', 'interest_rate',
+             'international_fees', 'international_visa', 'jump_start', 'last_maintenance', 'lost_luggage', 'make_call', 
+             'maybe', 'meal_suggestion', 'meaning_of_life', 'measurement_conversion', 'meeting_schedule', 
+             'min_payment', 'mpg', 'new_card', 'next_holiday', 'next_song',
+             'no', 'nutrition_info', 'oil_change_how', 'oil_change_when', 'order',
+             'order_checks', 'order_status', 'pay_bill', 'payday', 'pin_change', 
+             'play_music', 'plug_type', 'pto_balance', 'pto_request', 'pto_request_status',
+             'pto_used', 'recipe', 'redeem_rewards', 'reminder', 'reminder_update', 
+             'repeat', 'replacement_card_duration', 'report_fraud', 'report_lost_card', 'reset_settings', 
+             'restaurant_reservation', 'restaurant_reviews', 'restaurant_suggestion', 'rewards_balance', 'roll_dice', 
+             'rollover_401k', 'routing', 'schedule_maintenance', 'schedule_meeting', 'share_location', 
+             'shopping_list', 'shopping_list_update', 'smart_home', 'spelling', 'spending_history', 
+             'sync_device', 'taxes', 'tell_joke', 'text', 'thank_you', 
+             'time', 'timer', 'timezone', 'tire_change', 'tire_pressure', 
+             'todo_list', 'todo_list_update', 'traffic', "transactions", "transfer", 
+             "translate", "travel_alert", "travel_notification", "travel_suggestion", "uber", 
+             "update_playlist", "user_name", "vaccines", "w2", "weather",
+             "what_are_your_hobbies", "what_can_i_ask_you", "what_is_your_name", "what_song", "where_are_you_from", 
+             "whisper_mode", "who_do_you_work_for", "who_made_you", "yes"],
+
+    'banking':  
+                [
+                    "Refund_not_showing_up", "activate_my_card", "age_limit", "apple_pay_or_google_pay", "atm_support", 
+                    "automatic_top_up", "balance_not_updated_after_bank_transfer", "balance_not_updated_after_cheque_or_cash_deposit", "beneficiary_not_allowed", "cancel_transfer", 
+                    "card_about_to_expire", "card_acceptance", "card_arrival", "card_delivery_estimate", "card_linking", 
+                    "card_not_working", "card_payment_fee_charged", "card_payment_not_recognised", "card_payment_wrong_exchange_rate", "card_swallowed", 
+                    "cash_withdrawal_charge", "cash_withdrawal_not_recognised", "change_pin", "compromised_card", "contactless_not_working", 
+                    "country_support", "declined_card_payment", "declined_cash_withdrawal", "declined_transfer", "direct_debit_payment_not_recognised", 
+                    "disposable_card_limits", "edit_personal_details", "exchange_charge", "exchange_rate", "exchange_via_app", 
+                    "extra_charge_on_statement", "failed_transfer", "fiat_currency_support", "get_disposable_virtual_card", "get_physical_card", 
+                    "getting_spare_card", "getting_virtual_card", "lost_or_stolen_card", "lost_or_stolen_phone", "order_physical_card", 
+                    "passcode_forgotten", "pending_card_payment", "pending_cash_withdrawal", "pending_top_up", "pending_transfer", 
+                    "pin_blocked", "receiving_money", "request_refund", "reverted_card_payment?", "supported_cards_and_currencies", 
+                    "terminate_account", "top_up_by_bank_transfer_charge", "top_up_by_card_charge", "top_up_by_cash_or_cheque", "top_up_failed", 
+                    "top_up_limits", "top_up_reverted", "topping_up_by_card", "transaction_charged_twice", "transfer_fee_charged", 
+                    "transfer_into_account", "transfer_not_received_by_recipient", "transfer_timing", "unable_to_verify_identity", "verify_my_identity", 
+                    "verify_source_of_funds", "verify_top_up", "virtual_card_not_working", "visa_or_mastercard", "why_verify_identity", 
+                    "wrong_amount_of_cash_received", "wrong_exchange_rate_for_cash_withdrawal"
+                ], 
+
+    'stackoverflow':
+                        [
+                            "ajax", "apache", "bash", "cocoa", "drupal", 
+                            "excel", "haskell", "hibernate", "linq", "magento", 
+                            "matlab", "oracle", "osx", "qt", "scala", 
+                            "sharepoint", "spring", "svn", "visual-studio", "wordpress"
+                        ],
+        'snips':
+                [
+                    "AddToPlaylist", "BookRestaurant", "GetWeather", "PlayMusic", "RateBook", 
+                    "SearchCreativeWork", "SearchScreeningEvent"
+                ]
+}

TEXTOIR-main/open_intent_detection/dataloaders/base.py

@@ -0,0 +1,66 @@
+import numpy as np
+import os   
+import random
+import torch
+import logging
+
+from .__init__ import max_seq_lengths, backbone_loader_map, benchmark_labels
+
+
+def set_seed(seed):
+    random.seed(seed)
+    np.random.seed(seed)
+    torch.manual_seed(seed)
+    torch.cuda.manual_seed_all(seed)
+    torch.backends.cudnn.deterministic = True
+
+class DataManager:
+    
+    def __init__(self, args, logger_name = 'Detection'):
+        
+        self.logger = logging.getLogger(logger_name)
+
+        set_seed(args.seed)
+        args.max_seq_length = max_seq_lengths[args.dataset]
+        self.data_dir = os.path.join(args.data_dir, args.dataset)
+
+        self.all_label_list = self.get_labels(args.dataset)
+        self.n_known_cls = round(len(self.all_label_list) * args.known_cls_ratio)
+        self.known_label_list = np.random.choice(np.array(self.all_label_list), self.n_known_cls, replace=False)
+        self.known_label_list = list(self.known_label_list)
+
+        self.logger.info('The number of known intents is %s', self.n_known_cls)
+        self.logger.info('Lists of known labels are: %s', str(self.known_label_list))
+
+        args.num_labels = self.num_labels = len(self.known_label_list)
+
+        if args.dataset == 'oos':
+            self.unseen_label = 'oos'
+        else:
+            self.unseen_label = '<UNK>'
+        
+        args.unseen_label_id = self.unseen_label_id = self.num_labels
+        self.label_list = self.known_label_list + [self.unseen_label]
+
+        self.anum_labels = args.anum_labels = len(self.label_list)
+        self.dataloader = self.get_loader(args, self.get_attrs())
+
+    def get_labels(self, dataset):
+        
+        labels = benchmark_labels[dataset]
+
+        return labels
+    
+    def get_loader(self, args, attrs):
+        
+        dataloader = backbone_loader_map[args.backbone](args, attrs, args.logger_name)
+
+        return dataloader
+    
+    def get_attrs(self):
+
+        attrs = {}
+        for name, value in vars(self).items():
+            attrs[name] = value
+
+        return attrs

TEXTOIR-main/open_intent_detection/dataloaders/bert_loader.py

@@ -0,0 +1,285 @@
+import numpy as np
+import torch
+import os
+import csv
+import sys
+import logging
+from transformers import BertTokenizer
+from torch.utils.data import (DataLoader, RandomSampler, SequentialSampler, TensorDataset)
+
+
+class BERT_Loader:
+    def __init__(self, args, base_attrs, logger_name = 'Detection'):
+        self.logger = logging.getLogger(logger_name)
+        self.train_examples, self.train_labeled_examples, self.train_unlabeled_examples  = get_examples(args, base_attrs, 'train')
+        self.logger.info("Number of labeled training samples = %s", str(len(self.train_labeled_examples)))
+        self.logger.info("Number of unlabeled training samples = %s", str(len(self.train_unlabeled_examples)))
+
+        self.eval_examples = get_examples(args, base_attrs, 'eval')
+        self.logger.info("Number of evaluation samples = %s", str(len(self.eval_examples)))
+
+        self.test_examples = get_examples(args, base_attrs, 'test')
+        self.logger.info("Number of testing samples = %s", str(len(self.test_examples)))
+        self.base_attrs = base_attrs
+        self.init_loader(args)
+
+    def init_loader(self, args):
+        
+        self.train_labeled_loader = get_loader(self.train_labeled_examples, args, self.base_attrs['label_list'], 'train_labeled', sampler_mode = 'random')
+        self.train_unlabeled_loader = get_loader(self.train_unlabeled_examples, args, self.base_attrs['label_list'], 'train_unlabeled', sampler_mode = 'sequential')
+        self.eval_loader = get_loader(self.eval_examples, args, self.base_attrs['label_list'], 'eval', sampler_mode = 'sequential')
+        self.test_loader = get_loader(self.test_examples, args, self.base_attrs['label_list'], 'test', sampler_mode = 'sequential')
+        self.num_train_examples = len(self.train_labeled_examples)
+
+def get_examples(args, base_attrs, mode):
+
+    processor = DatasetProcessor()
+    ori_examples = processor.get_examples(base_attrs['data_dir'], mode)
+    if mode == 'train':
+
+        labeled_examples, unlabeled_examples = [], []
+        for example in ori_examples:
+
+            if (example.label in base_attrs['known_label_list']) and (np.random.uniform(0, 1) <= args.labeled_ratio):
+                labeled_examples.append(example)
+            else:
+                example.label = base_attrs['unseen_label']
+                unlabeled_examples.append(example)
+
+        return ori_examples, labeled_examples, unlabeled_examples
+
+    elif mode == 'eval':
+
+        examples = []
+        for example in ori_examples:
+            if (example.label in base_attrs['known_label_list']):
+                examples.append(example)
+        
+        return examples
+    
+    elif mode == 'test':
+
+        examples = []
+        for example in ori_examples:
+            if (example.label in base_attrs['label_list']) and (example.label != base_attrs['unseen_label']):
+                examples.append(example)
+            else:
+                example.label = base_attrs['unseen_label']
+                examples.append(example)
+        
+        return examples
+
+def get_loader(examples, args, label_list, mode, sampler_mode = 'sequential'):
+
+    tokenizer = BertTokenizer.from_pretrained(args.bert_model, do_lower_case=True)    
+    features = convert_examples_to_features(examples, label_list, args.max_seq_length, tokenizer)
+    input_ids = torch.tensor([f.input_ids for f in features], dtype=torch.long)
+    input_mask = torch.tensor([f.input_mask for f in features], dtype=torch.long)
+    segment_ids = torch.tensor([f.segment_ids for f in features], dtype=torch.long)
+
+    if mode == 'train_unlabeled':
+        label_ids = torch.tensor([-1 for f in features], dtype=torch.long)
+    else:
+        label_ids = torch.tensor([f.label_id for f in features], dtype=torch.long)
+    datatensor = TensorDataset(input_ids, input_mask, segment_ids, label_ids)
+
+    if sampler_mode == 'random':
+        sampler = RandomSampler(datatensor)
+    elif sampler_mode == 'sequential':
+        sampler = SequentialSampler(datatensor)
+
+    if mode == 'train_labeled':   
+        dataloader = DataLoader(datatensor, sampler = sampler, batch_size = args.train_batch_size)    
+
+    else:
+        if mode == 'train_unlabeled':
+            dataloader = DataLoader(datatensor, sampler=sampler, batch_size = args.train_batch_size)    
+        elif mode == 'eval':
+            dataloader = DataLoader(datatensor, sampler=sampler, batch_size = args.eval_batch_size)    
+        elif mode == 'test':
+            dataloader = DataLoader(datatensor, sampler=sampler, batch_size = args.test_batch_size)    
+    
+    return dataloader
+
+class InputExample(object):
+    """A single training/test example for simple sequence classification."""
+
+    def __init__(self, guid, text_a, text_b=None, label=None):
+        """Constructs a InputExample.
+
+        Args:
+            guid: Unique id for the example.
+            text_a: string. The untokenized text of the first sequence. For single
+            sequence tasks, only this sequence must be specified.
+            text_b: (Optional) string. The untokenized text of the second sequence.
+            Only must be specified for sequence pair tasks.
+            label: (Optional) string. The label of the example. This should be
+            specified for train and dev examples, but not for test examples.
+        """
+        self.guid = guid
+        self.text_a = text_a
+        self.text_b = text_b
+        self.label = label
+
+class InputFeatures(object):
+    """A single set of features of data."""
+
+    def __init__(self, input_ids, input_mask, segment_ids, label_id):
+        self.input_ids = input_ids
+        self.input_mask = input_mask
+        self.segment_ids = segment_ids
+        self.label_id = label_id
+
+class DataProcessor(object):
+    """Base class for data converters for sequence classification data sets."""
+
+    def get_train_examples(self, data_dir):
+        """Gets a collection of `InputExample`s for the train set."""
+        raise NotImplementedError()
+
+    def get_dev_examples(self, data_dir):
+        """Gets a collection of `InputExample`s for the dev set."""
+        raise NotImplementedError()
+
+    def get_labels(self):
+        """Gets the list of labels for this data set."""
+        raise NotImplementedError()
+
+    @classmethod
+    def _read_tsv(cls, input_file, quotechar=None):
+        """Reads a tab separated value file."""
+        with open(input_file, "r") as f:
+            reader = csv.reader(f, delimiter="\t", quotechar=quotechar)
+            lines = []
+            for line in reader:
+                if sys.version_info[0] == 2:
+                    line = list(unicode(cell, 'utf-8') for cell in line)
+                lines.append(line)
+            return lines
+
+class DatasetProcessor(DataProcessor):
+
+    def get_examples(self, data_dir, mode):
+        if mode == 'train':
+            return self._create_examples(
+                self._read_tsv(os.path.join(data_dir, "train.tsv")), "train")
+        elif mode == 'eval':
+            return self._create_examples(
+                self._read_tsv(os.path.join(data_dir, "dev.tsv")), "train")
+        elif mode == 'test':
+            return self._create_examples(
+                self._read_tsv(os.path.join(data_dir, "test.tsv")), "test")
+
+    def _create_examples(self, lines, set_type):
+        """Creates examples for the training and dev sets."""
+        examples = []
+        for (i, line) in enumerate(lines):
+            if i == 0:
+                continue
+            if len(line) != 2:
+                continue
+            guid = "%s-%s" % (set_type, i)
+            text_a = line[0]
+            label = line[1]
+
+            examples.append(
+                InputExample(guid=guid, text_a=text_a, text_b=None, label=label))
+        return examples
+
+def convert_examples_to_features(examples, label_list, max_seq_length, tokenizer):
+    """Loads a data file into a list of `InputBatch`s."""
+    label_map = {}
+    if label_list is not None:
+        for i, label in enumerate(label_list):
+            label_map[label] = i
+
+    features = []
+    for (ex_index, example) in enumerate(examples):
+        tokens_a = tokenizer.tokenize(example.text_a)
+
+        tokens_b = None
+        if example.text_b:
+            tokens_b = tokenizer.tokenize(example.text_b)
+            # Modifies `tokens_a` and `tokens_b` in place so that the total
+            # length is less than the specified length.
+            # Account for [CLS], [SEP], [SEP] with "- 3"
+            _truncate_seq_pair(tokens_a, tokens_b, max_seq_length - 3)
+        else:
+            # Account for [CLS] and [SEP] with "- 2"
+            if len(tokens_a) > max_seq_length - 2:
+                tokens_a = tokens_a[:(max_seq_length - 2)]
+
+        # The convention in BERT is:
+        # (a) For sequence pairs:
+        #  tokens:   [CLS] is this jack ##son ##ville ? [SEP] no it is not . [SEP]
+        #  type_ids: 0   0  0    0    0     0       0 0    1  1  1  1   1 1
+        # (b) For single sequences:
+        #  tokens:   [CLS] the dog is hairy . [SEP]
+        #  type_ids: 0   0   0   0  0     0 0
+        #
+        # Where "type_ids" are used to indicate whether this is the first
+        # sequence or the second sequence. The embedding vectors for `type=0` and
+        # `type=1` were learned during pre-training and are added to the wordpiece
+        # embedding vector (and position vector). This is not *strictly* necessary
+        # since the [SEP] token unambigiously separates the sequences, but it makes
+        # it easier for the model to learn the concept of sequences.
+        #
+        # For classification tasks, the first vector (corresponding to [CLS]) is
+        # used as as the "sentence vector". Note that this only makes sense because
+        # the entire model is fine-tuned.
+        tokens = ["[CLS]"] + tokens_a + ["[SEP]"]
+        segment_ids = [0] * len(tokens)
+
+        if tokens_b:
+            tokens += tokens_b + ["[SEP]"]
+            segment_ids += [1] * (len(tokens_b) + 1)
+
+        input_ids = tokenizer.convert_tokens_to_ids(tokens)
+
+        # The mask has 1 for real tokens and 0 for padding tokens. Only real
+        # tokens are attended to.
+        input_mask = [1] * len(input_ids)
+
+        # Zero-pad up to the sequence length.
+        padding = [0] * (max_seq_length - len(input_ids))
+        input_ids += padding
+        input_mask += padding
+        segment_ids += padding
+
+        assert len(input_ids) == max_seq_length
+        assert len(input_mask) == max_seq_length
+        assert len(segment_ids) == max_seq_length
+
+        label_id = label_map[example.label] if label_list is not None else None
+        # if ex_index < 5:
+        #     logger.info("*** Example ***")
+        #     logger.info("guid: %s" % (example.guid))
+        #     logger.info("tokens: %s" % " ".join(
+        #         [str(x) for x in tokens]))
+        #     logger.info("input_ids: %s" % " ".join([str(x) for x in input_ids]))
+        #     logger.info("input_mask: %s" % " ".join([str(x) for x in input_mask]))
+        #     logger.info(
+        #         "segment_ids: %s" % " ".join([str(x) for x in segment_ids]))
+        #     logger.info("label: %s (id = %d)" % (example.label, label_id))
+
+        features.append(
+            InputFeatures(input_ids=input_ids,
+                          input_mask=input_mask,
+                          segment_ids=segment_ids,
+                          label_id=label_id))
+    return features
+
+def _truncate_seq_pair(tokens_a, tokens_b, max_length):
+    """Truncates a sequence pair in place to the maximum length."""
+    # This is a simple heuristic which will always truncate the longer sequence
+    # one token at a time. This makes more sense than truncating an equal percent
+    # of tokens from each, since if one sequence is very short then each token
+    # that's truncated likely contains more information than a longer sequence.
+    while True:
+        total_length = len(tokens_a) + len(tokens_b)
+        if total_length <= max_length:
+            break
+        if len(tokens_a) > len(tokens_b):
+            tokens_a.pop(0)  # For dialogue context
+        else:
+            tokens_b.pop()

TEXTOIR-main/open_intent_detection/dataloaders/llama_loader.py

@@ -0,0 +1,289 @@
+import numpy as np
+import torch
+import os
+import csv
+import sys
+import logging
+from transformers import LlamaTokenizer
+from torch.utils.data import (DataLoader, RandomSampler, SequentialSampler, TensorDataset)
+
+
+class LLAMA_Loader:
+    def __init__(self, args, base_attrs, logger_name = 'Detection'):
+        self.logger = logging.getLogger(logger_name)
+        self.train_examples, self.train_labeled_examples, self.train_unlabeled_examples  = get_examples(args, base_attrs, 'train')
+        self.logger.info("Number of labeled training samples = %s", str(len(self.train_labeled_examples)))
+        self.logger.info("Number of unlabeled training samples = %s", str(len(self.train_unlabeled_examples)))
+
+        self.eval_examples = get_examples(args, base_attrs, 'eval')
+        self.logger.info("Number of evaluation samples = %s", str(len(self.eval_examples)))
+
+        self.test_examples = get_examples(args, base_attrs, 'test')
+        self.logger.info("Number of testing samples = %s", str(len(self.test_examples)))
+        self.base_attrs = base_attrs
+        self.init_loader(args)
+
+    def init_loader(self, args):
+        
+        self.train_labeled_loader = get_loader(self.train_labeled_examples, args, self.base_attrs['label_list'], 'train_labeled', sampler_mode = 'random')
+        self.train_unlabeled_loader = get_loader(self.train_unlabeled_examples, args, self.base_attrs['label_list'], 'train_unlabeled', sampler_mode = 'sequential')
+        self.eval_loader = get_loader(self.eval_examples, args, self.base_attrs['label_list'], 'eval', sampler_mode = 'sequential')
+        self.test_loader = get_loader(self.test_examples, args, self.base_attrs['label_list'], 'test', sampler_mode = 'sequential')
+        self.num_train_examples = len(self.train_labeled_examples)
+
+def get_examples(args, base_attrs, mode):
+
+    processor = DatasetProcessor()
+    ori_examples = processor.get_examples(base_attrs['data_dir'], mode)
+    if mode == 'train':
+
+        labeled_examples, unlabeled_examples = [], []
+        for example in ori_examples:
+
+            if (example.label in base_attrs['known_label_list']) and (np.random.uniform(0, 1) <= args.labeled_ratio):
+                labeled_examples.append(example)
+            else:
+                example.label = base_attrs['unseen_label']
+                unlabeled_examples.append(example)
+
+        return ori_examples, labeled_examples, unlabeled_examples
+
+    elif mode == 'eval':
+
+        examples = []
+        for example in ori_examples:
+            if (example.label in base_attrs['known_label_list']):
+                examples.append(example)
+        
+        return examples
+    
+    elif mode == 'test':
+
+        examples = []
+        for example in ori_examples:
+            if (example.label in base_attrs['label_list']) and (example.label != base_attrs['unseen_label']):
+                examples.append(example)
+            else:
+                example.label = base_attrs['unseen_label']
+                examples.append(example)
+        
+        return examples
+
+def get_loader(examples, args, label_list, mode, sampler_mode = 'sequential'):
+    tokenizer = LlamaTokenizer.from_pretrained(args.llama_model, do_lower_case=True, legacy=False)    
+    tokenizer.pad_token_id = (
+        0  # unk. we want this to be different from the eos token
+    )
+    tokenizer.padding_side = "left" 
+    tokenizer.pad_token = tokenizer.eos_token
+    features = convert_examples_to_features(examples, label_list, args.max_seq_length, tokenizer)
+    input_ids = torch.tensor([f.input_ids for f in features], dtype=torch.long)
+    input_mask = torch.tensor([f.input_mask for f in features], dtype=torch.long)
+    segment_ids = torch.tensor([f.segment_ids for f in features], dtype=torch.long)
+
+    if mode == 'train_unlabeled':
+        label_ids = torch.tensor([-1 for f in features], dtype=torch.long)
+    else:
+        label_ids = torch.tensor([f.label_id for f in features], dtype=torch.long)
+    datatensor = TensorDataset(input_ids, input_mask, segment_ids, label_ids)
+
+    if sampler_mode == 'random':
+        sampler = RandomSampler(datatensor)
+    elif sampler_mode == 'sequential':
+        sampler = SequentialSampler(datatensor)
+
+    if mode == 'train_labeled':   
+        dataloader = DataLoader(datatensor, sampler = sampler, batch_size = args.train_batch_size)    
+
+    else:
+        if mode == 'train_unlabeled':
+            dataloader = DataLoader(datatensor, sampler=sampler, batch_size = args.train_batch_size)    
+        elif mode == 'eval':
+            dataloader = DataLoader(datatensor, sampler=sampler, batch_size = args.eval_batch_size)    
+        elif mode == 'test':
+            dataloader = DataLoader(datatensor, sampler=sampler, batch_size = args.test_batch_size)    
+    
+    return dataloader
+
+class InputExample(object):
+    """A single training/test example for simple sequence classification."""
+
+    def __init__(self, guid, text_a, text_b=None, label=None):
+        """Constructs a InputExample.
+
+        Args:
+            guid: Unique id for the example.
+            text_a: string. The untokenized text of the first sequence. For single
+            sequence tasks, only this sequence must be specified.
+            text_b: (Optional) string. The untokenized text of the second sequence.
+            Only must be specified for sequence pair tasks.
+            label: (Optional) string. The label of the example. This should be
+            specified for train and dev examples, but not for test examples.
+        """
+        self.guid = guid
+        self.text_a = text_a
+        self.text_b = text_b
+        self.label = label
+
+class InputFeatures(object):
+    """A single set of features of data."""
+
+    def __init__(self, input_ids, input_mask, segment_ids, label_id):
+        self.input_ids = input_ids
+        self.input_mask = input_mask
+        self.segment_ids = segment_ids
+        self.label_id = label_id
+
+class DataProcessor(object):
+    """Base class for data converters for sequence classification data sets."""
+
+    def get_train_examples(self, data_dir):
+        """Gets a collection of `InputExample`s for the train set."""
+        raise NotImplementedError()
+
+    def get_dev_examples(self, data_dir):
+        """Gets a collection of `InputExample`s for the dev set."""
+        raise NotImplementedError()
+
+    def get_labels(self):
+        """Gets the list of labels for this data set."""
+        raise NotImplementedError()
+
+    @classmethod
+    def _read_tsv(cls, input_file, quotechar=None):
+        """Reads a tab separated value file."""
+        with open(input_file, "r") as f:
+            reader = csv.reader(f, delimiter="\t", quotechar=quotechar)
+            lines = []
+            for line in reader:
+                if sys.version_info[0] == 2:
+                    line = list(unicode(cell, 'utf-8') for cell in line)
+                lines.append(line)
+            return lines
+
+class DatasetProcessor(DataProcessor):
+
+    def get_examples(self, data_dir, mode):
+        if mode == 'train':
+            return self._create_examples(
+                self._read_tsv(os.path.join(data_dir, "train.tsv")), "train")
+        elif mode == 'eval':
+            return self._create_examples(
+                self._read_tsv(os.path.join(data_dir, "dev.tsv")), "train")
+        elif mode == 'test':
+            return self._create_examples(
+                self._read_tsv(os.path.join(data_dir, "test.tsv")), "test")
+
+    def _create_examples(self, lines, set_type):
+        """Creates examples for the training and dev sets."""
+        examples = []
+        for (i, line) in enumerate(lines):
+            if i == 0:
+                continue
+            if len(line) != 2:
+                continue
+            guid = "%s-%s" % (set_type, i)
+            text_a = line[0]
+            label = line[1]
+
+            examples.append(
+                InputExample(guid=guid, text_a=text_a, text_b=None, label=label))
+        return examples
+
+def convert_examples_to_features(examples, label_list, max_seq_length, tokenizer):
+    """Loads a data file into a list of `InputBatch`s."""
+    label_map = {}
+    if label_list is not None:
+        for i, label in enumerate(label_list):
+            label_map[label] = i
+
+    features = []
+    for (ex_index, example) in enumerate(examples):
+        tokens_a = tokenizer.tokenize(example.text_a)
+
+        tokens_b = None
+        if example.text_b:
+            tokens_b = tokenizer.tokenize(example.text_b)
+            # Modifies `tokens_a` and `tokens_b` in place so that the total
+            # length is less than the specified length.
+            # Account for [CLS], [SEP], [SEP] with "- 3"
+            _truncate_seq_pair(tokens_a, tokens_b, max_seq_length - 3)
+        else:
+            # Account for [CLS] and [SEP] with "- 2"
+            if len(tokens_a) > max_seq_length - 2:
+                tokens_a = tokens_a[:(max_seq_length - 2)]
+
+        # The convention in BERT is:
+        # (a) For sequence pairs:
+        #  tokens:   [CLS] is this jack ##son ##ville ? [SEP] no it is not . [SEP]
+        #  type_ids: 0   0  0    0    0     0       0 0    1  1  1  1   1 1
+        # (b) For single sequences:
+        #  tokens:   [CLS] the dog is hairy . [SEP]
+        #  type_ids: 0   0   0   0  0     0 0
+        #
+        # Where "type_ids" are used to indicate whether this is the first
+        # sequence or the second sequence. The embedding vectors for `type=0` and
+        # `type=1` were learned during pre-training and are added to the wordpiece
+        # embedding vector (and position vector). This is not *strictly* necessary
+        # since the [SEP] token unambigiously separates the sequences, but it makes
+        # it easier for the model to learn the concept of sequences.
+        #
+        # For classification tasks, the first vector (corresponding to [CLS]) is
+        # used as as the "sentence vector". Note that this only makes sense because
+        # the entire model is fine-tuned.
+        tokens = ["[CLS]"] + tokens_a + ["[SEP]"]
+        segment_ids = [0] * len(tokens)
+
+        if tokens_b:
+            tokens += tokens_b + ["[SEP]"]
+            segment_ids += [1] * (len(tokens_b) + 1)
+
+        input_ids = tokenizer.convert_tokens_to_ids(tokens)
+
+        # The mask has 1 for real tokens and 0 for padding tokens. Only real
+        # tokens are attended to.
+        input_mask = [1] * len(input_ids)
+
+        # Zero-pad up to the sequence length.
+        padding = [0] * (max_seq_length - len(input_ids))
+        input_ids += padding
+        input_mask += padding
+        segment_ids += padding
+
+        assert len(input_ids) == max_seq_length
+        assert len(input_mask) == max_seq_length
+        assert len(segment_ids) == max_seq_length
+
+        label_id = label_map[example.label] if label_list is not None else None
+        # if ex_index < 5:
+        #     logger.info("*** Example ***")
+        #     logger.info("guid: %s" % (example.guid))
+        #     logger.info("tokens: %s" % " ".join(
+        #         [str(x) for x in tokens]))
+        #     logger.info("input_ids: %s" % " ".join([str(x) for x in input_ids]))
+        #     logger.info("input_mask: %s" % " ".join([str(x) for x in input_mask]))
+        #     logger.info(
+        #         "segment_ids: %s" % " ".join([str(x) for x in segment_ids]))
+        #     logger.info("label: %s (id = %d)" % (example.label, label_id))
+
+        features.append(
+            InputFeatures(input_ids=input_ids,
+                          input_mask=input_mask,
+                          segment_ids=segment_ids,
+                          label_id=label_id))
+    return features
+
+def _truncate_seq_pair(tokens_a, tokens_b, max_length):
+    """Truncates a sequence pair in place to the maximum length."""
+    # This is a simple heuristic which will always truncate the longer sequence
+    # one token at a time. This makes more sense than truncating an equal percent
+    # of tokens from each, since if one sequence is very short then each token
+    # that's truncated likely contains more information than a longer sequence.
+    while True:
+        total_length = len(tokens_a) + len(tokens_b)
+        if total_length <= max_length:
+            break
+        if len(tokens_a) > len(tokens_b):
+            tokens_a.pop(0)  # For dialogue context
+        else:
+            tokens_b.pop()

TEXTOIR-main/open_intent_detection/examples/run_ADB.sh

@@ -0,0 +1,29 @@
+#!/usr/bin bash
+
+for dataset in 'banking' 'oos' 'stackoverflow'
+do
+    for known_cls_ratio in 0.25 0.5 0.75
+    do
+        for labeled_ratio in 1.0
+        do
+            for seed in 0 1 2 3 4 5 6 7 8 9
+            do 
+                python run.py \
+                --dataset $dataset \
+                --method 'ADB' \
+                --known_cls_ratio $known_cls_ratio \
+                --labeled_ratio $labeled_ratio \
+                --seed $seed \
+                --backbone 'bert' \
+                --config_file_name 'ADB' \
+                --loss_fct 'CrossEntropyLoss' \
+                --gpu_id '0' \
+                --train \
+                --pretrain \
+                --results_file_name 'results_ADB.csv' \
+                --save_results \
+                --save_model
+            done
+        done
+    done
+done

TEXTOIR-main/open_intent_detection/examples/run_ARPL.sh

@@ -0,0 +1,28 @@
+#!/usr/bin bash
+
+for dataset in 'stackoverflow'
+do
+    for known_cls_ratio in 0.25 0.5 0.75
+    do
+        for labeled_ratio in 1.0 0.2 0.4 0.6 0.8
+        do
+            for seed in 0 1 2 3 4 5 6 7 8 9
+            do 
+                python run.py \
+                --dataset $dataset \
+                --method 'ARPL' \
+                --known_cls_ratio $known_cls_ratio \
+                --labeled_ratio $labeled_ratio \
+                --seed $seed \
+                --backbone 'bert' \
+                --config_file_name 'ARPL' \
+                --pretrain \
+                --train \
+                --loss_fct 'ARPLoss' \
+                --gpu_id '0' \
+                --results_file_name 'results_ARPL.csv' \
+                --save_results
+            done
+        done
+    done
+done

TEXTOIR-main/open_intent_detection/examples/run_DA-ADB.sh

@@ -0,0 +1,29 @@
+#!/usr/bin bash
+
+for dataset in 'banking' 'oos' 'stackoverflow'
+do
+    for known_cls_ratio in 0.25 0.5 0.75
+    do
+        for labeled_ratio in 1.0
+        do
+            for seed in 0 1 2 3 4 5 6 7 8 9
+            do 
+                python run.py \
+                --dataset $dataset \
+                --method 'DA-ADB' \
+                --known_cls_ratio $known_cls_ratio \
+                --labeled_ratio $labeled_ratio \
+                --seed $seed \
+                --backbone 'bert_disaware' \
+                --config_file_name 'DA-ADB' \
+                --loss_fct 'CrossEntropyLoss' \
+                --gpu_id '0' \
+                --train \
+                --pretrain \
+                --results_file_name 'results_DA-ADB.csv' \
+                --save_results
+            done
+        done
+    done
+done
+