| Step | \n", "Training Loss | \n", "
|---|---|
| 10 | \n", "4.790900 | \n", "
| 20 | \n", "3.451500 | \n", "
| 30 | \n", "3.494700 | \n", "
| 40 | \n", "3.316100 | \n", "
| 50 | \n", "3.229200 | \n", "
| 60 | \n", "3.181000 | \n", "
| 70 | \n", "3.082100 | \n", "
| 80 | \n", "3.248200 | \n", "
| 90 | \n", "3.043200 | \n", "
| 100 | \n", "3.231200 | \n", "
| 110 | \n", "3.036500 | \n", "
| 120 | \n", "3.368400 | \n", "
| 130 | \n", "3.273700 | \n", "
| 140 | \n", "3.033700 | \n", "
| 150 | \n", "3.027700 | \n", "
| 160 | \n", "3.048600 | \n", "
| 170 | \n", "3.247900 | \n", "
| 180 | \n", "2.949100 | \n", "
| 190 | \n", "2.925200 | \n", "
| 200 | \n", "3.107500 | \n", "
" ] }, "metadata": {} }, { "output_type": "execute_result", "data": { "text/plain": [ "TrainOutput(global_step=200, training_loss=3.254313144683838, metrics={'train_runtime': 61.602, 'train_samples_per_second': 19.48, 'train_steps_per_second': 3.247, 'total_flos': 103816598925312.0, 'train_loss': 3.254313144683838, 'epoch': 0.16})" ] }, "metadata": {}, "execution_count": 98 } ] }, { "cell_type": "markdown", "source": [ "# ▶️ Try Out The Fine-Tuned Model" ], "metadata": { "id": "ybsXkeuF92sF" } }, { "cell_type": "code", "source": [ "# Generate lyrics using random segments of the test lyrics\n", "num_examples = 5 # Number of random examples to generate\n", "max_segment_length = 200 # Maximum length of each lyric segment" ], "metadata": { "id": "IIequcNDffhi" }, "execution_count": null, "outputs": [] }, { "cell_type": "code", "source": [ "for i in range(num_examples):\n", " # Randomly select a starting index for the lyric segment\n", " start_index = random.randint(0, len(test_lyrics) - max_segment_length)\n", " end_index = start_index + max_segment_length\n", "\n", " # Extract the lyric segment\n", " lyric_segment = ' '.join(test_lyrics[start_index:end_index])\n", "\n", " print(f\"Example {i+1}:\")\n", " print(\"INPUT:\")\n", " print(lyric_segment)\n", " print(\"OUTPUT:\")\n", " generate_lyrics(lyric_segment, model, tokenizer)\n", " print(\"\\n\")" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "Zcb44_A3VY8R", "outputId": "66a0c80e-4a1a-4106-a20f-17e33d8332d8" }, "execution_count": null, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Example 1:\n", "INPUT:\n", "Never be so polite You forget your power Never wield such power You forget to be polite And if I didn't know better I'd think you were listening to me now If I didn't know better I'd think you were still around What died didn't stay dead What died didn't stay dead You're alive, you're alive in my head What died didn't stay dead What died didn't stay dead You're alive, so alive The autumn chill that wakes me up You loved the amber skies so much Long limbs and frozen swims You'd always go past where our feet could touch And I complained the whole way there The car ride back and up the stairs I should've asked you questions I should've asked you how to be Asked you to write it down for me Should've kept every grocery store receipt 'Cause every scrap of you would be taken from me Watched as you signed your name Marjorie All your closets of backlogged dreams And how you left them all to me What died didn't stay dead What died didn't stay dead You're alive, you're alive in my head What died didn't stay dead What died didn't stay dead You're alive, so alive And if I didn't know better I'd think you were singing to me now If I didn't know better I'd think you were still around I know better But I still feel you all around I know better But you're still around It's been a long time And seeing the shape of your name Still spells out pain It wasn't right The way it all went down Looks like you know that now Yes, I got your letter Yes, I'm doing better It cut deep to know ya, right to the bone Yes, I got your letter Yes, I'm doing better I know that it's over, I don't need your Closure, your closure Don't treat me like Some situation that needs to be handled I'm fine with my spite And my tears, and my beers and my candles I can feel you smoothing me over Yes, I got your letter Yes, I'm doing better It cut deep to know ya, right to the bone Yes, I got your letter Yes, I'm doing better I know that it's over, I don't need your Closure, your closure Your closure, your closure I know I'm just a wrinkle in your new life Staying friends would iron it out so nice Guilty, guilty, reaching out across the sea That you put between you and me But it's fake and it's oh so unnecessary Yes, I got your letter Yes, I'm doing better It cut deep to know ya, right to the bone Yes, I got your letter Yes, I'm doing better I know that it's over, I don't need your Closure, closure, your closure Your closure Gray November I've been down since July Motion capture Put me in a bad light I replay my footsteps on each stepping stone Trying to find the one where I went wrong Writing letters Addressed to the fire And I was catching my breath Staring out an open window Catching my death And I couldn't be sure I had a feeling so peculiar That this pain would be for Evermore Hey December Guess I'm feeling unmoored Can't remember What I used to fight for I rewind the tape, but all it does is pause On the very moment all was lost Sending signals To be double crossed And I was catching my breath Barefoot in the wildest winter Catching my death And I couldn't be sure I had a feeling so peculiar That this pain would be for Evermore Evermore Can't not think of all the cost And the things that will be lost Oh, can we just get a pause? To be certain, we'll be tall again Whether weather be the frost Or the violence of the dog days I'm on waves, out being tossed Is there a line that I could just go cross? And when I was shipwrecked Can't think of all the cost I thought of you All the things that will be lost now In the cracks of light Can we just get a pause? I dreamed of you To be certain we'll be tall again, if you think of all the costs It was real enough Whether weather be the frost To get me through Or the violence of the dog days Or the violence of the dog days Out on waves, being tossed I'm on waves, out being tossed I swear Is there a line that we can just go cross? You were there And I was catching my breath Floors of a cabin creaking under my step And I couldn't be sure I had a feeling so peculiar This pain wouldn't be for Evermore Evermore Evermore This pain wouldn't be for evermore Evermore Friends break up, friends get married Strangers get born, strangers get buried Trends change, rumors fly through new skies But I'm right where you left me Matches burn after the other Pages turn and stick to each other Wages earned and lessons learned But I, I'm right where you left me Help, I'm still at the restaurant Still sitting in a corner I haunt Cross legged in the dim light They say, What a sad sight I, I swear you could hear a hair pin drop Right when I felt the moment stop Glass shattered on the white cloth Everybody moved on, I, I stayed there Dust collected on my pinned up hair They expected me to find somewhere Some perspective, but I sat and stared Right where you left me You left me no, you left me no You left me no choice but to stay here forever You left me, you left me no, oh, you left me no You left me no choice but to stay here forever Did you ever hear about the girl who got frozen? Time went on for everybody else, she won't know it She's still twenty three inside her fantasy How it was supposed to be Did you hear about the girl who lives in delusion? Breakups happen every day, you don't have to lose it She's still twenty three inside her fantasy And you're sitting in front of me At the restaurant when I was still the one you want Cross legged in the dim light, everything was just right I, I could feel the mascara run You told me that you met someone Glass shattered on the white cloth Everybody moved on Help, I'm still at the restaurant Still sitting in a corner I haunt Cross legged in the dim light They say, What a sad sight I, I stayed there Dust collected on my pinned up hair I'm sure that you got a wife out there Kids and Christmas, but I'm unaware 'Cause I'm right where I cause no harm Mind my business If our love died young I can't bear witness And it's been so long But if you ever think you got it wrong I'm right where You left me You left me no, oh, you left me no You left me no choice but to stay here forever You left me You left me no, oh, you left me no\n", "OUTPUT:\n", " Why then why are they gambray colors Again! Just plain old fashion black or gray No color or tone too late Now, yeah baby now Come look here brightened skylight colored sunshining So fair game play off both sides see what all ye doin bring us close together Ships come bearing sweet treasuries of flowers Flowers keep falling and dying Down by red telephone signal Two houses walk side-byon When nobody goes downthe door huh ah mum wooh Apry momma ohah Ah yes he says Yeah little girls There is nothing good about boys or men Let people enjoy themselves Oxygen their funny names For now you hold alight the lights and music Lift uh hand in mine hands and make your night into morning Alcoholic brews sparkling linger till dawn Like thunder, though. He said once I heard him coming outside, well played boy Look beautiful! I wanted a woman like you Last saw you in summer\n", "\n", "\n", "Example 2:\n", "INPUT:\n", "I've got some tricks up my sleeve Takes one to know one You're a cowboy like me Never wanted love Just a fancy car Now I'm waiting by the phone Like I'm sitting in an airport bar You had some tricks up your sleeve Takes one to know one You're a cowboy like me Perched in the dark Telling all the rich folks anything they wanna hear Like it could be love I could be the way forward Only if they pay for it You're a bandit like me Eyes full of stars Hustling for the good life Never thought I'd meet you here It could be love We could be the way forward And I know I'll pay for it You're a cowboy like me Perched in the dark Telling all the rich folks anything they wanna hear Like it could be love I could be the way forward Only if they pay for it You're a bandit like me Eyes full of stars Hustling for the good life Never thought I'd meet you here It could be love We could be the way forward And I know I'll pay for it And the skeletons in both our closets Plotted hard to fuck this up And the old men that I've swindled Really did believe I was the one And the ladies lunching have their stories about When you passed through town But that was all before I locked it down Now you hang from my lips Like the Gardens of Babylon With your boots beneath my bed Forever is the sweetest con I've had some tricks up my sleeve Takes one to know one You're a cowboy like me And I'm never gonna love again I'm never gonna love again I'm never gonna love again Fatefully I tried to pick my battles 'til the battle picked me Misery Like the war of words I shouted in my sleep And you passed right by I was in the alley, surrounded on all sides The knife cuts both ways If the shoe fits, walk in it 'til your high heels break And I fell from the pedestal Right down the rabbit hole Long story short, it was a bad time Pushed from the precipice Clung to the nearest lips Long story short, it was the wrong guy Now I'm all about you I'm all about you, ah Yeah, yeah I'm all about you, ah Yeah, yeah Actually I always felt I must look better in the rear view Missing me At the golden gates they once held the keys to When I dropped my sword I threw it in the bushes and knocked on your door And we live in peace But if someone comes at us, this time, I'm ready 'Cause I fell from the pedestal Right down the rabbit hole Long story short, it was a bad time Pushed from the precipice Clung to the nearest lips Long story short, it was the wrong guy Now I'm all about you I'm all about you, ah Yeah, yeah I'm all about you No more keepin' score Now I just keep you warm Keep you warm No more tug of war Now I just know there's more Know there's more No more keepin' score Now I just keep you warm Keep you warm And my waves meet your shore Ever and evermore Past me I wanna tell you not to get lost in these petty things Your nemeses Will defeat themselves before you get the chance to swing And he's passing by Rare as the glimmer of a comet in the sky And he feels like home If the shoe fits, walk in it everywhere you go And I fell from the pedestal Right down the rabbit hole Long story short, it was a bad time Pushed from the precipice Climbed right back up the cliff Long story short, I survived Now I'm all about you And now I'm all about you, ah And now I'm all about you And now I'm all about you, ah Yeah, yeah I'm all about you And now Yeah, yeah I'm all about you Long story short, it was a bad time Long story short, I survived Never be so kind You forget to be clever Never be so clever You forget to be kind And if I didn't know better I'd think you were talking to me now If I didn't know better I'd think you were still around What died didn't stay dead What died didn't stay dead You're alive, you're alive in my head What died didn't stay dead What died didn't stay dead You're alive, so alive Never be so polite You forget your power Never wield such power You forget to be polite And if I didn't know better I'd think you were listening to me now If I didn't know better I'd think you were still around What died didn't stay dead What died didn't stay dead You're alive, you're alive in my head What died didn't stay dead What died didn't stay dead You're alive, so alive The autumn chill that wakes me up You loved the amber skies so much Long limbs and frozen swims You'd always go past where our feet could touch And I complained the whole way there The car ride back and up the stairs I should've asked you questions I should've asked you how to be Asked you to write it down for me Should've kept every grocery store receipt 'Cause every scrap of you would be taken from me Watched as you signed your name Marjorie All your closets of backlogged dreams And how you left them all to me What died didn't stay dead What died didn't stay dead You're alive, you're alive in my head What died didn't stay dead What died didn't stay dead You're alive, so alive And if I didn't know better I'd think you were singing to me now If I didn't know better I'd think you were still around I know better But I still feel you all around I know better But you're still around It's been a long time And seeing the shape of your name Still spells out pain It wasn't right The way it all went down Looks like you know that now Yes, I got your letter Yes, I'm doing better It cut deep to know ya, right to the bone Yes, I got your letter Yes, I'm doing better I know that it's over, I don't need your Closure, your closure Don't treat me like Some situation that needs to be handled I'm fine with my spite And my tears, and my beers and my candles I can feel you smoothing me over Yes, I got your letter Yes, I'm doing better It cut deep to know ya, right to the bone Yes, I got your letter Yes, I'm doing better\n", "OUTPUT:\n", " It looked pretty but too late Nope! Took three days, six weeks. Wish then two years later This made me smile Dirty wet lace-knottow That has no place under those sheets There isn't any mistake what to do But trying it won't hurt anymore A feeling almost suffocating? Slowly moving So smooth my blood runs cold Or so close next thing to see how far gone My eyes glaze Over white silk sat me missing my train Here come rosas red wings Black wings black hair blue cheekbudder brown lipstick on yellow green Oh dear oh sweet! Poppy blossoms little kayaking in the quiet stream Oh ohh, ohhh...oh ah Ahooi ahanahaaaaa ahainna ain´nnggghh Oops oop ouuuh!! Not another party na haaaha! Another round yukunai daii ioi? Ok poppoppyyyyyy ma\n", "\n", "\n", "Example 3:\n", "INPUT:\n", "It's a goddamn blaze in the dark And you started it You started it So yeah, it's a war It's the goddamn fight of my life And you started it You started it Oh, I can't Stop you putting roots in my dreamland My house of stone, your ivy grows And now I'm covered In you, in you Now I'm covered in you In you And the tennis court was covered up With some tent like thing And you asked me to dance But I said, Dancin' is a dangerous game Oh, I thought This is gonna be one of those things Now I know I'm never gonna love again I've got some tricks up my sleeve Takes one to know one You're a cowboy like me Never wanted love Just a fancy car Now I'm waiting by the phone Like I'm sitting in an airport bar You had some tricks up your sleeve Takes one to know one You're a cowboy like me Perched in the dark Telling all the rich folks anything they wanna hear Like it could be love I could be the way forward Only if they pay for it You're a bandit like me Eyes full of stars Hustling for the good life Never thought I'd meet you here It could be love We could be the way forward And I know I'll pay for it You're a cowboy like me Perched in the dark Telling all the rich folks anything they wanna hear Like it could be love I could be the way forward Only if they pay for it You're a bandit like me Eyes full of stars Hustling for the good life Never thought I'd meet you here It could be love We could be the way forward And I know I'll pay for it And the skeletons in both our closets Plotted hard to fuck this up And the old men that I've swindled Really did believe I was the one And the ladies lunching have their stories about When you passed through town But that was all before I locked it down Now you hang from my lips Like the Gardens of Babylon With your boots beneath my bed Forever is the sweetest con I've had some tricks up my sleeve Takes one to know one You're a cowboy like me And I'm never gonna love again I'm never gonna love again I'm never gonna love again Fatefully I tried to pick my battles 'til the battle picked me Misery Like the war of words I shouted in my sleep And you passed right by I was in the alley, surrounded on all sides The knife cuts both ways If the shoe fits, walk in it 'til your high heels break And I fell from the pedestal Right down the rabbit hole Long story short, it was a bad time Pushed from the precipice Clung to the nearest lips Long story short, it was the wrong guy Now I'm all about you I'm all about you, ah Yeah, yeah I'm all about you, ah Yeah, yeah Actually I always felt I must look better in the rear view Missing me At the golden gates they once held the keys to When I dropped my sword I threw it in the bushes and knocked on your door And we live in peace But if someone comes at us, this time, I'm ready 'Cause I fell from the pedestal Right down the rabbit hole Long story short, it was a bad time Pushed from the precipice Clung to the nearest lips Long story short, it was the wrong guy Now I'm all about you I'm all about you, ah Yeah, yeah I'm all about you No more keepin' score Now I just keep you warm Keep you warm No more tug of war Now I just know there's more Know there's more No more keepin' score Now I just keep you warm Keep you warm And my waves meet your shore Ever and evermore Past me I wanna tell you not to get lost in these petty things Your nemeses Will defeat themselves before you get the chance to swing And he's passing by Rare as the glimmer of a comet in the sky And he feels like home If the shoe fits, walk in it everywhere you go And I fell from the pedestal Right down the rabbit hole Long story short, it was a bad time Pushed from the precipice Climbed right back up the cliff Long story short, I survived Now I'm all about you And now I'm all about you, ah And now I'm all about you And now I'm all about you, ah Yeah, yeah I'm all about you And now Yeah, yeah I'm all about you Long story short, it was a bad time Long story short, I survived Never be so kind You forget to be clever Never be so clever You forget to be kind And if I didn't know better I'd think you were talking to me now If I didn't know better I'd think you were still around What died didn't stay dead What died didn't stay dead You're alive, you're alive in my head What died didn't stay dead What died didn't stay dead You're alive, so alive Never be so polite You forget your power Never wield such power You forget to be polite And if I didn't know better I'd think you were listening to me now If I didn't know better I'd think you were still around What died didn't stay dead What died didn't stay dead You're alive, you're alive in my head What died didn't stay dead What died didn't stay dead You're alive, so alive The autumn chill that wakes me up You loved the amber skies so much Long limbs and frozen swims You'd always go past where our feet could touch And I complained the whole way there The car ride back and up the stairs I should've asked you questions I should've asked you how to be Asked you to write it down for me Should've kept every grocery store receipt 'Cause every scrap of you would be taken from me Watched as you signed your name Marjorie All your closets of backlogged dreams And how you left them all to me What died didn't stay dead What died didn't stay dead You're alive, you're alive in my head What died didn't stay dead What died didn't stay dead You're alive, so alive And if I didn't know better I'd think you were singing to me now If I didn't know better I'd think you were still around I know better But I still feel you all around I know better But you're still around It's been a long time\n", "OUTPUT:\n", " since anybody has seen me Before anyone'seen Me with blue eyes A lotta redness He gets caught twice when his wings beat away They come close As soon sometime Anywhere anytime anywhere Reddy girl For you, ladyfriend Love ain'da hit everything clean But something unsaid means nothing no doubt you betrayed That wasn't who you knew but now see yow best friend Is really tired? So why do I hide Where my friends kiss I kiss out loudly Nothin' else could happen. Lines across your canvas Layers overlinе eachother Sometimes things fall apart Don; tha\\; she lik ean\\you d\\'oh I need a bigger pound loaf A big baggy purple ball in it Wiggle-Wriggl glabbehte hahahahaa She doesn’thi€« make cute sho**girl girls Gine yoù hyeuh! Ahhhhaaa halle\n", "\n", "\n", "Example 4:\n", "INPUT:\n", "All your closets of backlogged dreams And how you left them all to me What died didn't stay dead What died didn't stay dead You're alive, you're alive in my head What died didn't stay dead What died didn't stay dead You're alive, so alive And if I didn't know better I'd think you were singing to me now If I didn't know better I'd think you were still around I know better But I still feel you all around I know better But you're still around It's been a long time And seeing the shape of your name Still spells out pain It wasn't right The way it all went down Looks like you know that now Yes, I got your letter Yes, I'm doing better It cut deep to know ya, right to the bone Yes, I got your letter Yes, I'm doing better I know that it's over, I don't need your Closure, your closure Don't treat me like Some situation that needs to be handled I'm fine with my spite And my tears, and my beers and my candles I can feel you smoothing me over Yes, I got your letter Yes, I'm doing better It cut deep to know ya, right to the bone Yes, I got your letter Yes, I'm doing better I know that it's over, I don't need your Closure, your closure Your closure, your closure I know I'm just a wrinkle in your new life Staying friends would iron it out so nice Guilty, guilty, reaching out across the sea That you put between you and me But it's fake and it's oh so unnecessary Yes, I got your letter Yes, I'm doing better It cut deep to know ya, right to the bone Yes, I got your letter Yes, I'm doing better I know that it's over, I don't need your Closure, closure, your closure Your closure Gray November I've been down since July Motion capture Put me in a bad light I replay my footsteps on each stepping stone Trying to find the one where I went wrong Writing letters Addressed to the fire And I was catching my breath Staring out an open window Catching my death And I couldn't be sure I had a feeling so peculiar That this pain would be for Evermore Hey December Guess I'm feeling unmoored Can't remember What I used to fight for I rewind the tape, but all it does is pause On the very moment all was lost Sending signals To be double crossed And I was catching my breath Barefoot in the wildest winter Catching my death And I couldn't be sure I had a feeling so peculiar That this pain would be for Evermore Evermore Can't not think of all the cost And the things that will be lost Oh, can we just get a pause? To be certain, we'll be tall again Whether weather be the frost Or the violence of the dog days I'm on waves, out being tossed Is there a line that I could just go cross? And when I was shipwrecked Can't think of all the cost I thought of you All the things that will be lost now In the cracks of light Can we just get a pause? I dreamed of you To be certain we'll be tall again, if you think of all the costs It was real enough Whether weather be the frost To get me through Or the violence of the dog days Or the violence of the dog days Out on waves, being tossed I'm on waves, out being tossed I swear Is there a line that we can just go cross? You were there And I was catching my breath Floors of a cabin creaking under my step And I couldn't be sure I had a feeling so peculiar This pain wouldn't be for Evermore Evermore Evermore This pain wouldn't be for evermore Evermore Friends break up, friends get married Strangers get born, strangers get buried Trends change, rumors fly through new skies But I'm right where you left me Matches burn after the other Pages turn and stick to each other Wages earned and lessons learned But I, I'm right where you left me Help, I'm still at the restaurant Still sitting in a corner I haunt Cross legged in the dim light They say, What a sad sight I, I swear you could hear a hair pin drop Right when I felt the moment stop Glass shattered on the white cloth Everybody moved on, I, I stayed there Dust collected on my pinned up hair They expected me to find somewhere Some perspective, but I sat and stared Right where you left me You left me no, you left me no You left me no choice but to stay here forever You left me, you left me no, oh, you left me no You left me no choice but to stay here forever Did you ever hear about the girl who got frozen? Time went on for everybody else, she won't know it She's still twenty three inside her fantasy How it was supposed to be Did you hear about the girl who lives in delusion? Breakups happen every day, you don't have to lose it She's still twenty three inside her fantasy And you're sitting in front of me At the restaurant when I was still the one you want Cross legged in the dim light, everything was just right I, I could feel the mascara run You told me that you met someone Glass shattered on the white cloth Everybody moved on Help, I'm still at the restaurant Still sitting in a corner I haunt Cross legged in the dim light They say, What a sad sight I, I stayed there Dust collected on my pinned up hair I'm sure that you got a wife out there Kids and Christmas, but I'm unaware 'Cause I'm right where I cause no harm Mind my business If our love died young I can't bear witness And it's been so long But if you ever think you got it wrong I'm right where You left me You left me no, oh, you left me no You left me no choice but to stay here forever You left me You left me no, oh, you left me no You left me no choice but to stay here forever When the dinner is cold and the chatter gets old You ask for the tab Or that moment again, he's insisting that friends Look at each other like that When the words of a sister come back in whispers That prove she was not in fact what she seemed Not a twin from your dreams She's a crook who was caught That old familiar body ache The snaps from the same little breaks in your soul You know when it's time to go Twenty years at your job Then the son of the boss gets the spot that was yours Or trying to stay for the kids When keeping it how it is will only break their hearts worse That old familiar body ache The snaps from the same little breaks in your soul You know when it's time to go Sometimes giving up is the strong thing Sometimes to run is the brave thing Sometimes walking out is the one thing That will find you the right thing Sometimes giving up is the strong thing Sometimes to run is the brave thing Sometimes walking out is the one thing\n", "OUTPUT:\n", " No longer must we stand Watcher in place watcha do I see the eyes they are never more then blinking lights From another room, they walk away as nothing happened Why canceled wedding made simple After getting engaged Just before telling everyone off One last kiss Before I knew why He walked softly home His arms wide Open ended by ghosted night Never knowing what waited too late There comes breaking sunshine Here stands his daughter Who called him once Now seems too much, yet well isn’twistling it did something amaze yourself Wherever she led A caravan has driven down to join us\\“Daddy said”But those boys hides gone blue Oppression is wicked \\He screamed loud\\\"You’re going broke!\\”Now it looks black\\The hotter you drink,\\They dance round at losing money on dads-date and marriage We are talking bout dad drinking tonight<\\/\\{\\}\\_\\=\\\\\\__=-~\n", "\n", "\n", "Example 5:\n", "INPUT:\n", "Now you hang from my lips Like the Gardens of Babylon With your boots beneath my bed Forever is the sweetest con I've had some tricks up my sleeve Takes one to know one You're a cowboy like me And I'm never gonna love again I'm never gonna love again I'm never gonna love again Fatefully I tried to pick my battles 'til the battle picked me Misery Like the war of words I shouted in my sleep And you passed right by I was in the alley, surrounded on all sides The knife cuts both ways If the shoe fits, walk in it 'til your high heels break And I fell from the pedestal Right down the rabbit hole Long story short, it was a bad time Pushed from the precipice Clung to the nearest lips Long story short, it was the wrong guy Now I'm all about you I'm all about you, ah Yeah, yeah I'm all about you, ah Yeah, yeah Actually I always felt I must look better in the rear view Missing me At the golden gates they once held the keys to When I dropped my sword I threw it in the bushes and knocked on your door And we live in peace But if someone comes at us, this time, I'm ready 'Cause I fell from the pedestal Right down the rabbit hole Long story short, it was a bad time Pushed from the precipice Clung to the nearest lips Long story short, it was the wrong guy Now I'm all about you I'm all about you, ah Yeah, yeah I'm all about you No more keepin' score Now I just keep you warm Keep you warm No more tug of war Now I just know there's more Know there's more No more keepin' score Now I just keep you warm Keep you warm And my waves meet your shore Ever and evermore Past me I wanna tell you not to get lost in these petty things Your nemeses Will defeat themselves before you get the chance to swing And he's passing by Rare as the glimmer of a comet in the sky And he feels like home If the shoe fits, walk in it everywhere you go And I fell from the pedestal Right down the rabbit hole Long story short, it was a bad time Pushed from the precipice Climbed right back up the cliff Long story short, I survived Now I'm all about you And now I'm all about you, ah And now I'm all about you And now I'm all about you, ah Yeah, yeah I'm all about you And now Yeah, yeah I'm all about you Long story short, it was a bad time Long story short, I survived Never be so kind You forget to be clever Never be so clever You forget to be kind And if I didn't know better I'd think you were talking to me now If I didn't know better I'd think you were still around What died didn't stay dead What died didn't stay dead You're alive, you're alive in my head What died didn't stay dead What died didn't stay dead You're alive, so alive Never be so polite You forget your power Never wield such power You forget to be polite And if I didn't know better I'd think you were listening to me now If I didn't know better I'd think you were still around What died didn't stay dead What died didn't stay dead You're alive, you're alive in my head What died didn't stay dead What died didn't stay dead You're alive, so alive The autumn chill that wakes me up You loved the amber skies so much Long limbs and frozen swims You'd always go past where our feet could touch And I complained the whole way there The car ride back and up the stairs I should've asked you questions I should've asked you how to be Asked you to write it down for me Should've kept every grocery store receipt 'Cause every scrap of you would be taken from me Watched as you signed your name Marjorie All your closets of backlogged dreams And how you left them all to me What died didn't stay dead What died didn't stay dead You're alive, you're alive in my head What died didn't stay dead What died didn't stay dead You're alive, so alive And if I didn't know better I'd think you were singing to me now If I didn't know better I'd think you were still around I know better But I still feel you all around I know better But you're still around It's been a long time And seeing the shape of your name Still spells out pain It wasn't right The way it all went down Looks like you know that now Yes, I got your letter Yes, I'm doing better It cut deep to know ya, right to the bone Yes, I got your letter Yes, I'm doing better I know that it's over, I don't need your Closure, your closure Don't treat me like Some situation that needs to be handled I'm fine with my spite And my tears, and my beers and my candles I can feel you smoothing me over Yes, I got your letter Yes, I'm doing better It cut deep to know ya, right to the bone Yes, I got your letter Yes, I'm doing better I know that it's over, I don't need your Closure, your closure Your closure, your closure I know I'm just a wrinkle in your new life Staying friends would iron it out so nice Guilty, guilty, reaching out across the sea That you put between you and me But it's fake and it's oh so unnecessary Yes, I got your letter Yes, I'm doing better It cut deep to know ya, right to the bone Yes, I got your letter Yes, I'm doing better I know that it's over, I don't need your Closure, closure, your closure Your closure Gray November I've been down since July Motion capture Put me in a bad light I replay my footsteps on each stepping stone Trying to find the one where I went wrong Writing letters Addressed to the fire And I was catching my breath Staring out an open window Catching my death And I couldn't be sure I had a feeling so peculiar That this pain would be for Evermore Hey December Guess I'm feeling unmoored Can't remember What I used to fight for I rewind the tape, but all it does is pause On the very moment all was lost Sending signals To be double crossed And I was catching my breath Barefoot in the wildest winter Catching my death\n", "OUTPUT:\n", " and depressed So perfect I wanted him or I knew There was depth, who knows the other side? Ah oh oh ahhhhh Hahhaaohaaa Just let her free (That she will become...) How easy those days when you spent away to die a second later I remembered what gave rise A woman turned herself into a beautiful hobgawker Into nothing good For Goddess Gather round my throne My seat may lack comforting features We come here looking down at Her face only two hearts Warm hands against cold breasts 278 miles separated They might see their beauty as red thread. Making noises heard Every eye seen blue night dad sees in mirror Always makes sense Oh, oh, oh, Oh-oh\n", "So awkward you were playing games She made fun of his lamentations you did make something up Loud voice: Shut down! Hehe Hello girlfriend why are u da… hahaha hurray hihi I\n", "\n", "\n" ] } ] }, { "cell_type": "markdown", "source": [ "# 🗣️ What's Next" ], "metadata": { "id": "mGzP7pUQ9yg-" } }, { "cell_type": "markdown", "source": [ "### 😭 But the lyrics don't make sense?????" ], "metadata": { "id": "F0N6qhPO1HSf" } }, { "cell_type": "markdown", "source": [ "Even though we've already fine-tuned the model on a dataset of song lyrics, the generated output should ideally be more coherent and relevant to Taylor Swift.\n", "\n", "\n", "There could be a few reasons for this:\n", "\n", "* **Insufficient fine-tuning**\n", "\n", "Depending on the size and quality of your fine-tuning dataset, the model may require more training to effectively capture the patterns and styles of song lyrics. Fine-tuning for a longer duration or with a larger dataset may help improve the model's performance.\n", "\n", "\n", "* **Overfitting**\n", "\n", "If the fine-tuning dataset is small or not diverse enough, the model may overfit to the specific examples in the dataset. This can lead to the model generating lyrics that are too similar to the training data or not generalizing well to new inputs.\n", "\n", "\n", "\n", "* **Model architecture limitations**\n", "\n", "The TinyLlama model architecture may have limitations in capturing the complexities and nuances of song lyrics. Some nonsense or incoherent output may be inherent to the model's design and capacity." ], "metadata": { "id": "C1R4nkk20_cG" } }, { "cell_type": "markdown", "source": [ "### 🤔 So what can we do instead?" ], "metadata": { "id": "rmV4PN471wQ5" } }, { "cell_type": "markdown", "source": [ "Try the following:\n", "\n", "* **`Expand and diversify the fine-tuning dataset`**\n", "\n", "Ensure that your fine-tuning dataset is large enough and covers a wide range of song styles, themes, and vocabularies. A diverse dataset can help the model learn more robust and generalizable patterns.\n", "\n", "* **`Adjust fine-tuning hyperparameters`**\n", "\n", "Experiment with different hyperparameters during fine-tuning, such as learning rate, batch size, and number of epochs. Fine-tuning with optimal hyperparameters can help the model better adapt to the song lyrics domain.\n", "\n", "* **`Implement post-processing techniques`**\n", "\n", "As mentioned earlier, applying post-processing techniques to filter out or modify nonsensical parts can help improve the quality of the generated lyrics. This can include removing or replacing specific words, applying language rules, or using semantic similarity measures.\n", "\n", "* **`Use a different model architecture`**\n", "\n", "If the TinyLlama model consistently produces nonsensical output even after fine-tuning, it may be worth exploring alternative model architectures that are better suited for creative text generation, such as transformer-based models like GPT-2 or GPT-3.\n", "\n", "* **`Combine multiple approaches`**\n", "\n", "Integrating fine-tuned models with other techniques, such as template-based generation, rule-based generation, or retrieval-based generation, can help provide a framework or structure for the generated lyrics while leveraging the model's creativity.\n", "\n", "* **`Iterative refinement`**\n", "\n", "Treat the generated lyrics as a starting point and iteratively refine them through human curation and editing. Identify the most promising parts, make necessary modifications, and use them as inspiration for further generation or manual refinement." ], "metadata": { "id": "DbsHCu_93EDR" } } ] }