Spaces:

---

IndianAIDevs

/

README

like 3

Running

Has anyone pre-trained LLM model from scratch ? If yes then share your experience, things to consider while training, notes, tips etc.

Hi i am also intrested into LLM Model , i am about to start this reserach from next week please give any inputs

Hi i am also intrested into LLM Model , i am about to start this reserach from next week please give any inputs

Hey @Shashank2k3 , if you want your own LLM model, first you need huge data. You can start with fine tuning already available good LLM models like Gemma, Phi, LLAMA, mistral etc with your dataset. Start with small models of sizes like 4 to 7B parameters. For pre-training LLM from scratch you need enormous data, good resources like heavy duty GPUs and CPUs and also have knowledge of training techniques, NLP, etc . You can always brainstorm with ChatGPT to get more knowledge.

Hey @kalashshah19 , thanks for the input! I already have a solid foundation in these areas from my Bachelor's degree in AIML, and now I’m looking to dive deeper into the world of LLMs.

Hey @kalashshah19 , thanks for the input! I already have a solid foundation in these areas from my Bachelor's degree in AIML, and now I’m looking to dive deeper into the world of LLMs.

Great !

Yupp so what you guys do, i mean profession!!!

Yupp so what you guys do, i mean profession!!!

I am an Associate Data Scientist at Casepoint.
What about you ?

btw this is the second time i got the access last time my access was around 6 months

tumne kharide hai yeh gpus?

Yeah by fundings bruh

yaar yeh bhaut zyda phek diya aapne 1 h100 is around 80lacs

i think it's around 30 lakhs

Nhi yaar h100 are around 80k$ per gpu 30k$ mein toh a100 or ls40 jaise gpus milte hai

tumne kharide hai yeh gpus?

Yeah by fundings bruh

yaar yeh bhaut zyda phek diya aapne 1 h100 is around 80lacs

​Ek minute me H100 ka price 80 lakh se 30 lakh kar diya, aur agle hi minute me owned bare-metal cluster ko third-party cloud credits bana diya! Enterprise cluster manage karna aur RunPod ya Lambda labs par temporary cloud instance rent karne me zameen-aasman ka farq hota hai, mere bhai.

​Shuruat tumne hume 'basic knowledge nahi hai' bol kar chaud me ki thi, phir custom architecture par ghir gaye, uske baad untrained code ko production bol kar fase, aur ab infrastructure par bhi pakde gaye. Chalo, cloud credits milne ki badhai, par agli baar khud ke kharide hue servers bol kar sasta flex mat maarna jab real scale ka pata na ho

aabe woh message nhi kya hai gahde username toh padh lee yeh anpadh hai kya? personally call pr aaj bhai agar bss ke hai toh wala prove krlete hai

Parvesh bhai, technical gyan par ghire toh seedha personal call aur lafda-lafda khelne par utar aaye? 😭😂 Jab chat par code aur infra ka sach jhela nahi gaya, toh ab call par chilla-chilli karke dimaag kharab karna hai kya?

​Username se lekar model tak ka poora logic chat par hi dho diya hai, sabke saamne dikh raha hai ki u-turn kisne maara hai aur kisne untrained code ko research bolkar pichha chhudaya hai. Tumhe agar lagta hai ki call par chilla kar tum computer architecture badal doge ya apne credits ko bare-metal cluster bana loge, toh ye tumhara weham hai.

​Mujhe call par aakar apni knowledge ka certificate lene ki zaroorat nahi hai. Group me baat chal rahi thi, toh gyan dene se pehle repo check karna seekho. Aur haan, Aikosh aur Vultr se minimal price me access chahiye ho toh personally ping kar dena, saste me setup karwa dunga Chalo, take care

yaar tune mujhe brain tumer dediya hai tuu mere messages toh dekhe le pehele ke mein kya keh rha pta nhi tu sabe mix krke aa rha hai keh rha hai 80h100 hai tere pass pagal wagal hai kya thoda?

Parvesh bhai, dimaag thanda rakho, sach sunkar hyper hone ki zaroorat nahi hai! 😂 Tumor tumhe isliye mehsoos ho raha hai kyunki tum ek sath do logon ke technical replies jhel nahi pa rahe ho.
​Username aur logs dhyan se check karo—80 H100 maine nahi, Neural-Hacker ne galti se mix karke likha tha. Mera clear mathematical compute allocation 10x H100 aur 80x A100 ka hai. Jab tum 30-80 Lakh ke price tag me hi darr gaye, toh itne badhe bare-metal cluster ka infrastructure logic tumhare dimaag ke core architecture se crash hona hi tha!

​Rahi baat 2T+ model ki, toh haan, SKT-SURYA-H (2.6T MoE) hamara ek 'Failed Attempt' thha! ❌ Hum real parameters par actual hardware scale testing karte hain, isliye failures accept karne ka jigra bhi rakhte hain. Tumhari tarah untrained dabba model Hugging Face par daal kar 'AI Researcher' ka tag lekar nahi ghumte. Agli baar se bina crash hue messages padhna seekho, warna dimaag ka system aise hi hang hota rahega!

yaar phekne ke hadd hoti hai agar aapke pass cluster hai toh photo bhej do group mein and also tell me why was you returning loss? in config and why was the vocab size wrong there are a lot of more thing that are wrong in it
'''
bhai, lagta hai tumhari memory sach me bohot short hai! Cluster par cloud resources aur allocation ke baare me maine pehle hi clear bol diya tha, isiliye toh tumhe Aikosh, DG aur Vultr se saste me access dilwane ki baat boli thi taaki tumhara bhi thoda bhala ho jaye. Ab jab har baar ghir jaate ho, toh topic kyun badal dete ho? 😂

aare piche tumne hee likha thi ke you bought it through funding?

Now changed Topic well I have Cloud Infrastructure I'll als Told You well and The Finding Proof Go and Check Base Batches 🤣🤣

your messages are being marked as off topic and Low Qualit so i can;t verify things now

Arey Parvesh bhai, Google Meet par aakar alag se screen-share karke dabba logics validate karne ka koi faida nahi hai. Baat group me shuru hui thi, aur group me hi saare technical facts line-by-line clear ho chuke hain.

And the message i hide because there's Quote make me frustrated 🤣

@Shrijanagain @Parveshiiii don't behave like sam and elon
there's no point on fighting for being right, jo galat hoga use baad mein khud pata chal jayega

@Shrijanagain also i would suggest that you are still learning so please don't argue with me

Yeahh until You must Learn Be Updated Don't Argue you Don't have Vaild Knowledge 😅

ok bro you are building a billion-dollar business i should not argue with you and i don't have knowledge this is why iam working as AI-Research and have onboarded MS-Dhoni on my product you are great

MS Dhoni? please elaborate

@Shrijanagain @Parveshiiii don't behave like sam and elon
there's no point on fighting for being right, jo galat hoga use baad mein khud pata chal jayega

Nope We never Fight

@Neural-Hacker
hehehehe he is now one of the first adopter of one of our systems that helps celebs to find out where is their identity being used like memes promotions sometimes people use photos of big celebs without signing official contracts and misleading posts also so we made a system that takes those images video from x, insta and yt and we filter it using our internal VLM and other pipelines to make a very clean filtered info for celebrities and they can take legal actions and more if their identity is being used without concent

that's a nice idea, can u share website link of ur company/lab?