The competition among companies to develop advanced artificial intelligence is intense. To gain an edge against rivals like OpenAI’s GPT-4 and Meta’s Llama 2, Google has launched its new AI model called Gemini. This model has been built from scratch and is capable of handling multiple types of data at once, including text, code, audio, images, and video. The AI model will be available in three different sizes: Ultra, Pro, and Nano, for tackling complex tasks, scaling across a broad range of tasks, and on-device tasks, respectively. Gemini is the first model from Google DeepMind after the merger of the company’s AI research units, DeepMind, and Google Brain.
Late last year, OpenAI’s ChatGPT, which is powered by GPT-3.5, was launched and created quite a buzz around the world. Initially caught off guard, Google is now preparing itself for competition. It recently launched Gemini, which has made some bold claims, but questions have been raised about its accuracy. Google DeepMind claims that Gemini outperforms GPT-4 on 30 of the 32 standard performance measures, though the margins are narrow. Despite presenting a futuristic vision to the public, accuracy concerns are now at the forefront.
Gemini, a language model developed by Google, scored an impressive 90 percent on the Massive Multitask Language Understanding (MMLU) test, surpassing human experts (89.8 percent) and outperforming GPT-4 (86.4 percent) in several benchmark tests conducted by Google. MMLU uses 57 subjects, such as math, physics, history, law, medicine, and ethics, to evaluate both problem-solving abilities and world knowledge.
It’s worth noting that Google used different prompting techniques for the two models. While GPT-4’s score of 86.4 percent relied on the industry-standard “5-shot” prompting technique, Gemini Ultra’s 90 percent result was based on a different method— “chain-of-thought with 32 samples.”
Also, it’s important to mention that Google conducted these tests using an outdated version of GPT-4, as indicated in the yellow box in the image above. The note mentions that they used a “previous state-of-the-art” (SOTA) version for GPT-4.
When the 5-shot MMLU was used to evaluate both models, GPT-4 achieved a score of 86.4 percent, surpassing Gemini Ultra’s score of 83.7 percent. Additionally, when the 10-shot HellaSwag was used to measure commonsense reasoning, GPT-4 outperformed both Gemini Ultra (87.8 percent) and Gemini Pro (84.7 percent) by scoring 95.3 percent.
In machine learning, the term “shot” indicates the number of instances or examples used during training. For instance, in few-shot learning, the model is trained on a small number of examples per class, and the number, such as 5 shots, indicates that the model is trained with only five instances of each class.
Chain of thought (CoT) refers to the logical progression or sequence of steps that an AI model follows to arrive at a decision or output. In other words, CoT prompting involves guiding the model to think step by step before generating an answer.
Google used different prompting techniques for benchmarks like GSM8K, DROP, and HumanEval. Their Gemini Ultra model achieved the highest accuracy by using a chain-of-thought prompting approach. This involves generating a sequence of responses using multiple samples and considering model uncertainty. If a consensus is reached, it selects that answer, otherwise, it resorts to a greedy sample.
What the heck?
Did you know that Google used different techniques to test their AI’s ability to reason in different subjects like math, reading comprehension, and coding? The best-performing approach was the “chain-of-thought” method used in their Gemini Ultra model. This approach generates a sequence of responses using multiple samples and considers model uncertainty before making a decision. If there’s a clear consensus, it selects that answer, otherwise, it uses a greedy approach to find the best answer. Pretty cool, huh?
The op-ed by Olson clarified that the voice in the demo was simply reciting predetermined prompts that humans had fed into Gemini, while also displaying static images. It is important to note that this is different from what Google appeared to be suggesting, which was that individuals could engage in a seamless, real-time conversation with Gemini as it observed and responded to its surroundings. In the demo video, Google had reduced latency and shortened Gemini’s outputs for the sake of brevity. Oriol Vinyals, Google DeepMind’s VP of Research and co-lead of Gemini stated in an X post that he was pleased with the attention garnered by the “Hands-on with Gemini” video. Vinyals confirmed that all of the user prompts and outputs shown in the video were authentic, but abbreviated for the sake of concision. The video served as an example of what multimodal user experiences developed with Gemini might look like and was intended to inspire developers.
Also Read | Satellite Communication on Smartphones
125 comments
linetogel
wow, amazing
Spectacular, keep it up
Spectacular, keep it up
Outstanding, superb effort
nice content!nice history!! boba 😀
Terrific, continue
nice content!nice history!! boba 😀
nice content!nice history!! boba 😀
wow, amazing
nice content!nice history!! boba 😀
nice content!nice history!! boba 😀
nice content!nice history!! boba 😀
nice content!nice history!! boba 😀
wow, amazing
wow, amazing
nice content!nice history!! boba 😀
Outstanding feature
wow, amazing
wow, amazing
wow, amazing
nice content!nice history!! boba 😀
wow, amazing
Fantastic job
Splendid, excellent work
Excellent effort
Super, fantastic
Incredible, well done
wow, amazing
wow, amazing
nice content!nice history!! boba 😀
nice content!nice history!! boba 😀
wow, amazing
nice content!nice history!! boba 😀
nice content!nice history!! boba 😀
hello
wow, amazing
wow, amazing
nice content!nice history!! boba 😀
lost money
scam
lost money
lost money
lost money
phising
lost money
phising
phising
phising
scam
lost money
phising
lost money
Outstanding feature
phising
lost money
phising
phising
I highly advise to avoid this platform. My personal experience with it was nothing but frustration and concerns regarding scamming practices. Proceed with extreme caution, or better yet, look for an honest platform to fulfill your requirements.
I highly advise to avoid this platform. The experience I had with it was only frustration along with suspicion of fraudulent activities. Proceed with extreme caution, or even better, look for an honest platform for your needs.
I urge you steer clear of this platform. The experience I had with it has been only frustration as well as suspicion of fraudulent activities. Proceed with extreme caution, or better yet, seek out a more reputable platform to meet your needs.
I urge you to avoid this platform. My personal experience with it has been purely frustration and concerns regarding fraudulent activities. Proceed with extreme caution, or alternatively, look for a trustworthy site for your needs.
I highly advise to avoid this platform. My personal experience with it was only disappointment and suspicion of scamming practices. Exercise extreme caution, or even better, find an honest service to fulfill your requirements.
I urge you to avoid this site. My personal experience with it was only disappointment and concerns regarding scamming practices. Exercise extreme caution, or better yet, look for a more reputable site for your needs.
I urge you to avoid this site. My own encounter with it was purely disappointment along with concerns regarding scamming practices. Be extremely cautious, or alternatively, look for a trustworthy service to fulfill your requirements.
I strongly recommend to avoid this site. My personal experience with it was nothing but disappointment and concerns regarding deceptive behavior. Be extremely cautious, or even better, find an honest site for your needs.
I highly advise to avoid this platform. My personal experience with it has been only dismay along with suspicion of scamming practices. Proceed with extreme caution, or alternatively, find a more reputable service for your needs.
I strongly recommend steer clear of this site. My own encounter with it was only disappointment as well as suspicion of fraudulent activities. Exercise extreme caution, or even better, find an honest site for your needs.
I highly advise stay away from this platform. My own encounter with it has been only dismay and suspicion of fraudulent activities. Exercise extreme caution, or alternatively, look for an honest service to fulfill your requirements.
I strongly recommend to avoid this site. The experience I had with it was only frustration as well as doubts about fraudulent activities. Exercise extreme caution, or better yet, look for a trustworthy platform to meet your needs.
I strongly recommend to avoid this site. The experience I had with it was purely dismay along with concerns regarding deceptive behavior. Proceed with extreme caution, or better yet, find an honest platform for your needs.
I highly advise to avoid this platform. The experience I had with it was purely frustration as well as suspicion of deceptive behavior. Exercise extreme caution, or alternatively, look for an honest platform for your needs.
Thanks for every other informative web site. Where else may just I am getting that type of information written in such an ideal manner? I’ve a mission that I’m simply now working on, and I have been at the look out for such info.
I strongly recommend to avoid this platform. The experience I had with it was nothing but dismay as well as suspicion of fraudulent activities. Proceed with extreme caution, or even better, look for a trustworthy service for your needs.I urge you steer clear of this site. The experience I had with it has been purely disappointment as well as doubts about fraudulent activities. Proceed with extreme caution, or alternatively, look for a more reputable platform to meet your needs.
HerpaGreens is a novel dietary supplement created to help destroy the herpes simplex virus
I strongly recommend steer clear of this site. My personal experience with it has been nothing but disappointment and suspicion of scamming practices. Proceed with extreme caution, or alternatively, find a trustworthy service to fulfill your requirements.
I like forgathering useful info, this post has got me even more info! .
Thank you a lot for giving everyone an extremely pleasant opportunity to check tips from this blog. It is usually so beneficial plus jam-packed with a great time for me personally and my office colleagues to search your web site really three times in one week to find out the new tips you will have. And of course, I’m at all times impressed with all the special principles you give. Certain 1 tips in this posting are certainly the most effective I’ve ever had.
I strongly recommend to avoid this site. My own encounter with it has been nothing but dismay and doubts about deceptive behavior. Proceed with extreme caution, or even better, find a trustworthy platform to meet your needs.
I highly advise stay away from this platform. My own encounter with it has been purely dismay along with doubts about scamming practices. Exercise extreme caution, or better yet, look for an honest site to meet your needs.
I strongly recommend stay away from this platform. My personal experience with it has been purely dismay along with doubts about scamming practices. Proceed with extreme caution, or even better, seek out an honest site to fulfill your requirements.
What Is Sugar Defender?Sugar Defender is a new blood sugar-balancing formula that has been formulated using eight clinically proven ingredients that work together to balance sugar levels.
What is ZenCortex? ZenCortex is a cutting-edge dietary supplement meticulously crafted to provide essential nutrients that support and enhance healthy hearing
I highly advise stay away from this platform. My personal experience with it was purely dismay as well as doubts about deceptive behavior. Exercise extreme caution, or even better, look for a trustworthy service to meet your needs.
I do accept as true with all the ideas you have offered on your post. They are really convincing and will definitely work. Nonetheless, the posts are very brief for starters. Could you please lengthen them a bit from subsequent time? Thank you for the post.
I strongly recommend stay away from this site. My own encounter with it has been purely disappointment as well as doubts about scamming practices. Be extremely cautious, or even better, find a trustworthy platform for your needs.
I strongly recommend stay away from this platform. My personal experience with it has been only frustration along with suspicion of scamming practices. Proceed with extreme caution, or alternatively, seek out a more reputable service to fulfill your requirements.
I strongly recommend stay away from this platform. My own encounter with it has been purely frustration as well as doubts about scamming practices. Exercise extreme caution, or better yet, find an honest platform to meet your needs.
I loved as much as you will receive carried out right here. The sketch is attractive, your authored material stylish. nonetheless, you command get bought an shakiness over that you wish be delivering the following. unwell unquestionably come further formerly again as exactly the same nearly very often inside case you shield this hike.
This is a topic close to my heart cheers, where are your contact details though?
This website is my inhalation, rattling good layout and perfect content.
I urge you stay away from this site. My personal experience with it was purely frustration as well as suspicion of deceptive behavior. Proceed with extreme caution, or even better, look for an honest site to fulfill your requirements.
Lottery Defeater is an automated, plug-and-plug lottery-winning software.
What is Leanbiome? LeanBiome is a dietary supplement that is formulated with nine critically-researched lean bacteria species.
I gotta bookmark this site it seems extremely helpful extremely helpful
I urge you steer clear of this site. The experience I had with it was nothing but frustration along with concerns regarding fraudulent activities. Proceed with extreme caution, or alternatively, find a more reputable platform to meet your needs.
I urge you steer clear of this site. My personal experience with it was nothing but disappointment and suspicion of fraudulent activities. Proceed with extreme caution, or even better, find a more reputable site to meet your needs.
I urge you steer clear of this platform. My personal experience with it was nothing but disappointment along with concerns regarding deceptive behavior. Proceed with extreme caution, or alternatively, seek out a trustworthy site to fulfill your requirements.
Nice weblog right here! Also your website rather a lot up fast! What web host are you using? Can I get your affiliate hyperlink to your host? I desire my website loaded up as quickly as yours lol
I haven?¦t checked in here for some time as I thought it was getting boring, but the last few posts are good quality so I guess I?¦ll add you back to my daily bloglist. You deserve it my friend 🙂
But wanna input that you have a very nice internet site, I like the design and style it really stands out.
Fantastic web site. Plenty of helpful information here. I am sending it to several buddies ans also sharing in delicious. And naturally, thanks to your effort!
I truly appreciate this post. I have been looking everywhere for this! Thank goodness I found it on Bing. You’ve made my day! Thank you again
My brother recommended I would possibly like this web site. He was entirely right. This post actually made my day. You can not imagine just how so much time I had spent for this information! Thanks!
Usually I don’t read post on blogs, but I would like to say that this write-up very forced me to try and do it! Your writing style has been amazed me. Thanks, very nice post.
What is Renew? Renew is a dietary supplement designed to support blood flow while also aiming to boost testosterone levels andprovide an explosive energy drive
Thanks , I have recently been searching for info about this subject for a while and yours is the best I have discovered till now. However, what in regards to the bottom line? Are you positive in regards to the source?
I reckon something really special in this website .
Dead indited written content, thanks for selective information.
Hello! I could have sworn I’ve been to this blog before but after browsing through some of the post I realized it’s new to me. Anyways, I’m definitely happy I found it and I’ll be book-marking and checking back frequently!
Only wanna state that this is very beneficial, Thanks for taking your time to write this.
Hey there! Do you know if they make any plugins to safeguard against hackers? I’m kinda paranoid about losing everything I’ve worked hard on. Any tips?
Yay google is my world beater aided me to find this outstanding site! .
I like this post, enjoyed this one regards for posting.
Hi, Neat post. There’s a problem together with your web site in internet explorer, could check this… IE still is the market leader and a huge element of people will omit your excellent writing due to this problem.
What Is FitSpresso? The effective weight management formula FitSpresso is designed to inherently support weight loss. It is made using a synergistic blend of ingredients chosen especially for their metabolism-boosting and fat-burning features.
What Is Sugar Defender? Sugar Defender is a meticulously crafted natural health supplement aimed at helping individuals maintain balanced blood sugar levels. Developed by Jeffrey Mitchell, this liquid formula contains 24 scientifically backed ingredients meticulously chosen to target the root causes of blood sugar imbalances.
I am typically to running a blog and i really admire your content. The article has actually peaks my interest. I am going to bookmark your web site and maintain checking for brand spanking new information.
Way cool, some valid points! I appreciate you making this article available, the rest of the site is also high quality. Have a fun.
Just desire to say your article is as astonishing. The clarity in your post is simply nice and i can assume you’re an expert on this subject. Well with your permission allow me to grab your feed to keep up to date with forthcoming post. Thanks a million and please carry on the enjoyable work.
I?¦ve been exploring for a bit for any high quality articles or blog posts on this kind of house . Exploring in Yahoo I finally stumbled upon this site. Reading this information So i am satisfied to convey that I’ve an incredibly good uncanny feeling I discovered exactly what I needed. I such a lot no doubt will make sure to don?¦t fail to remember this website and provides it a look on a relentless basis.
Well I really enjoyed reading it. This information procured by you is very effective for correct planning.
obviously like your web-site but you need to check the spelling on several of your posts. Many of them are rife with spelling issues and I to find it very troublesome to tell the reality then again I’ll surely come again again.
Hi my loved one! I wish to say that this post is amazing, great written and include approximately all vital infos. I’d like to look more posts like this .
I’m not sure why but this web site is loading incredibly slow for me. Is anyone else having this problem or is it a problem on my end? I’ll check back later on and see if the problem still exists.