GPT-3 Is A Coding Moron, is GPT-4 Any Better?

[Roli]

After spending some time with GPT I have come to understand its limitations. I've mainly been using it for character and place names for my books (also fairly reasonable at coming up with pen names), however recently I reignited my interest in coding (still have ideas burning a place in my skull) so I decided to see if GPT could code some stuff for me.

I decided to test it in the graphics program Blender, which if you don't know is an extremely powerful 3D/Animation opensource software which uses Python.

I asked it a fairly simple question - "code me a generic looking flower"

To say the results were awful, is being extremely kind to GPT, apart from the fact it couldn't understand that the 'diameter' argument is not used in Blender, the final result (which took about thirty-five minutes of repeating myself and wishing GPT was a person so I could slap him) was a floating sphere with a one "atom" thick ring around it. #facepalm.

Next I thought okay, let's go simple, code me a simple head, after which, my head hurt... a lot.

I won't paste the code, but here's a snippet of my infuriating convo with it.

Me - Okay, let's see how you do coding a generic human head shape. Please remember not to use any 'diameter' arguments and please remember to use 4 vector functions for any mat.diffuse lines or similar.

GPT-3 -
Sure! Here's an example of how you could create a basic human head shape:

Code

This code creates a new mesh object for the head, adds a UV sphere to a bmesh, applies proportional editing to shape the head, and then creates a new mesh from the bmesh and sets it as the mesh for the head object. It also sets the color of the head material to a light skin tone using RGBA vector values, and positions the head in the scene and adds it to the scene.

(It did none of those things)

Me - The code returned the following error Python: Traceback (most recent call last): File "\Text", line 23, in <module> ValueError: bpy_struct: item.attr = val: sequences of dimension 0 should contain 3 items, not 4

GPT-3 - I apologize for the error. This might have been caused by using a 4-dimensional vector instead of a 3-dimensional one. Here's the corrected code:

(note: I'd already told it to put in 4 values for something specific, but hey, possibly my bad.)

This code creates a head mesh using a UV sphere, and two eye meshes using circles that are translated to the appropriate positions. The materials for the head and eyes are also set, with the head material having a skin tone and the eyes material being black.

Me - Please rewrite the code without any 'diameter' arguments. (As I mentioned in the first goddamn place!)

GPT-3 - Sure, here's a revised version of the code for creating a generic human head shape without using any diameter arguments:

code

This code uses create_icosphere instead of create_uvsphere to create the head shape without using any diameter arguments. I also added a from mathutils import Vector statement to use the Vector function to set the color of the head material with four float values (R, G, B, alpha).


(Proceeds to make human head so small I can't even zoom in on it.)

Me -
I can hardly see it, rewrite the above code EXACTLY, except make the head ten times bigger

GPT-3 -
Sure! Here's the code with a ten times bigger head:

code

I increased the diameter of the UV sphere to 1.0 to start with, then scaled the head by a factor of 10 at the end using the scale property of the head_obj.

Me - (For the love of God)
rewrite the code without any 'diameter' arguments, remember Blender does not use them any more (like I said in the first f***ing place! Plus rewrite exactly means do the same thing again you moron!)


GPT-3 -
Sure, here's the rewritten code without any 'diameter' arguments:

(I will show you one line of the code it came up with)

bmesh.ops.create_uvsphere(bm, u_segments=32, v_segments=16, diameter=1)

Like, what the actual???? It was like having a conversation with a genius with advanced Alzheimer's.

I even asked it if it could read the changelog if I linked it, GPT said "sure, send me the link." I sent the link and then asked it if it understood what had been put in place instead of the 'diameter' argument.

Yes it said, then proceeded to produce the code above, with the defunct 'diameter' argument.


(Even though this is a tiny fraction of my conversation with GPT-3), this is all a very long way of saying, has anyone tried GPT-4 for coding yet? Is it better than its moronic cousin GPT-3? Or will it just give me the wrong answer faster?

 

Dislike ads? Remove them and support the forum: Subscribe to Fastlane Insiders.

[MarcusRich]

After spending some time with GPT I have come to understand its limitations. I've mainly been using it for character and place names for my books (also fairly reasonable at coming up with pen names), however recently I reignited my interest in coding (still have ideas burning a place in my skull) so I decided to see if GPT could code some stuff for me.

I decided to test it in the graphics program Blender, which if you don't know is an extremely powerful 3D/Animation opensource software which uses Python.

I asked it a fairly simple question - "code me a generic looking flower"

To say the results were awful, is being extremely kind to GPT, apart from the fact it couldn't understand that the 'diameter' argument is not used in Blender, the final result (which took about thirty-five minutes of repeating myself and wishing GPT was a person so I could slap him) was a floating sphere with a one "atom" thick ring around it. #facepalm.

Next I thought okay, let's go simple, code me a simple head, after which, my head hurt... a lot.

I won't paste the code, but here's a snippet of my infuriating convo with it.

Me - Okay, let's see how you do coding a generic human head shape. Please remember not to use any 'diameter' arguments and please remember to use 4 vector functions for any mat.diffuse lines or similar.

GPT-3 -
Sure! Here's an example of how you could create a basic human head shape:

Code

This code creates a new mesh object for the head, adds a UV sphere to a bmesh, applies proportional editing to shape the head, and then creates a new mesh from the bmesh and sets it as the mesh for the head object. It also sets the color of the head material to a light skin tone using RGBA vector values, and positions the head in the scene and adds it to the scene.

(It did none of those things)

Me - The code returned the following error Python: Traceback (most recent call last): File "\Text", line 23, in <module> ValueError: bpy_struct: item.attr = val: sequences of dimension 0 should contain 3 items, not 4

GPT-3 - I apologize for the error. This might have been caused by using a 4-dimensional vector instead of a 3-dimensional one. Here's the corrected code:

(note: I'd already told it to put in 4 values for something specific, but hey, possibly my bad.)

This code creates a head mesh using a UV sphere, and two eye meshes using circles that are translated to the appropriate positions. The materials for the head and eyes are also set, with the head material having a skin tone and the eyes material being black.

Me - Please rewrite the code without any 'diameter' arguments. (As I mentioned in the first goddamn place!)

GPT-3 - Sure, here's a revised version of the code for creating a generic human head shape without using any diameter arguments:

code

This code uses create_icosphere instead of create_uvsphere to create the head shape without using any diameter arguments. I also added a from mathutils import Vector statement to use the Vector function to set the color of the head material with four float values (R, G, B, alpha).


(Proceeds to make human head so small I can't even zoom in on it.)

Me -
I can hardly see it, rewrite the above code EXACTLY, except make the head ten times bigger

GPT-3 -
Sure! Here's the code with a ten times bigger head:

code

I increased the diameter of the UV sphere to 1.0 to start with, then scaled the head by a factor of 10 at the end using the scale property of the head_obj.

Me - (For the love of God)
rewrite the code without any 'diameter' arguments, remember Blender does not use them any more (like I said in the first f***ing place! Plus rewrite exactly means do the same thing again you moron!)


GPT-3 -
Sure, here's the rewritten code without any 'diameter' arguments:

(I will show you one line of the code it came up with)

bmesh.ops.create_uvsphere(bm, u_segments=32, v_segments=16, diameter=1)

Like, what the actual???? It was like having a conversation with a genius with advanced Alzheimer's.

I even asked it if it could read the changelog if I linked it, GPT said "sure, send me the link." I sent the link and then asked it if it understood what had been put in place instead of the 'diameter' argument.

Yes it said, then proceeded to produce the code above, with the defunct 'diameter' argument.


(Even though this is a tiny fraction of my conversation with GPT-3), this is all a very long way of saying, has anyone tried GPT-4 for coding yet? Is it better than its moronic cousin GPT-3? Or will it just give me the wrong answer faster?

I subscribe to Plus and have used GPT-4 while consulting and let me just say... its different. I wouldn't say its is better or worse at coding but I would say it is a lot better at infuring what you might leave out of requirements. I have also used screen text grab and tried a lot of code examples from PluralSight and CodeWars and can say those it absolutely destroys in a positive way with very fast response.

 

[Roli]

I subscribe to Plus and have used GPT-4 while consulting and let me just say... its different. I wouldn't say its is better or worse at coding but I would say it is a lot better at infuring what you might leave out of requirements. I have also used screen text grab and tried a lot of code examples from PluralSight and CodeWars and can say those it absolutely destroys in a positive way with very fast response.

Interesting, thank you. Can I ask, what level of coding are you at?

 

[MarcusRich]

Interesting, thank you. Can I ask, what level of coding are you at?

Roli, I am of Software Architect with 12+ years of experience using OO and Functional languages. (In developer terms I have been around doing enterprise software since the Monolith was considered a solid paradigm lol) (Additionally, if you wish to test it in a similar manner I leveraged CleanShot X and the text grab feature then paste that and do minor corrections)

 

Dislike ads? Remove them and support the forum: Subscribe to Fastlane Insiders.

[Roli]

Roli, I am of Software Architect with 12+ years of experience using OO and Functional languages. (In developer terms I have been around doing enterprise software since the Monolith was considered a solid paradigm lol) (Additionally, if you wish to test it in a similar manner I leveraged CleanShot X and the text grab feature then paste that and do minor corrections)

That's what I thought, it seems like if you have experience then GPT is a good tool, however if you are like me, with only a very rudimentary understanding, forget it!

 

[srodrigo]

Man, It's just $20 for one month, get it and try it out.

 

[SharpeningBlade]

It's significantly better but the quality of its output will rely on your knowledge and abilities. If you're a total noob it can do some things but the more complexity you add, the more you need to actually know what you're doing.

It's not going to be very useful unless you give it very specific requirements and you're still going to need to mess around with it, tweak it, try different prompts, etc.

I've been working on a next.js and tailwind app recently and it's been helpful for some aspects of it but not all.

Give it a shot, it's only $20/month as the user above said. For that price, it's incredibly worth it.

I use it often for coding productivity and also any random things, for example yesterday I was reading about how Van Gogh survived on 3.5 francs per day and I wanted to get the conversion rate between 1888 francs and 2021 US dollars and it came out to about $19.50 per day.

 

Dislike ads? Remove them and support the forum: Subscribe to Fastlane Insiders.

[MarcusRich]

Something you might find valuable if you’re attempting to learn leveraging GPT 3 or 4 is doing a daily kata on CodeWars or similar sight and once you reach your end in expertise use a text grab and see what the bot has to say then continue.

 

[Roli]

Man, It's just $20 for one month, get it and try it out.

The problem is, I have ADHD and cognitively I can't unsubscribe from stuff, so I only ever sign up to things I'm happy to have for years.

I discover subscriptions I've forgotten about on an almost monthly basis.

So if it's good, I'll do the $20 and forget it about it.

 

[Andy Black]

The problem is, I have ADHD and cognitively I can't unsubscribe from stuff, so I only ever sign up to things I'm happy to have for years.

I discover subscriptions I've forgotten about on an almost monthly basis.

So if it's good, I'll do the $20 and forget it about it.

I have a Google sheet listing things I subscribe to. I even try to keep it up to date. Have you tried something like that?

 

Dislike ads? Remove them and support the forum: Subscribe to Fastlane Insiders.

[Roli]

I have a Google sheet listing things I subscribe to. I even try to keep it up to date. Have you tried something like that?

That's the sort of thing I would get into enthusiastically at the start (novelty), but would forget about within 6 months.

Starting to do my head in actually, I'm on the waiting list to get seen, I'm trying to get some Elvanse from the dark web. I took some of my friend's and my god! I suddenly realised how normal people must think, almost brought me to tears.

Anyway, I find the best solution is not to subscribe to things unless I'm 100% sure I'm going to like it. It's the same reason I don't buy clothes from the internet, because regardless of how badly they fit, I will not send them back.

 

[Andy Black]

That's the sort of thing I would get into enthusiastically at the start (novelty), but would forget about within 6 months.

Starting to do my head in actually, I'm on the waiting list to get seen, I'm trying to get some Elvanse from the dark web. I took some of my friend's and my god! I suddenly realised how normal people must think, almost brought me to tears.

Anyway, I find the best solution is not to subscribe to things unless I'm 100% sure I'm going to like it. It's the same reason I don't buy clothes from the internet, because regardless of how badly they fit, I will not send them back.

I don't know if we can ever be 100% sure we're going to like something before we try it.

Would it help to have a recurring monthly calendar event that tells you to check your credit card for new payments that you need to list on your Google sheet?

I suppose you could always cancel your credit card each year and update every subscription that fails?

 

[Simon Angel]

The problem is, I have ADHD and cognitively I can't unsubscribe from stuff, so I only ever sign up to things I'm happy to have for years.

I discover subscriptions I've forgotten about on an almost monthly basis.

So if it's good, I'll do the $20 and forget it about it.

That's a limiting belief. You're just disorganized.

Not to say that it doesn't happen to me. It does, but that's all on me because I could keep track of my subscriptions somewhere and set up reminders to unsubscribe.

 

Dislike ads? Remove them and support the forum: Subscribe to Fastlane Insiders.

[Roli]

I suppose you could always cancel your credit card each year and update every subscription that fails?

I probably lose a card once every few years, this is when I discover my subscriptions!

I don't know if we can ever be 100% sure we're going to like something before we try it.

True, however if I'm in a shop, I can take the option of not buying it, whereas if I get it delivered, it's just never going to happen, I'll keep it. I buy stuff online, but only things with massive reviews, no reviews, no Roli.

Would it help to have a recurring monthly calendar event that tells you to check your credit card for new payments that you need to list on your Google sheet?

Been there.

It's really hard to explain to someone with normal thought processes, but the above is actually a nightmare for me, I've tried it....

Imagine an old style radio being tuned from channel to channel, buzzing in and out of reception, interference, talking, music, singing, news, you can hear snippets of the lot as you tune in and out.

That's my head every single day.

I meditate and take Modafinil to try and quiet it down and get on with stuff, but I need the proper drugs. Like I said I tried Elvanse and it was amazing. But I'm on a waiting list, if I could afford to go private I would.

Oh well, enough moaning for the day! Onwards and upwards!

 

[Roli]

That's a limiting belief. You're just disorganized.

Not to say that it doesn't happen to me. It does, but that's all on me because I could keep track of my subscriptions somewhere and set up reminders to unsubscribe.

Oh Doctor! I didn't realise you were on here. Strange to give me a diagnosis via this forum, surely your office would be better?

I suppose you are busy.

Take care.

 

[Simon Angel]

Oh Doctor! I didn't realise you were on here. Strange to give me a diagnosis via this forum, surely your office would be better?

I suppose you are busy.

Take care.

Have fun with your external locus and learned helplessness.

 

Dislike ads? Remove them and support the forum: Subscribe to Fastlane Insiders.

[Roli]

Have fun with your external locus and learned helplessness.

Have fun with your arrogance.

 

[Roli]

Have fun with your external locus and learned helplessness.

Listen, I don't want any animosity between us, as far as I can remember we've had good interactions on here over the years. The reason I reacted with sarcasm to your post is when it comes to ADHD the default setting for people is they think it is made up.

If I said I had schizophrenia or clinical depression you wouldn't react by telling me I was just a bit scatterbrained or needed to cheer up because you recognise them as genuine mental disorders.

I think a lot of this comes from the guy who did those studies and said he couldn't find any evidence that adult ADHD existed, however just a couple of years later, he did some meta studies and realised he was wrong. He said in his original analysis he used metrics for children and his work was flawed.

He has posted many rebuttals to his own work, however human nature is such people only remember the first thing he said and stick with that.

Anyway, let's not have a medical argument/discussion in a thread whereby I was trying to evaluate GPT-4.

Peace 'n' love.

 

[John Clancy]

@Roli if you'd like to be able to sign up for ChatGPT Premium without needing to remember to cancel your sub before they renew... here are two solutions you might be interested in:

1. I see that the ChatGPT Premium uses Link, which is a product from Stripe. Unless there's something different with their setup, you should be able to cancel your subscription immediately after checkout and still retain access until the end of your billing period. I.e. sign up today to the monthly plan, cancel immediately, use for 30 days until your subscription expires.
2. Using a mobile-first banking service like Revolut (not sure how big they are internationally, but they're popular in Europe), you can create one-time use virtual cards . It's an extra step in the process, but if you're signing up for something that you definitely don't want to renew, something like this would ensure that you can only be charged once.

Just two general ideas in response to your points above that would remove the need for you to remember to cancel your plan. Either would let you test out the service on a one-month basis for $20. Might be worth it if you'd like to give GPT-4 a spin.

Edit: also, I have my own "coding with ChatGPT" horror stories, so I feel your pain lol.

 

Dislike ads? Remove them and support the forum: Subscribe to Fastlane Insiders.

[srodrigo]

The problem is, I have ADHD and cognitively I can't unsubscribe from stuff, so I only ever sign up to things I'm happy to have for years.

I discover subscriptions I've forgotten about on an almost monthly basis.

So if it's good, I'll do the $20 and forget it about it.

Yeah. What I do is I set up a reminder on Google Calendar that repeats for a few days, multiple times a day if needed. And I set it up even before I purchase. That helps a bit.

EDIT: I've just noticed that you talk about setting up reminders on a reply to another user. Not sure then.

 

[Simon Angel]

Listen, I don't want any animosity between us, as far as I can remember we've had good interactions on here over the years. The reason I reacted with sarcasm to your post is when it comes to ADHD the default setting for people is they think it is made up.

If I said I had schizophrenia or clinical depression you wouldn't react by telling me I was just a bit scatterbrained or needed to cheer up because you recognise them as genuine mental disorders.

I think a lot of this comes from the guy who did those studies and said he couldn't find any evidence that adult ADHD existed, however just a couple of years later, he did some meta studies and realised he was wrong. He said in his original analysis he used metrics for children and his work was flawed.

He has posted many rebuttals to his own work, however human nature is such people only remember the first thing he said and stick with that.

Anyway, let's not have a medical argument/discussion in a thread whereby I was trying to evaluate GPT-4.

Peace 'n' love.

There's no animosity and I'm not denying that ADHD exists. Any psychiatrist would diagnose me with ADHD too. I'm just saying you can't and shouldn't let that define you.

In fact, they'd diagnose me with OCD and depression as well. And then what?

Pills, mental health "support" groups (basically workshops on acquiring a defeatist attitude), and carrying around labels that harm you in every area of your life.

Saying "I'm more forgetful about this stuff since I'm diagnosed with ADHD" is one thing. But saying that you CAN'T find a simple way to deal with this is just not true.

You could literally set a quick alarm on your phone that says "cancel X subscription TODAY" at the end of the month. I also don't bother with other forms of tracking because it makes me anxious (obviously, this won't make sense to most people because tracking makes them feel LESS anxious). I do it all in my head or via alarms because I hate clutter and like easy solutions. This goes for anything from doing my taxes to updating clients and working on projects.

Anyway, you've got people on one end of the spectrum going "There's no such thing as mental health or depression, you just have to stop being a pussy" which is idiotic. But I can't say I favor the other end where people wallow in their conditions and point to them as reasons why they can't do X thing.

The thing is, nobody gives a crap about our reasons. You either do something or you don't. So you're doing yourself a disservice by hiding behind those labels and automatically "disqualifying" yourself.

Why not be the person with ADHD that does X thing despite the odds being against them? Why not stray from the herd and be extraordinary?

These are my thoughts on the matter and I have no intention of arguing. I hope it helps.

 

[Andreas Thiel]

code me a generic looking flower

Dafuq?

 

Dislike ads? Remove them and support the forum: Subscribe to Fastlane Insiders.

[MarcusRich]

Dafuq?

What is funny about the prompt really is if you say. Using c# create a visualization of a rose. That one works really well in my experience. If you talk to GPT like it is a developer and you’re very precise it will work really well. What I have found is that maximum description while still being as logical as possible yields the greatest value in results.

 

[Spenny]

I'm just saying you can't and shouldn't let that define you.

carrying around labels that harm you in every area of your life.

The thing is, nobody gives a crap about our reasons.

Why not be the person with ADHD that does X thing despite the odds being against them? Why not stray from the herd and be extraordinary?

Top notch mindset. Very well done. It's astonishing the amount of prison cells people voluntarily walk into.