ChatGPTExperiences

Context: ChatGPT

This page for recording my current experiences.

Some of my most extraordinary ChatGPT sessions:

• ChatGPT explains the LightningNetwork to me https://chat.openai.com/share/44a2e66a-0db0-4922-a924-66bf2e70c804
• ChatGPT helps me try to reverse engineer MuProc files : https://chat.openai.com/share/862e7058-8a2b-4173-95c6-dc627802bf53
• We discuss the possibility of creating a single version of a library and translating it into multiple programming languages: https://chat.openai.com/share/168ff65e-8bf6-4976-89bf-3eb76b008580
• We investigate ThreadDescriptionLanguage for ArgumentMapping : https://chat.openai.com/c/c35dac98-51e8-45cb-8e87-3f25d10df53d
• It writes me a tool I've wanted for decades! https://chat.openai.com/share/b8fb90d8-41ae-4595-a5c2-84a624612c1e

I've been using ChatGPT to write code for a while now.

Here are a few observations.

What it's really good at is translating code from one language to another. It's successfully translated a small CoffeeScript app into PythonLanguage for me, doing about 90% of the work. I've had it translate Python into Java. Python into Clojure. Etc. And it doesn't make many mistakes at that level. Though you have to feed it smallish chunks at a time.

When asked to do small, well defined coding tasks it's OK. I've had it write me a Scrapy script to scrape a website and pull the images down from it. I've had it write autocomplete for a field on a web-form in ClojureScript. And various other small tasks. It's OK at this.

It's pretty good as a kind of front-end to boring-to-read documentation. You can ask it a direct question about, say, the right method to use for something in an Android app, and it gives good pointers. (Given I find the Android documentation horrendously hard to navigate, this is a big win)

It's pretty good as a kind of "rubber duck" to talk to about your problems. If you describe a programming problem and say "why might that be" it can give insightful ideas. It won't solve the problem for you but can help you think through the problem.

Where it's not so great is the bigger picture. For example, it's not good at knowing if a solution it's giving you is deprecated. Or even in joining up knowledge it has in one context and another.

I had one case of asking it to help me write a Processing sketch to make music visualizers by reading in an MP3 file and then recording it with an animation into a video file.

It said something along the lines of "sure, here's code for that, using the MovieMaker class". I tried using the code GPT wrote, there's no MovieMaker class. I ask where it is and we go through a couple of rounds of me telling it that the MovieMaker class is available in X library". Or maybe the Y library. I establish that there is no MovieMaker class. I ask ChatGPT if MM really exists and it tells me that it was taken out of Processing in Processing 3.0. I ask why; it knows that the MovieMaker class was removed because it depended on an open-source QuickTime library that hadn't been maintained for years.

So it DOES know there's no MovieMaker class. And when and why it was removed. But that doesn't stop it trying to use that class in code. And making random guesses about where it can be found, until I force it to admit it's not there.

So I ask for an alternative. It suggests using the OpenCV library which IS still available in the Processing library.

Somehow it veers into Python and starts writing Python OpenCV code. I remind it we're in (Java) Processing. It writes me Java Processing code to record the animation to a video with OpenCV. However I notice it's not writing the music into the video, just playing the music while writing the file.

So I ask GPT to actually put the audio into the video file too. It says "sure ... here's how to put the audio into the video file". It starts writing the code then freezes.

I ask it to try again and it freezes again.

Eventually I ask if it's possible to use this library to incorporate audio into the video file it's making. Ah, no. "Sorry about that" it admits. The OpenCV library for making videos doesn't handle audio (Obviously. It's a computer vision library). Again ChatGPT "knows" this when asked, but can't use that knowledge when generating the code.

I ask it for another solution. And it goes back to using the MovieMaker class. ;-|

Another way it led me up the garden path a couple of days ago. I wanted to write some code to check broken links from my site. I had this great idea that the code for checking for broken links could itself run in the browser, alerting me if any of the links on a page were broken. After a bit of back and forth, Chat helped me write the code.

I run the code. And get a CORS error. For those who know anything about this, it's OBVIOUS that you can't really write code to check for broken links in the browser, because CORS is going to prevent an application that came from one server hitting any other web sites. That's part of the security protection of the web.

But Chat happily goes along with me for an hour helping me write this useless code that can't possibly do what it's meant to. Again because it has no wider perspective.

So, I think ChatGPT is phenomenal. I've just signed up to the paid tier. I think it's very useful. But you have to be VERY careful. You can't use it to write things you don't already understand and can't eyeball the correctness of. It WILL go wrong at the drop of a hat and unless you are able to spot that immediately, it can waste you a whole lot of time.