| » Forum Index » General comments » Topic: "The Good, The Bad and The Ugly" what generative fill can and cannot do for you. |
|
Posted on 06/06/23 6:32:49 PM |
|
DavidMac
Director of Photoshop Posts: 5554 Reply  |
"The Good, The Bad and The Ugly" what generative fill can and cannot do for you.
This week Steve posted the following challenge: "A lot of effort has gone into the construction of this flood wall, but so far it seems to have no point. Can you add some water so we can see if it works?" 
I put in several different attempts with the intention of then letting AI have a go when I had finished. I wanted to do all these first myself by hand with no preconceptions of where AI might take afterwards. Here are the results. I must make absolutely clear that the AI images are completely untouched and are exactly as generative fill produced them. When using any kind of content aware fill we expect to have to do some touching up afterwards and generative fill is no exception. But in the interest of direct man and machine comparison I have done none of the final retouching I would anticipate in a real situation. This was my first attempt at just the simple basic creation of water. 
Here is the best of the several AI attempts at the same thing. The others were less good but still for the most part very acceptable. Even though on close analysis some of the geometry doesn't really hold up, on the face of it AI has done a better job than I. Certainly it feels more lifelike. 
My next attempt assumed the landscape at the rear to be much flatter than I had interpreted it in the first and allowed me to reflect more sky. I also added a duck taking off. I think this image holds up very well. 
When I gave this to AI to do it got very lost. It took numerous attempts to get anything good. Here I stumbled on the first big problem with generative fill which is syntax and vocabulary. How you describe the fill is key. You need to find clear, simple unequivocal terms. Adjectives and qualifiers are liable to either get ignored or misinterpreted if they are too sophisticated or too ephemeral. It's a bit like instructing a gifted child who is a visual genius but a has the reading level of a ten year old. It had difficulties understanding "duck taking off". Here is one of the better attempts. It has flying ducks(?) but rather disappointing. The rest of the fill, however, is just astonishingly good. 
In this next one the request for a duck taking off has been understood and attempted but we come across another difficulty. Generative fill can be amazing with landscape, nature and man made objects but grotesquely bad on occasions with animals and people. Clearly these ducks don't pass muster. You will see more of this later. 
My next attempt was to create muddy turbid water. The version shown here includes a modification by Frank which improved some of the splashing and turbidity. I think this works well and was disappointed by the generative attempts. 
Here is the best of the generative version. For some reason it completely ignored all requests for the duck. 
Next I moved on this rather unpleasant scene. I don't feel I have really pulled it off but I am really not quite sure why. 
When I came to the AI version I described the water as 'foamy' and the results all looked too bubble bath. So I changed this to 'floating scum' and ran into generative fill's 'morality' filter. This is a language filter that picks up key words in an attempt to prevent the the technology being used to improper or dangerous ends. The word 'scum' was rejected as offending community guidelines. Anyway here are a couple of what it produced. This first one did an excellent job with the stagnant pool but lacked the cow cadaver. 
Another attempt turned the dead cow into a mild epidemic. 
Another failed attempt that I have not included here littered the pool with about a dozen random cow fragments. It was just a simple example of AI's inability to understand human and animal anatomy very well yet, but end result look a bit as if a mad butcher had gone to work with a chainsaw. Whilst I can't call these last an AI fail, I do feel my own result, even though I am not entirely happy with it, is much better. My last offering in the challenge was this. 
It's an image with a lot of separate elements and it would seem highly unlikely that a generative fill could do anything remotely like it. The problem is that there is simply too much going on. How do you even describe it adequately? I tried all sorts of combinations of 'garden pool with flowered borders and miniature waterfall'. The waterfall attempts all came out huge and very fake looking other elements were good in one and bad in another. But the huge let down was once again people! 
Now if you look back through these images you can see, unsurprisingly, that the more complex the result you ask for the less satisfactory the result. There is a world of difference in saying 'replicate this and create me a new or extended background' and saying 'create this imaginary scene for me'. I have quite deliberately pushed each image a little further from 'content aware' to 'content creation'. This is of course quite unreasonable of me but it is a way of finding out where the current limits lie and how it can be employed to good use. I'll look at some good stuff in a minute but the key to success is accepting that AI works best with stuff well within it's experience. AI relies on it's current 'knowledge' of the world and it's contents. It's a child that is still learning .... and it's an American child. On another test entirely I asked for a 'red pillar box'. Every Brit knows this is our classical red cylindrical post box but this anglicism is not included in AI's vocabulary. I got all manner of pillar shaped boxes but not what I wanted. So I then tried 'red post box' and 'red mailbox' these got me the classically shaped American mail box in red. So finally I tried 'British red mailbox' and got once again the classic American post box in red but ........ with a union jack painted on it! I still don't know how to ask it for one of our red pillar boxes even though the Adobe image library upon which all this is built must have dozens of them! Learning the 'new' photoshop could be as much a question of language as graphic skills. So having come to the conclusion that it's use is currently limited and that, given enough time, I can probably do as well, and on occasions far better, lets look at what it really shines at. What you get out is very what you put in. Ask a silly question and you'll get a silly answer. Ask for something clearly and simply that lies well with the range of everyday experience and the results can be truly astonishing especially if what you ask for lies in the same style and materials of the source image. I am going to use the same image but ask for quite different fills. Stone buildings are an obvious start. Here is a small selection. 'Stone tower' requested: 
'Mediaeval Castle' requested: 
Now here's something interesting. In the last image AI has treated the original wall and terrain as fairly flat in the other three it has treated them as rising. Generative fill attempts to read perspective clues from the original image. If these are strong with clear vanishing points it's incredibly good at it. If, like Steve's image, there are no strong clues it will take a best guess which may be right or wrong but, having taken it, it will build consistently upon it. In the examples below it has chosen to interpret the foreground wall as much more steeply rising than is the case but it works because it has built consistently within that assumption and the perspective of the newly created structures conform to this. 
In the next images it has built on the assumption that the terrain is much flatter and once again has constructed consistently with this. In the second image the wall has been completely misinterpreted but very cleverly built upon in a manner completely consistent with the misinterpretation. Once more this is because the source image contains very few concrete clues. 
Lastly I decided to try quite deliberately asking for things which shouldn't be there and seeing how it cope. The first I asked for was 'Cable Car'. Its sort of found the various elements but failed completely in their proportions and assembly. 
This one I asked for 'Carousel'. It's made an appalling mash up of the carousel ....... but look how cleverly it has used the curvature of wall and built upon it. 
Request here was 'skyscraper city'. Its silly in the context of Steve's image but is well with AI comfortable limits to produce a plausible end result. Although his image has no horizon or obvious eye level it has created one and set the background city consistent with it. 
Once again let me remind you these are all untouched AI with none of the usual touch up we would expect to do in normal circumstances. Although I have tried to be positive where this extraordinary new tool has succeeded I cannot pretend that it gives me pleasure to have years of acquired skill replaced. Adobe would have us believe that this will boost our creativity .... I am rather more afraid that, in the vast majority of cases, it may start to substitute for it. So there we are. Right now it's an infant just starting to toddle. It will learn swiftly! I cannot even begin to imagine what capabilities lie ahead. Some of them very perturbing and frightening ...... but I'm not going there in this thread. _________________ The subtlety and conviction of any Photoshop effect is invariably inversely proportional to the number of knobs on it ....... |
Posted on 06/06/23 8:33:30 PM |
|
michael sinclair
Off-Topic Opportunist Posts: 1853 Reply  |
Re: "The Good, The Bad and The Ugly" what generative fill can and cannot do for you.
Very interesting David: A sort of unusual adventure. The only area that it really scores to my mind is the "conceptual visualisation"; that's to say, Its ability to create scenarios that might not occur to you: thinking out of the box so to speak when you are stuck for an idea. We could have a new section on this forum entitled "Generative fill". 
|
Posted on 06/06/23 9:25:43 PM |
|
vibeke
Kreative Kiwi Posts: 2166 Reply  |
Re: "The Good, The Bad and The Ugly" what generative fill can and cannot do for you.
Thank you Dave, very interesting. I'm still enjoying AI but agree someone who knows PS can often do a much better job but I suspect that AI will learn faster than most people. A niece send me a set of photos, taken straight into the sun, she asked me to improve the photo and open the eyes. After picking the best photo face of each person and moving them into one photo, I was hoping AI could do it for me and after a lot of tries I decided that the eyes might be open, but the people didn't really look like themself. So I had to try the 'old fashion' way. Still a long way from perfect, but she was happy.  Open the eyes by Vibeke Friis, on Flickr_________________ Perfect confidence is granted to the less talented as a consolation prize. |
Posted on 06/06/23 9:56:49 PM |
|
DavidMac
Director of Photoshop Posts: 5554 Reply |
Re: "The Good, The Bad and The Ugly" what generative fill can and cannot do for you.
That’s an extremely difficult request by any standards Vibeke. I definitely wouldn’t want to try it! _________________ The subtlety and conviction of any Photoshop effect is invariably inversely proportional to the number of knobs on it ....... |
Posted on 06/06/23 10:15:01 PM |
|
DavidMac
Director of Photoshop Posts: 5554 Reply |
Re: "The Good, The Bad and The Ugly" what generative fill can and cannot do for you.
Strangely Michael I found it got easily lost when asked to create outside of known recognisable parameters. But, on the other hand, it occasionally produced some astonishingly unusable and unusual nonsense that could definitely provoke new ideas. _________________ The subtlety and conviction of any Photoshop effect is invariably inversely proportional to the number of knobs on it ....... |
Posted on 07/06/23 06:52:26 AM |
|
Ben Boardman
Printing Pro Posts: 601 Reply  |
"The Good, The Bad and The Ugly" what generative fill can and cannot do for you.
So there we are. Right now it's an infant just starting to toddle. It will learn swiftly! I cannot even begin to imagine what capabilities lie ahead. Some of them very perturbing and frightening ...... but I'm not going there in this thread. [/quoted] Coming soon to a movie theatre near you! If only someone else hadn't pinched the title! I enjoyed the article David - Ben |
Posted on 07/06/23 11:33:54 AM |
|
Frank
Eager Beaver Posts: 1733 Reply  |
Re: "The Good, The Bad and The Ugly" what generative fill can and cannot do for you.
Interesting,Still feeling it out . http://www.bing.com/videos/search?&q=photoshop+beta&view=detail&mid=A06C11554BED28A88C19A06C11554BED28A88C19&FORM=VDQVAP&ru=%2Fvideos%2Fsearch%3Fq%3Dphotoshop%2Bbeta%26FORM%3DHDRSC4&ajaxhist=0&rvsmid=089BDA349B4C917D44C0089BDA349B4C917D44C0 |
Posted on 07/06/23 1:10:43 PM |
|
DavidMac
Director of Photoshop Posts: 5554 Reply |
Re: "The Good, The Bad and The Ugly" what generative fill can and cannot do for you.
Thanks Frank. Yes a powerful demonstration indeed. What I find bizarre Frank is why this man and so many others like him find this so 'super cool' and are getting so excited. The commentator in this video keeps laughing and smiling saying "I've been using photoshop professionally for fifteen years and I couldn't even get close to doing it this well!". Don't they realise that if AI gets to do it much better, which inevitably it will, it could render them obsolete? He also touched on something else, which is how everyone having online access to this power can be financed. It's free right now because it's beta (and presumably because the more people use it the better it will learn) but soon it will be governed by a points system which will allow you a certain number of generations. Adobe's intention is that generative fill be commercially available to anyone. On this basis anyone whose images are lodged with Adobe's library will be able to receive royalties from generative fill. He doesn't make it clear whether this is simply proportionally across the board or whether AI will actually be able to track how much of whose images it is using. Either way, sooner or later, it's going to get rationed with a payment system attached. Presumably an allowance based on your subscription plus some kind of pay by usage after. One has to accept that is quite logical and reasonable. This thing is going to move fast! So I suppose the big question is will it run out of control? _________________ The subtlety and conviction of any Photoshop effect is invariably inversely proportional to the number of knobs on it ....... |
Posted on 07/06/23 7:10:03 PM |
|
michael sinclair
Off-Topic Opportunist Posts: 1853 Reply |
Re: "The Good, The Bad and The Ugly" what generative fill can and cannot do for you.
WOW! WOW! WOW! I didn't know it could REMOVE things so perfectly and easily. I'm sold! Thanks Frank  |
Posted on 08/06/23 6:41:24 PM |
|
Steve Caplin
Administrator Posts: 6994 Reply  |
Re: "The Good, The Bad and The Ugly" what generative fill can and cannot do for you.
Impressive stuff! |