This project is an exploration into playing with watercolour imagery using Machine Learning. I have no idea where this will end up, but here goes....
I started off by getting almost 2000 watercolour paintings. I fed them into StyleGAN2 using RunwayML and generated these beautiful abstract watercolour artworks.
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/a1cb2d9b-ac64-415c-9563-393bcd58e33b_rw_1200.jpeg?h=bab1460b80470d622a7241b9a334d04f)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/5e8482a9-4163-49a4-bb1a-26a9c373c6ef_rw_1200.jpeg?h=a0e4dfe8c2483af169f5861ac53dabb0)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/84a03519-46f8-43ce-8675-d281126e140a_rw_1200.jpeg?h=a163cf0c582a69502902b0c30c32635d)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/0f0ee3aa-afcc-466b-9f5d-207ed5c8f853_rw_1200.jpeg?h=95161e434af8258b85921bb5379283ad)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/97f249f5-eea5-446d-bd47-27e1e0054d0f_rw_1200.jpeg?h=7e586c888ffcc0ac7ca1ca74883795c9)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/26a45b15-cc10-4b82-84f6-4f0409ca1705_rw_1200.jpeg?h=400bf048abccbc8707302d6314d0ca63)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/98c3f135-babd-4942-9c13-517d8d8bf810_rw_1200.jpeg?h=fffd8356ccfc70c9abb6a5c2d12c44bb)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/0922bd35-ef2a-45aa-b1c7-eaed82669344_rw_1200.jpeg?h=017e3a0a8c018ba81f43672ade82092a)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/17664c51-6b77-4141-a594-febf62feee47_rw_1200.jpeg?h=97c65b4d527bdb90244d583e4b4bf97b)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/c8c63951-188b-43bb-9311-71d0995f117d_rw_1200.jpeg?h=5224df1bddd1f3f04029a680d8ffcacc)
I asked the AI what does it see in the abstract watercolour paintings it created (using annotation algos) . Interestingly enough it said that it sees some forms in the abstracts, which I previously never saw. But when read them, I did see some of them too. I think I might have something going on in this dance with the AI.
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/b93f1eb5-1bab-46c9-899a-84c53c49e7f9_rw_1200.jpg?h=fe7c3cc5a176f9d1b03cebf0ff566880)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/ae880673-fabd-455e-a859-a0e1d7837b92_rw_1200.jpg?h=ae20c63e1cedba1b1606b766fb640cc7)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/3b26d08f-6cfd-4709-bc36-f4143db3df82_rw_1200.jpg?h=2eacfc72e6fd4866dd28a1f8211177db)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/9d25349b-daf5-494a-8d79-8e993ca551b3_rw_1200.jpg?h=dd3506fa47e356021aca7f796b026a08)
What I did next was to feed the text back to the machine using AttnGAN to find out "is that what you are really seeing machine? Or do you have a preconceived notion in mind". The machine did have a similar yet different ideas about them. I love this dance!
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/78ad5d88-fffe-4b23-bee9-e3e5742f72f8_rw_1920.png?h=3174a5370a985af79416342026cde768)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/1246ff19-1c4f-496c-a312-35a5d1ebb549_rw_1920.png?h=3cbef9e4ff041369d1a8938085857de8)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/c3d2bb17-b9a5-4ea9-8412-d44381926bff_rw_1920.png?h=86d91670376cb74b660355fbd9bac084)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/b8135bbf-441b-4ff9-8a9f-1364d6050338_rw_1920.png?h=9829bfaa79925bab86d98c0564b1d01a)
My friend and Georgia Tech Professor Devi Parikh used some of the abstracts to have a conversation with a bot to find out what exactly it sees. Check these out:
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/f98c43de-ff47-44e9-a21b-a38cda363513_rw_1200.png?h=a9ffc2139f105baa867988a1a4c8deaa)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/b0efb767-5927-4ea7-8908-4e0781a58d48_rw_1200.png?h=5c582753044c6f7a52b78efbdf9549fd)
I tried feeding one of the watercolour abstracts as a seed image and then running a loop image-text-image-text. Some of them generated interesting results after sometime:
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/ddad8ea9-2a65-4ef5-8d0f-9a572acabaa4_rw_1200.png?h=f0f683c7322299d868dafc2ad0e088eb)
While others went in an infinite loop
I started generating some interesting artworks out of the system, by cherry picking what looked best
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/2028a274-6130-4ae8-9b8f-60d97bf433bf_rw_1200.jpeg?h=dc0ab4faaf333db27232f50543fdcc29)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/c64b3f06-560f-4296-936f-61ea9d01babf_rw_1200.jpeg?h=10f083b827cba7b877d4f162815c4112)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/ecf99c28-064e-4758-9650-031946065d82_rw_1200.jpg?h=c9adf9de02a0d1ad7ff1d2f0a16611e7)
Making some beautiful rocks in Watercolour
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/ea02975d-d9c0-4c3b-af5f-6b8a58e05a53_rw_1200.jpeg?h=de23617265c80be56f3a6f5bb1aa74df)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/d660e869-dc5b-4a06-8826-252df629d790_rw_1200.jpeg?h=5801ec6e9a7e400528ebd2942a8dc4bc)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/f3ebe707-0785-4a70-94c1-ecfec28b4bff_rw_1200.jpeg?h=a1c43e27ab5400ebce5d6829f0128c74)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/95850abf-bc0a-4d06-abe4-1e207ccf7cba_rw_1200.jpeg?h=5b437147a7858cbce1db9c9ec628303c)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/afcf74b3-fd5a-446f-bf3a-15d8e9644a87_rw_1200.jpeg?h=dfd625b302671d5937f7a9ac598bc31b)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/d51a00c5-f01c-4e28-8a48-581b82169bfd_rw_1200.jpeg?h=d5cb05ebbf968e3d0bc374e3fa4405c3)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/7d33cf3a-ae3a-4c23-a9ab-b94a9db17ef4_rw_1200.jpeg?h=b5c734ae9129c960185fd3feadb37395)
Up next I used one of the watercolour abstracts as a seed to redraw it with my own human imagination. No drastic changes though.
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/d1093586-db98-46b7-8543-6a273266854b_rw_3840.jpg?h=ad50cd1d472d06cd82d0b014d2209d2c)
Then, I tried using the watercolour abstracts in a 3D inpainting algorithm. Here are few results:
One of the reasons I cherry pick images is because this was trained on the pre-trained faces dataset. So there is an amount of noise in some images. I hope to get more GPU capacity to get better results. But as of now, even these noisy images are interstinig.
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/fd63b741-8b97-4737-ba2b-39a6aca74423_rw_1200.jpg?h=a2bf5d4cedfe1e709143c431755cea6b)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/9cff819d-207c-4eac-87a5-843698584628_rw_1200.jpeg?h=73518951e17d1c6e19d427140cddd043)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/11592f0a-ffe0-4619-a0a0-dcdc5567199b_rw_1200.jpg?h=9dba5841c9719466f7a76ffccc5620bd)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/212d3e1c-bab3-4b4e-a160-a42b886ec660_rw_1200.jpeg?h=b347d18bfb697565f4af7b9ff841d7db)
I reverted to a model in the earlier stages of the transfer learning cycle and got these interesting Faces + Watercolour mashups.
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/c2553743-1b34-48b1-b21c-62cfaa11c014_rw_1200.jpeg?h=41344e913125610d9b522e5d87414a61)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/1c4cffbd-4cf9-41f5-9f3f-ee23a447b2a4_rw_1200.jpeg?h=5d3781e329471e6aa3ff44e3742187e2)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/6e3270c1-242e-486e-a98e-e46145d0aab2_rw_1200.jpeg?h=82b41c57ee8d14165b1411e3d5f096d8)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/e4686c22-285e-4570-82ff-75c84bd343cd_rw_1200.jpeg?h=0980ae28ffdce28fe67f7d5edcb184e1)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/154e6f2b-0f63-4289-8390-f60004cf9391_rw_1200.jpeg?h=8b8cc3f0b43aab9335e1397a13c6491f)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/6a5f940e-dfa8-40cb-88ff-e2b4960356c7_rw_1200.jpeg?h=62306eea19acd922f0d3fe7ebeffb94a)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/3cc6b119-bd8e-4a7d-9fa6-0084421cf5de_rw_1200.jpeg?h=01c0cb435a2ededad4aac6d7bc1ec133)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/2567123b-7e1e-4303-b15c-f4cdf65b05a3_rw_1200.jpeg?h=fff2fc7a3413612b2c6353bc341f701a)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/53dae21d-7602-4787-9343-f9f09ca198f5_rw_1200.jpeg?h=a16b22f5da3807ae2d36f969c54133ce)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/82368f06-9e6d-4642-8dc0-17c974a4ce46_rw_1200.jpeg?h=56e7b740b84808ba8e2885c5f13030fe)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/4a4abcb7-3938-4499-991b-183993530c74_rw_1200.jpeg?h=09cf2dc9c6b076596aee312ff6c9efc7)
Midnight Falls - yet another series of abstracts. (Or are they new fashion jeans?)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/bc8d8fe0-77af-4341-9bb8-2771856023e2_rw_1200.jpeg?h=9655359b63906f900e32f725f9d1cd62)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/e3746802-4cb7-416f-98c6-9932b2c0cf90_rw_1200.jpeg?h=5939d85278e5cd26414d465947ce7a69)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/feb25243-4c1f-4dba-b6e7-757fc68275e7_rw_1200.jpeg?h=e9015471a81577a04b5ff84b97aa0ff9)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/39d663bc-a493-49c8-9a2a-b8ef72b0a8ab_rw_1200.jpeg?h=aa70f4f234756368619b745c353aeec6)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/4eb17e21-09ca-4f49-be86-5c847fd2a6e2_rw_1200.jpeg?h=f67858798c78ed0b3f3b47384f46b049)
A little more swim in the latent waters of colors
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/f021ca8a-1f8a-45f7-bfc1-03284258ef75_rw_1200.jpeg?h=d00788eae311f07965c64407730e13cb)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/ce9e19fe-41e8-40f6-926a-482ad4ce6558_rw_1200.jpeg?h=c50c1b7626e6722abf85cd14f98221cb)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/c14409f3-cc4d-4e8b-89e7-fbc6185ebefe_rw_1200.jpeg?h=d058851babe64f5803fa6c8823493b3f)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/f572eba9-f866-432e-9470-36fd4fffbbf5_rw_1200.jpeg?h=a0acdcda4f3be09ea7f5eaee482486ed)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/c9754e93-e772-4ddb-8bd3-f627d82d5730_rw_1200.jpeg?h=bffdbdeea4ca5a865d730f7c504407f6)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/267f5d9f-28d5-4f0b-aa28-b8fdd7496927_rw_1200.jpeg?h=6004d7f13b46edc97484c65a97e80d16)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/3334f3d4-4b39-40c2-a49e-c565da27bba5_rw_1200.jpeg?h=691fd3c7e0904db339f154e8114763d5)
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/61616fb8-abfb-4b3a-a994-6588e5339051_rw_1200.jpeg?h=e0c18b0467ef0d1d75c432533f97fc07)
Up next is not exactly watercolour, but I Illustrated an Aghori over the weekend. I then used the face in a First order motion model to create a video. Check it out:
![](https://cdn.myportfolio.com/02c5201375c3706250b26cf840ace79c/f780f83f-017f-4087-8dda-4782081eaba8_rw_1920.jpg?h=b9e96e85862639407d0d5f336deb99a8)