Info
This post was imported from a personal note. It may contain inside jokes, streams of consciousness, errors, and other nonsense.
Overnight I realized my mistake (or at least, one of them). I’m measuring the distance to the food source from each sensor relative to the position of the BraitenBoid but I forgot to add the position of the BraitenBoid. So basically I was measuring the distance to food sources from the top left corner (ish) of the screen and telling the BraitenBoids to make decisions on which way to turn based on that. Not helpful.
I ran evaluate after making the change and the results were surprising:
→ BraitenBoids git:(main) ✗ ./x64/Debug/BraitenBoids.exe evaluate
Command-line arguments:
1 = evaluate
evaluate command
Running generation 0... complete (3.39577 seconds, 6341 steps)
Running generation 1... complete (1.85802 seconds, 3904 steps)
Running generation 2... complete (0.886292 seconds, 1725 steps)
Running generation 3... complete (1.69776 seconds, 2764 steps)
Running generation 4... complete (1.71878 seconds, 3148 steps)
Running generation 5... complete (1.57847 seconds, 2951 steps)
Running generation 6... complete (1.35149 seconds, 2619 steps)
Running generation 7... complete (1.20664 seconds, 2561 steps)
Running generation 8... complete (0.989215 seconds, 1709 steps)
Running generation 9... complete (1.13717 seconds, 2411 steps)
Running generation 10... complete (1.00406 seconds, 1941 steps)
Running generation 11... complete (1.25062 seconds, 2840 steps)
Running generation 12... complete (1.62532 seconds, 2976 steps)
Running generation 13... complete (0.844412 seconds, 1502 steps)
Running generation 14... complete (1.37239 seconds, 2391 steps)
Running generation 15... complete (1.01241 seconds, 1730 steps)
Running generation 16... complete (1.0274 seconds, 1977 steps)
Running generation 17... complete (1.53298 seconds, 2885 steps)
Running generation 18... complete (1.02246 seconds, 1773 steps)
Running generation 19... complete (1.2534 seconds, 2133 steps)
Running generation 20... complete (1.04019 seconds, 2064 steps)
Running generation 21... complete (1.17926 seconds, 2077 steps)
Running generation 22... complete (1.08841 seconds, 2260 steps)
Running generation 23... complete (0.823213 seconds, 1522 steps)
Running generation 24... complete (1.61009 seconds, 2876 steps)
Running generation 25... complete (1.3409 seconds, 2661 steps)
Running generation 26... complete (1.05357 seconds, 1977 steps)
Running generation 27... complete (2.4703 seconds, 5377 steps)
Running generation 28... complete (1.05577 seconds, 2164 steps)
Running generation 29... complete (2.35638 seconds, 5124 steps)
Total time: 41.8045s
Saving file output/boids.json
Saving evolution log file output/evolution_log.csv
For Boids the first generation usually takes 10k steps. Maybe even the second or third generations before they get smarter. These seemed to be pretty smart right away but they didn’t get too much better
Actually, they’re mostly just going in circles, still. I see a tiny bit of behaviour of steering toward food sources but…
There’s an orange boid that comes up from the bottom at the middle of the screen that actually seems to be trying a bit.
Tried another run and this time it got worse and better, worse and better.
Here are my expectations for BraitenBoid weights I worked out yesterday:
w0 (bias to speed) - Positive to tend to keep moving.
w1 (bias to turn) - Ideally zero but it usually doesn't pan out that way.
w2 (left food detect to speed) - Slowing down might actually be good. Negative?
w3 (left food detect to turn) - Negative so when it's activated it'll turn left.
w4 (right food detect to speed) - Maybe slow down? So a little negative?
w5 (right food detect to turn) - Positive so when it's activated it'll turn right.
Here are the results of the second run:
Generation: 30
Steps: 4449
Fitness scores, weights:
7 0.59 0.04 0.55 -0.74 0.95 0.40
4 0.46 -0.03 0.61 -1.00 1.00 0.40
4 0.68 0.12 0.36 -0.77 0.87 0.61
2 0.63 0.02 0.28 -0.76 1.00 0.47
2 0.46 0.17 0.63 -0.81 0.93 0.30
2 0.37 0.02 0.46 -0.91 1.00 0.30
1 0.75 0.10 0.34 -0.92 1.00 0.57
1 0.64 0.12 0.32 -0.91 0.98 0.56
1 0.73 0.07 0.33 -0.82 1.00 0.46
0 0.46 0.04 0.43 -0.94 1.00 0.38
They’re not far off. That’s … not sure how I feel about that. I’d expect them to look better. Maybe I should try hard-coding them with the weights I think would be good and see what that looks like.
First I’ll try running 100 generations to see if it’s just slow to find a good solution and they _can_ get to a better point. The first two runs took 40 seconds and 80 seconds so this might be around 120-240 seconds?
…165 seconds.
100 Generations Later #
Generation: 100
Steps: 2757
Fitness scores, weights:
4 0.38 0.65 0.04 -0.65 0.71 0.81
4 0.43 0.73 0.00 -0.78 0.89 0.96
4 0.20 0.70 -0.08 -0.78 0.56 0.93
3 0.29 0.77 0.02 -0.88 0.70 1.00
3 0.48 0.80 -0.07 -0.69 0.66 0.79
2 0.16 0.93 -0.15 -0.88 0.76 0.90
1 0.27 0.89 -0.04 -0.76 0.70 1.00
1 0.38 0.70 0.10 -0.60 0.82 0.90
1 0.19 0.70 -0.08 -0.78 0.60 0.88
1 0.17 0.96 -0.14 -0.76 0.75 0.93
#
BraitenBoids #
Gross. No real improvement there. I guess it stops bottoming out after like 70 generations, at least on this run. Let me dig up an old Boid chart for comparison.
#
Old Boids #
Yeah, there’s a visible improvement real quick _and_ it converges more or less on a score that’s better than the BraitenBoids’ best.
Hard-Coded BraitenBoid Weights #
I ran a bunch of debug sessions, adjusting the weights. Turns out my record of which weight does what was completely wrong. Oof.
The actuals:
w0 (bias to speed)
w1 (left to speed)
w2 (right to speed)
w3 (bias to turn)
w4 (left to turn)
w5 (right to turn)
I tried with these weights: {0.2f, 0.f, 0.f, 0.f, 1.f, -1.f}
That’s works.
Looking back at what the BraitenBoids learned before, they both decided a strong turning bias was a good idea. Going in circles instead of going straight. Reckon spreading the food out would help that because then they can cover more territory in their explorations.
Experimenting with it some more, it looks like the sensor range is very important. With a range of 200 pixels it’s detecting stuff that’s far away and tries to turn toward it but can’t turn fast enough.
Actually, it’s probably the ratio of turning rate (including forward speed) to sensor range.
Also, right now the “speed neuron” is affecting the boid acceleration not the desired speed. That pretty much results in the boid always traveling at its max speed. No wonder they don’t slow down to make that turn for food! If the boid ever has a negative acceleration there’s too much danger of ending up dead in the water.
Refocusing #
Okay, where am I at?
I know the dumbed down version of the BraitenBoid sensor code is working because my hard-coded weights work.
Evolving still sucks. Here’s what I got after 30 generations:
Generation: 30
Steps: 2342
Fitness scores, weights:
5 1.00 0.85 -0.06 0.29 -0.29 -0.22
4 0.85 0.86 -0.05 0.52 -0.38 -0.30
3 0.97 0.71 -0.21 0.27 -0.57 -0.40
3 0.96 0.76 -0.24 0.21 -0.48 -0.33
3 1.00 0.77 -0.14 0.40 -0.61 -0.30
2 0.97 0.76 0.07 0.39 -0.36 -0.31
2 1.00 0.82 0.01 0.38 -0.46 -0.30
1 0.94 0.85 -0.23 0.37 -0.48 -0.43
1 0.89 0.94 -0.14 0.52 -0.44 -0.35
0 1.00 0.77 -0.11 0.17 -0.49 -0.31
Go fast, faster if the left sensor detects something but not the right. Turn a little to the right in general but if either sensor detects anything turn more to the left.
Here’s another one. Looking promising:
Generation: 30
Steps: 2405
Fitness scores, weights:
5 0.69 0.74 0.52 0.03 -0.94 0.88
4 0.64 0.68 0.60 -0.02 -0.86 0.94
4 0.49 1.00 0.54 -0.03 -0.89 1.00
3 0.67 0.95 0.48 -0.12 -0.93 0.98
2 0.51 0.96 0.63 0.07 -0.97 0.97
2 0.60 0.78 0.56 -0.03 -0.84 1.00
2 0.58 1.00 0.64 0.02 -1.00 0.88
1 0.64 0.97 0.45 -0.11 -0.87 1.00
1 0.50 0.83 0.71 -0.11 -0.92 0.79
0 0.37 0.89 0.63 -0.06 -0.88 0.82
Finally seeing opposing values for the left sensor to turn and right sensor to turn. Also near zero for the bias to turn. The left sensor triggering more speed than the right is unusual but maybe it just doesn’t matter much.
Here’s how it evolved.
It took a while but eventually found its way around the 15th generation.
It would be much harder to evolve opposing pairs of values like the left sensor to turn and right sensor to turn than it is independent values like bias to speed. In that way Boids had it much easier because they just had to map the “direction to food” neuron directly to the “turn amount” neuron and it was golden.
Encodings. So important.
I’m’a make the food more spread out. Increasing from 800x800 to 1920x1080 and dropping the amount of food from 20 to 30. The number of boids will stay at 10.
Oh and sensorRange down from 200 to 100.
This breaks consistency with my previous experiments on Boids but I think it’ll give me better results.
Ooh except my screenshots will be friggin’ huge. That’s a problem.
This is a problem, too:
evaluate command
Running generation 0... complete (6.11631 seconds, 10000 steps)
Running generation 1... complete (5.85101 seconds, 10000 steps)
Running generation 2... complete (5.95548 seconds, 10000 steps)
Running generation 3... complete (6.0817 seconds, 10000 steps)
Running generation 4... complete (6.36373 seconds, 10000 steps)
Running generation 5... complete (6.29585 seconds, 10000 steps)
Running generation 6... complete (5.69415 seconds, 10000 steps)
Running generation 7... complete (6.11323 seconds, 10000 steps)
Running generation 8... complete (5.95985 seconds, 10000 steps)
Running generation 9... complete (6.33109 seconds, 10000 steps)
Running generation 10... complete (6.22635 seconds, 10000 steps)
Running generation 11... complete (6.31088 seconds, 10000 steps)
Running generation 12... complete (6.28821 seconds, 10000 steps)
Running generation 13... complete (6.36906 seconds, 10000 steps)
Running generation 14... complete (5.63399 seconds, 10000 steps)
Running generation 15... complete (5.83566 seconds, 10000 steps)
Running generation 16... complete (5.97669 seconds, 10000 steps)
Running generation 17... complete (5.88945 seconds, 10000 steps)
Running generation 18... complete (6.27718 seconds, 10000 steps)
Running generation 19... complete (6.39526 seconds, 10000 steps)
Running generation 20... complete (6.44678 seconds, 10000 steps)
Running generation 21... complete (5.99443 seconds, 10000 steps)
Running generation 22... complete (5.70831 seconds, 10000 steps)
Running generation 23... complete (5.89647 seconds, 10000 steps)
Running generation 24... complete (5.71147 seconds, 10000 steps)
Running generation 25... complete (5.70992 seconds, 10000 steps)
Running generation 26... complete (5.20226 seconds, 10000 steps)
Running generation 27... complete (6.22801 seconds, 10000 steps)
Running generation 28... complete (6.11892 seconds, 10000 steps)
Running generation 29... complete (6.10709 seconds, 10000 steps)
So much for getting better results.
Okay, I’m backing down to 800x800.
Dumb luck might be having too much influence, too. I can start all the boids in the center and make sure no food can spawn within a certain distance of the center.
I should commit my code at some point… not gonna get nice BraitenBoids working before that so I guess I’ll commit the WIP ones. Or maybe do these experiments with Boids for now.
- Commit sensorRange change first.
- Then commit BraitenBoidRenderer.
Well, it’s good to be back to regular Boids. Look at this performance.
Running generation 0... complete (4.65398 seconds, 10000 steps)
Running generation 1... complete (1.93325 seconds, 4041 steps)
Running generation 2... complete (1.50227 seconds, 3377 steps)
Running generation 3... complete (4.0496 seconds, 10000 steps)
Running generation 4... complete (1.55531 seconds, 3739 steps)
Running generation 5... complete (4.85956 seconds, 10000 steps)
Running generation 6... complete (1.70808 seconds, 3492 steps)
Running generation 7... complete (3.82896 seconds, 10000 steps)
Running generation 8... complete (1.26799 seconds, 2437 steps)
Running generation 9... complete (1.03221 seconds, 2222 steps)
Running generation 10... complete (1.55543 seconds, 3158 steps)
Running generation 11... complete (0.582025 seconds, 1154 steps)
Running generation 12... complete (0.369264 seconds, 703 steps)
Running generation 13... complete (0.222297 seconds, 447 steps)
Running generation 14... complete (0.395936 seconds, 705 steps)
Running generation 15... complete (0.351555 seconds, 702 steps)
Running generation 16... complete (0.298285 seconds, 560 steps)
Running generation 17... complete (0.329303 seconds, 618 steps)
Running generation 18... complete (0.396322 seconds, 772 steps)
Running generation 19... complete (0.271962 seconds, 516 steps)
Running generation 20... complete (0.311434 seconds, 592 steps)
Running generation 21... complete (0.305117 seconds, 515 steps)
Running generation 22... complete (0.327648 seconds, 617 steps)
Running generation 23... complete (0.435351 seconds, 789 steps)
Running generation 24... complete (0.360142 seconds, 684 steps)
Running generation 25... complete (0.377612 seconds, 616 steps)
Running generation 26... complete (0.365746 seconds, 604 steps)
Running generation 27... complete (0.229073 seconds, 390 steps)
Running generation 28... complete (0.502652 seconds, 1046 steps)
Running generation 29... complete (0.414807 seconds, 856 steps)
Sensor range 100, by the way.
Okay, I managed to clean up my git history and clear my stash.
Going back to Boids for now.
Reducing Dumb Luck #
I’m starting the boids near the center in a food-free zone so the ones that barely move but manage to land right next to some food won’t be so lucky anymore.
Speeding Things Up #
Before I was instantiating a new vector of floats every step for doing neural network calculations. Here are the times for the first five generations:
Running generation 0... complete (3.51725 seconds, 5000 steps)
Running generation 1... complete (3.58735 seconds, 5000 steps)
Running generation 2... complete (3.47705 seconds, 5000 steps)
Running generation 3... complete (3.44008 seconds, 5000 steps)
Running generation 4... complete (3.27431 seconds, 5000 steps)
Getting rid of those instantiations and using NeuralNetwork::input and NeuralNetwork::output members the times improve to:
Running generation 0... complete (3.24172 seconds, 5000 steps)
Running generation 1... complete (2.86802 seconds, 5000 steps)
Running generation 2... complete (2.22767 seconds, 5000 steps)
Running generation 3... complete (1.96475 seconds, 5000 steps)
Running generation 4... complete (2.10566 seconds, 5000 steps)
Not bad.
Now if Eigen docs would come back online I could use vector and matrix multiplication instead of doing the maths by hand. That should give a performance bump, too.
Taking a Step Back #
I tried to do this earlier and got distracted. I’ve lost sight of the goal a little bit with the BraitenBoid not working out as I had hoped. I think I’m a little frustrated that the behaviour is so… ugly. Then I started working on speeding up evolution.
I didn’t realize being aesthetically pleasing was part of the criteria but I guess in some way, I’m doing the natural selection so if I don’t like it then it goes extinct. I think that’s part of what motivated having the boids all start in a food-free zone. I don’t like them just spinning in circles constantly. I want them to look like they’re going somewhere.
It’s strange but I think that’s what’s going through my head.
Making the amount of food more sparse might help but I think there’s something I can change to improve it even more, and it’ll probably contribute to the elegance. Wasted time and energy is not elegant. But these creatures have no concept of energy and their only concept of time is getting to the food before another boid does. So I can give them these concepts.
I can include time and energy as input nodes in the neural network and see how they come to affect performance. I can also include energy as a penalty in the selection criteria. The more energy consumed the bigger the penalty.
How far should a boid be able to travel using the energy from a single piece of food. 1000 px? How should speed affect energy consumption?
I also watched a few videos from Primer on YouTube and I like the approach taken in those environments. Well, a lot of things are awesome but the ones I’m considering for BraitenBoids are changing the reproduction criteria, allowing population size variations, and death.
My current setup:
- Right now boids can’t die.
- Every generation has only ten spots.
- The top three boids get to reproduce.
Primer’s setup:
- Blobs can run out of energy and die.
- The population is not controlled directly.
- All Blobs that meet the selection criteria get to reproduce.
Maybe it’ll fix it but really, I’m ignoring the real question:
Why didn’t BraitenBoids work? #
Really, it was having a hard time evolving the weights necessary to steer toward food. This behaviour relies on two separate weights changing in the correct direction over time and my simulation is just too small for that to work. Maybe I need to go full BraitenBoid; don’t just implement the sensors but implement the motors as well. That way the strong connection from the sensor to the opposing motor can evolve independently…
…wait, that’s not actually, true, is it? To get a behaviour of going straight toward a food source once the boid is going in the right direction requires even more weights to be balanced! Four of them instead of just two! Oof, that’ll be heaps worse.
What if the boid bodies inherently return to a forward direction? Activating the motors can steer a bit left and right but it always returns to moving forward in the absence of stimulus? The reason that won’t work is because a stimulus directly infront of the boid is still a stimulus.
Interesting… I always thought of the setup I had with the Boids as being lazy but really, I’m probably pretty lucky that I started with that. It works really well! It does still need to evolve a few different weights to get a balanced steering behaviour but really, the weight w5 is basically “how much to turn toward food” and works on its own.
So. I _am_ going to simulate more complex boids as I continue on this adventure. And if my current simulation can’t evolve BraitenBoids or FlowerPetalBoids for that matter then the problem is not the BraitenBoids but the simulation itself.
Maybe I’m gonna need to do a layered approach like Code Bullet did teaching Alfred to walk.
- Teach them to move at all. Reward forward movement.
- Teach them to turn toward food. Reward getting food.
- Teach them to use energy efficiently. Reward getting food. Punish for energy consumption.
If so, how do I structure my application to run a population through these different simulations with different mechanics and selection criteria?
And do the points above about Primer’s setup help me?