As we read the story, I stopped after reading each page and had Jake search the room for items that matched the color we just read in the story. "There is a green tree on a red rock in the middle of the deep blue sea." I had him use our colored buckets to find items of the same color.
Every time he found something of the same color he would say, "Same!" I did have him add saying the name of the color as well. He still has trouble with his colors and this is just more practice. By having him say the name it reiterates the name, over and over.
I have also added clues to the color names to help him try to remember the color words. For instance, if he cannot remember the name of the color blue I say, "It is daddy's favorite color, remember?" This usually sparks him to remember the name. Or, "It is the color of a fire truck," for the color red. Just a little trick to help him learn his colors. Nothing has been harder for him to learn than colors. I would have never guessed colors would have been so hard!