Default voice keyboard

I can select to use Google Voice input as opposed to SwiftKey, but the very next time, it reverts back. Want to be able to have a default of Google voice input.