The SwiftKey Keyboard vibration is set to user Android's default vibration. But then I turn off haptic feedback on Android settings and the keyboard is still vibrating on key press. This doesn't make any sense whatsoever! Fix it!
When using GBoard Nd turning off haptic feedback on Android, the keyboard key press also turn silent as it should.