After setting a speed keyframe, you can move it to a different location in a clip.
Move an unsplit speed keyframe
In the Timeline, Alt-click (Windows)
or Option-click (Mac OS) the unsplit speed keyframe, and drag it
into its new position.
Move a split speed keyframe