The Fermi bubbles (FBs) are large gamma-ray emitting lobes extending up to $55°$ in latitude above and below the Galactic center (GC). Although the FBs were discovered 8 years ago, their origin and the nature of the gamma-ray emission are still unresolved. Understanding the properties of the FBs near the Galactic plane may provide a clue to their origin. Previous analyses of the gamma-ray emission at the base of the FBs, what remains after subtraction of Galactic foregrounds, have shown an increased intensity compared to the FBs at high latitudes, a hard power-law spectrum without evidence of a cutoff up to approximately 1 TeV, and a displacement of the emission to negative longitudes relative to the GC. We analyze 9 years of Fermi Large Area Telescope data in order to study in more detail the gamma-ray emission at the base of the FBs, especially at energies above 10 GeV. We confirm that the gamma-ray emission at the base of the FBs is well described by a simple power law up to 1 TeV energies. The 95% confidence lower limit on the cutoff energy is about 500 GeV. It has larger intensity than the FBs emission at high latitudes and is shifted to the west (negative longitudes) from the GC. If the emission at the base of the FBs is indeed connected to the high-latitude FBs, then the shift of the emission to negative longitudes disfavors models where the FBs are created by the supermassive black hole at the GC. We find that the gamma-ray spectrum can be explained either by gamma rays produced in hadronic interactions or by leptonic inverse Compton scattering. In the hadronic scenario, the emission at the base of the FBs can be explained either by several hundred supernova remnants (SNRs) near the Galactic center or by about 10 SNRs at a distance of ~ 1 kpc. In the leptonic scenario, the necessary number of SNRs is a factor of a few larger than in the hadronic scenario. (abridged)