I noticed you're using PdfView. The latest library uses PDFLayoutVew, as PdfView is now deprecated. PDFLayoutView does not have a way currently that I've found to set the page scale to fix the issue of certain pages not filling the screen due to scaling. When using PDFLayoutView.PDFSetView(3), that should fix up Global.def_view to match Single page view mode, which is great, but certain pages still aren't scaling into view properly if the PDF has pages that aren't the same size.
*EDIT*
To solve this problem, I used the PDFViewPager instead of the PDFLayoutView, which has the ability to open a PDF using the following code snippet.
pdfView.PDFOpen(document, 0);
This opens the PDF as normal, but fits it to fill the screen regardless of page size. You can extend the PDFViewPager to add in functionality that you need, such as registering an OnPageChangeListener as normal for a ViewPager to track page turns, etc. I strongly suggest using the PDFViewPager rather than PDFLayoutView if you need to display a PDF page by page with each page fit to screen individually.