The paper like stuff inside the hole is the pith. It's usually moldy and gross.
Not to sound redundant (in other words, yes I realize that I said this in the last post but it bears repeating) but you will need a dust mask of some kind...something more substantial than a bandanna or dew rag or t-shirt tied around your face. I strongly suggest a full on respirator, I use one-P100 hot pink filters and everything. At the very least you will need a dust mask made for filtering fine particulate matter.
The other tools I use for cleaning the inside of a gourd are my trusty variable speed dremel tool with various sized grinding/sanding attachments (a drill and a hand saw or handheld jigsaw/router would work fine too). I like the variable speed dremel because it cuts down on the amount of dust and offers better control. I also use an old jigsaw blade, curved awl, and a thin wire brush.
For making a birdhouse the first step is to drill four small holes into the bottom of the gourd to help ensure proper drainage, it is a birdhouse after all not a birdbath. The next step is to determine where you want to place your opening. I use a compass to draw a circle that is 1"-1 1/2" (the type of bird you want to attract will determine the size of hole you need to drill...check out a chart here) in diameter and use a pencil to mark the center of the circle. That's where you'll drill your starter hole. Then you need to drill two small holes into the handle of the gourd for your hanging wire, leather strips, strong string or whatever other sturdy material you'll use to hang your birdhouse.
I use the various grinders and sanders for creating the hole opening instead of a jigsaw or handsaw in part so that I can sand the hole opening while I'm making it...you know, killing two birds with one stone and all that...sorry for the poor choice of idiom but you get the point. Anywho, once the opening is made it's time to clean out the inside of the gourd. I use the old saw blade to break up the pith and the seeds and I shake out the loose material. Then I use the awl and wire brush to loosen and grab whatever is left for a completely clean result. Some people leave a bit of pith in the bottom to help the birds gain footing. Personally the birds that come to my birdhouses never seem to have any trouble with footing without the pith and the one or two times I've left some in there the birds usually throw it right out anyways. Ultimately though the choice to pith or not to pith is really one of personal preference.
The inside of the gourd after it has been cleaned and the pith has been removed.
That's it, now the gourd is officially cleaned. After it's cleaned you can finish it off by painting or staining it and sealing it and it will last for decades.
Finished (or nearly finished) product. Please ignore the extremely cluttered work table :)