What kind of plants are you growing? Generally my first guess would be that the problem of flowering when very small is because of day length sensitivity. Many plants, including vegetables, use the change in day length over the year to signal what season it is. When farmers in Niger plants rainy season vegetables such as okra or bissap (aka yakwa or roselle) at the beginning of the rainy season the plants grow big with the longer days in June or July, and then begin flowering in August or September when the days start to shorten. Your plants may “think” that since the days are shorter this time of year (by a little less than one and half hours according to Wikipedia) that dry season has begun and they need to flower and fruit quickly before the rains end. I remember having both okra and bissap do this when we tried growing them in the dry season in Maïné-Soroa.
If farmers in your area practice dry season gardening in low lying areas you can ask them which plants they grow in dry season with irrigation, and which ones only in rainy season. If not, choosing day neutral varieties of each vegetable you want to grow when you buy seed can give you flexibility for which season you plant in. However, some vegetables (especially tomatoes) require the cooler night time temperatures from December through March to fruit well, so there are other factors besides day length that go into which seasons to grow different vegetables.
I’m not sure what is up with the seedlings beds vs compost holes. It might have to do with how sheltered the garden is compared to the seedling beds, or some other factor. Someone in your community may have some input as to what is going on. Often wind exposure is a big factor in growing things in Niger. Are your seedling beds more sheltered? If you have a picture of the plants that might help as well.