Backdoor detection in embedded system firmware without file system

Any embedded system firmware without file system will integrate its system code and user application code into a single file. This setting has brought some additional difficulties to analyze them. Aimed at this kind of firmware, the problem of library function identification was analyzed, and several heuristic methods to recognize some important function relevant with manipulating network socket and character string/memory were proposed. Based on this analysis, the backdoor detection problem of some typical types including unauthorized listener, unintended function, hidden function, outward connection request etc. were discussed, and several backdoors (one is critical level) in a real world firmware were found. The result shows this method of identifying library function can be useful for security analysis to this type of firmware.