This research aims to integrate constructive neural networks and virtual robot placement strategy to achieve obstacle avoidance and navigation for multiple robots in a generalized map environment. Deep reinforcement learning theory is applied to design neural networks, which are trained in free space to improve the performance of the dual robot system in obstacle avoidance and navigation. Additionally, a constructive architecture and the concept of social repulsion are introduced to extend the social navigation network of the dual-robot system to multi-agent system. The paper proposes a novel virtual robot placement strategy that effectively utilizes virtual robots to replace fixed obstacles of nearby robots in different map environments. The purpose of this strategy is to use virtual robots to warn robots, keep them away from obstacles and avoid collisions. Furthermore, the paper utilizes the Hybrid A* algorithm to generate waypoints, which enables the obstacle avoidance navigation system to operate effectively in general map environments.