The End of Offshore Development

The offshore of software jobs could in the long term decimate the American IT industry and reduce American software competitiveness. Software is vital because it fuels the U.S. economy and serves as a catalyst for innovation, communication, and potential economic growth. However, over the last decade, American corporations, citing cheap labor costs transferred 2.4 million jobs to India. Concurrently, since 2002, the number of students graduating with computer science and related degrees has declined 58.5%. A 2006 study on “Globalization and Off-shoring of Software” by the ACM indicated that because of off-shoring, there is no way of ensuring lifetime employment for American software engineers. Who would want to study, graduate, and enter a profession with no guarantee of lifetime employment?  The research firm Gartner Inc. predicts that up to 15 percent of tech workers will drop out of the profession by 2010, not including those who retire or die. According Business Week “The U.S. high-technology industry lost 115,800 jobs in 2010, a 2 percent decline that marked the industry’s second consecutive year of falling employment.” The situation, however, is different in India, particularly among the three largest IT companies. Tata Consultancy Services Ltd., the largest Indian IT-service provider, currently with over 202,000 employees plans to hire a total of 60,000 workers this year. Infosys Ltd, the second largest over 133,000 employees plan to hire 45,000, while Wipro the third largest with over 122,000 workers plans to increase from 25 to 35%, 30,000 to 40,000 respectively. Since there is no domestic industry for India IT to support, the vast majority of these are American based.

The result is a diminishing labor pool of software engineers and the end of American software dominance. As the IT industry contracts, American developers limited to Aerospace work and beset by the high costs of software development and poor quality problems will face survival problems similar to the American automobile and commercial electronic workers in the 1980s. Eventually software research and development as well as innovation will follow the jobs.  The DOD will be particularly hard hit because of its reliance on software, the risk of offshore development, and a dwindling pool of citizen software talent eligible for security clearances.  

Software Development Life Cycle

The Software Development Life Cycle (SDLC) that underpins current software engineering practices and methodologies is the reason for offshore development. Simply stated, it does not work for software engineering. It is the core software development problem because it applies a manufacturing, assembly line, manual work, and linear oriented approach to software development. Software requirements are so volatile that a mechanistic, monolithic process that focuses on task management and ignores human issues and behaviors cannot work. Ignored are the human social dynamics that include people, their needs, desires, and fears as well as the impact upon project behavior, motivation, and morale. Projects are more than a sophisticated planning and control systems; projects require collaborating, communications, coordination and cooperating between people. It is the behavioral aspects of software development that make large software projects as much human systems than technical systems. As the late Watts Humphries once stated, software developers resent being treated like grunge. The assumption that human behavior can be ignored and software engineers treated like manual laborers create project performance problems and reduce staff productivity.

Organic Software Engineering

The process of software development and evolution is an ambitious undertaking involving complex, incomplete, sometimes inconsistent and often fuzzy factors.  Variables concerning design, quality, reliability, stakeholder interests and objectives, moving targets, and constraints such as budget and timeline must all be considered throughout a dynamic life cycle. The challenge is to provide sound methodological support for enabling good decisions about processes and products, risks and bottlenecks as well as for selection of tools, methods and techniques.  A need also exists to certify critical software systems to ensure their dependability, relying on evaluation of the software development process as well as the properties of the system.  Software engineering research topics will identify and develop innovative means of meeting the challenge. TECHNOLOGIES OF INTEREST – DARPA 

Closed system software development approaches like the SDLC cause software projects to operate like machines. The requirements phase cranks up the project machine that activates a series of sequential phases, activities, and tasks until the system is implemented. It uses a common solution approach that includes detailed processes and methodologies. The problem is that the machine response is too inflexible to handle DARPA’s accurate description of software reality. How could a machine deal with the inconsistencies, fuzzy factors, uncontrollable variables, performance factors and changing targets?

Open systems such as biological and social systems however operate quite differently from mechanistic systems. Organic systems are designed for the inconsistencies, fuzzy factors, uncontrollable variables, performance factors and changing targets. Instead of a common solution approach, there is the concept of equifinality. All software projects are unique because each project has a set of different system objects including purpose, environment, boundary, inputs, outputs, process, restrictions, and feedback control. It suggests that similar results may be achieved from different initial conditions and in from many different ways. In software development, the implication is that software projects may achieve project system objectives from different situational contexts. Instead of cranking up the monolithic project machine, biological projects define their objectives and goals, organize the project ecosystem, evaluate their situational context, develop the strategies, identify the quality needs, select the tactics, and create the operational plans that when executed will deliver the software in stages of systems growth

The Organic Software Engineering Approach (OSEA) solves the problems associated with the almost infinite number of project variations that make each software projects unique. Software projects are like the Tower of Babel. There are the multitudes of technologies and development methods, tools, techniques, approaches and processes. Software projects are also rife with context variations, unknowns, initial starting conditions, restrictions, constraints, and controllable and uncontrollable variables. The OSEA consists of a perception, paradigm, conceptual framework, behavioral model, and a software development practice called the Organic Software Development Practice (OSDP). The OSDP replaces the mechanistic characteristic associated with the SDLC with biological characteristics associated with living systems. Software projects are not simple, causal, linear systems where the requirements are to simply define the tasks, estimate the effort and duration, and then schedule and monitor the progress. Software projects are complex adaptive systems that fit the above DARPA description of software reality. Software projects that share biological characteristics best handle the uncertainty and need for adaptation that is associated with software development. These systems require a corporate memory in terms of history data and feedback mechanisms that allow a project to learn, grow, survive and be successful. Some would call this organizational learning.

The OSDP consists of two components, the Organic Software Engineering Life Cycle (OSELC) and the Organic Software Engineering Environment (OSSEE). While the OSELC provides the universal system processes that are applicable to all software projects, the OSSEE provides the infrastructure, paradigm, models, and methodologies that allow adaptation to any type of software project situation. The OSELC replaces the SDLC. While the latter structure projects around the tasks that must be performed, the OSELC structure projects around the system processes that underlie software projects.

The OSELC system processes include systems engineering, concurrent engineering, software functional elements, organic design, quality engineering, and joint optimization. Software systems engineering defines how complex software engineering projects should be designed and managed. Concurrent engineering promotes the concurrent development of project plans and system products. It manages the entire software system engineering lifecycle though parallelization, participation of all the constituents in the ecosystems, and an emphasis on business value, customer satisfaction, productivity, and quality. Software functional elements are the new software functional model whose parts consist of systems, subsystems, software functions, capabilities, and features. Organic design partition projects, systems, and software functional elements like biological organisms that support distinct body functions. Think in terms of cardiovascular, endocrine, digestive, lymphatic and respiratory systems. Quality engineering embeds the required quality elements into the software functional elements. Joint optimization balances efficiency in terms of time, costs with effectiveness in terms of business value, customer satisfaction, and productivity.

The OSSEE provides the capabilities to assess, design, and manage the project as a socio-technical system. The Software Paradigm Model (SPM) enables a project to assess, design and manage its social system, including conceptual, managerial, organizational, and cultural constructs. Cultures in terms of attitudes, values, norms, beliefs, customs, practices, and social behavior have a significant impact on project performance. Cultures can be high performance where practitioners thrive on achievement and success or cultures can be negative like the schedule-first cultures that implement immature system when projects fall too far behind schedule. Management has a very significant impact of the project culture. So do the organizational factors that include job engineering, people, information and decision-making, project structure and the reward systems.

The system methodologies determine the manner in which software projects will plan, organized, managed, developed, and realized software systems. The system methodologies differ from the traditional methodologies in that the latter are very detailed and designed for strict adherence. System methodologies offer a set of guidelines, principles, and practices that can be tailored to a specific situational context. The OSSEE methodologies are strategic project management, project ecosystem design, tactical project management, functional element management, quality engineering, and distributed team project management. The system methodologies allow the design of a project, definition of strategic objectives and strategies, development of the tactical plans, management the software functional elements, embedment of quality, and creation of the software services. The software developers and users are assigned to distributed teams that are tasked to develop high-level software functional element like a subsystem. These integrated teams identify and prioritize the capabilities and features needed to support their business processes and then develop the operational plans needed to create the software. These practices are not ad-hoc, at project time, the systems engineer selects from the capability libraries the processes, techniques, methods, and tools that best fit the needs of the technical, project management, human resource, and organization subsystems.

 The Cause of Offshore Development

The traditional software development practices ignore the human aspects of software engineering, particularly the interactions between developers and the customers. Generally, IT staff is not the loyal “company” people that focus on the needs of the business units that they support. The problem is the closed system, ivory tower, mental model that keeps software developers aloof and disengaged from the business units.  Software engineers are trained, encouraged, and rewarded to deliver requirements specifications on time and within budget. Developers attempt to regulate requirements changes in order to meet schedules and when project fall too far behind schedule they will implement immature systems. However, business customers want high quality software systems that meet business purpose, add customer value, satisfy users, and increase quality, productivity and innovation. These differences have hardened the attitudes of many business units and executives against IT organizations. More often than not, software projects are characterized by negative interactions including finger pointing, lack of support and user involvement, conflicts, excessive politics, personnel resistance, and lack of cooperation and communication. These conflict severely affects the quality of the delivered systems.

It requires no stretch of the imagination to realize that if project developers and the business customers could improve their relationships, the results would better. The quality of the interaction and the nature of the relationships are more important to the development of quality, high performance systems than discipline processes. How well people interact with each other is essential to creating business value. People interact in order to identify the business problems, solutions, needs, project objectives, and business requirements. People also interact in order to determine and analyze the software requirements, create the designs, code the software, and test the systems. People interact in order to deploy, support, maintain, operate and train other people how to use the system.

The rancor between internal business units and internal software organizations has made it easy to offshore software engineering jobs. Business units are reluctant to pay the high American salaries for results that do not significantly improve the business. Both inshore and offshore development projects use the same problem-prone, archaic development approaches; subsequently the results from American developers and their offshore counterparts are about the same. Viewed by senior executives as costs centers that fail to produce significant business value, the decision to obtain the cheapest IT job rate, via H-1B and offshore development, is both business and social driven.   

Offshore Incompatibility

The open systems oriented OSEA provides the innovation that will significantly improve the business, technical and human-social aspects of software development. It provides business, developer, and economic needs and wants. The business units receive high quality systems that meet business purpose, add customer value, and satisfy customers through increases in system quality, productivity and innovation. Developer receives reductions in software costs by almost a factor of 10, decreases in cycle time, and attainment of schedules and budgets. The economy will see the end of the productivity paradox, the return of American-based software jobs, and a revitalized economic system. Software will become even more a significant part the value chain than enables American businesses to successful compete despite higher labor costs. As software has increased the value of physical products from cars to cameras, it can also increase the value of business services.

The OSEA is incompatible with offshore development because offshore software companies cannot support the constantly high level of interactive and relationship needs including collaboration, communication, coordination, and collaboration. High interactions are required in order to identify the right objectives, invent the right requirements, and design the right system. High interactions are needed to adapt to various situational contexts, develop the strategies to achieve objectives, develop the tactical plans needed to the concurrently engineer the project, and create and execute the operational plans needed to realize the system. High interactions are needed to receive the feedback, measure against objectives, identify the meaning of any variances, and select solutions.

Although, the American software developer cannot compete with offshore companies on a basis of labor costs, the OSEA allow them to compete on the basis of overall costs, efficiency, effectiveness, quality, time to market, and productivity. The American software engineer now has the winning home court advantage because in order to improve business value, customer satisfaction, and productivity business and developers teams must meet “eye to eye to help see things eye to eye”. Offshore developers and their H1-B counterparts will have difficult time teaming with the business managers, customers, and users because of cultural, accents, and skill nuances. Client-facing generalists with knowledge of the organizational science, behavioral sciences and system theory will replace some of the back-end specialists. Companies will have to hire American workers for customer facing jobs creating even more jobs for Americans including veterans.

There are internal risks in not changing our approach to software development, however. The Washington Time reported that “China is stepping up its long-running struggle with the United States over access to technology. China this year embarked on a campaign to target advanced industries such as aerospace, medicine and information technology for its next stage of development”. However, the Chinese may not share the affliction of many American software engineers. Some American software engineers ignore the need for change although the vast majority of software projects fail to add the business value, customer satisfaction and economic productivity. Like Keebler Elves, they happily pursue their goals of meeting schedules on time and within budget despite the constant failures. Many will never change their view because they are trapped in the dogma of the mechanistic paradigm and the hopelessness of the SDLC. While others will use the OSEA to explore the software universe, others will remain trapped in the mines of reductionism digging for that ultimate technical solution, the “Silver Bullet.”