AI-based organizational network analysis

Leveraging LLMs in HR analytics

The new generation of AI tools provide out-of-the-box solutions for complex problems that were not possible (or scalable) before — or only a minority of skilled IT professionals were able to solve them.
This applies to widely known areas like natural language processing or image processing — but also to the different areas of psychology. Nowadays, psychologists and HR people are one big step closer to be able to solve these complexities with the support of AI.

As someone with Masters in Psychology and a senior AI engineer, I’d like to share one possible way to leverage AI to improve an organization.
Let’s build a simple network extraction pipeline with LLMs and Python to detect key people in a group.

Fusing network analysis and organisational psychology can be an exciting interdisciplinary journey. E.g. Casciaro et al. (2015) advocate for the integration of network and psychological perspectives in organizational scholarship, highlighting that such interdisciplinary approaches can significantly enrich our understanding of organizational behaviors and structures. They emphasize that combining these perspectives reveals complex dynamics within organizations that would otherwise remain obscured, especially in areas such as leadership, turnover, and team performance. This fusion not only advances theoretical models but also suggests practical implications for organizational management, urging further exploration of underrepresented areas and methodologies (Casciaro et al., 2015).

Brass (2012) emphasizes the importance of recognizing how personal attributes and network structures collectively impact organizational outcomes, suggesting that a dual focus on structural connections and individual characteristics is crucial for a deeper insight into organizational dynamics.

But why is that interesting?

Because according to Briganti et al. (2018), who examined one specific psychological subject, empathy, concluded that central people within the network are crucial in predicting the overall network dynamics, highlighting their significance in understanding (empathic) interactions.

So, in short, literature has already shown that by examining key people in a network can help us predict factors for the whole network.

How to build a network?

Well, you could choose a traditional way to explore the interconnections of an organization, e.g. questionnaires, focus groups, interviews, etc. Focus groups and interviews are difficult to scale. The validity of social science research data has been the subject of a deep and serious concern in the past decades (Nederhof and Zwier, 1983) — and this was expressed in 1983! Traditional surveys often suffer from different types of biases. These might be difficulties with psychology surveys:

• social desirability bias (Nederhof, 1985) the tendency of survey respondents to answer questions in a manner that will be viewed favorably by others
• recency bias (Murdock, 1962) when more recent information is better remembered or has more influence on your perceptions than earlier data
• halo effect (Thorndike, 1920) when an overall impression of a person to influence how we feel and think about their character. Essentially, your overall impression of a person (“He is nice!”) impacts your evaluations of that person’s specific traits (“He is also smart!”)
• self-serving bias (Zuckerman, 1979) attributing positive events to one’s own character but attributing negative events to external factors, thus preserving self-esteem.

One could work with blind assessments (Rosenthal and Fode, 1963), or go to qualitative methods which are great options by giving more depth and detail about a topic (Braun and Clarke, 2006), captures complexities better (Denzin and Lincoln, 2011). Qualitative methods also suffer from limitations like the difficulty to unify and organize the results, researcher bias (Malterud, 2001), researchers’ confirmation bias (Nickerson, 1998), but in the age of LLMs, with proper prompting and automation, these limitations can be overcome.

I’ll present a simple LLM-based method to extract network relationships from natural language. Since this article’s goal is the presentation of the method, not any psychological result, I will use synthetic data: with GPT4o, I’ll generate 5 virtual people with professions and personalities. I’ll ask them three questions with the least possible fixed topics:

• Tell the story of your average workday. Your story needs to be very detailed, including tasks, people — and anything that comes to your mind. There is no good or bad answer.
• Tell a story where you felt really bad at work. There is no good or bad answer — but please make it as detailed as you can.
• Tell a story where you felt great and successful at work. There is no good or bad answer — but please make it as detailed as you can.

Let the game begin

We need some details about our imaginary team in order to generate realistic textual data to our input.

I have passed this prompt to the LLM:

Generate 5 virtual personas to me. They work on the same project. They should have
- first name
- age
- profession/position at the company
- a short description of their personalities. 
Their personalities should have negative sides as well.
At least one pair should be in a manager-subordinate relationship.

It generated the following people:

Name: Emily
Age: 29
Profession/Position: Project Manager
Personality Description: Emily is highly organized and decisive, often seen as the backbone of the team. 
She is empathetic and approachable but can sometimes be overly controlling and resistant to deviating from her planned schedules. 
Emily's insistence on efficiency can occasionally overlook the need for creative freedom.

Name: Josh
Age: 34
Profession/Position: Lead Software Engineer
Personality Description: Josh is tech-savvy and a problem solver with a passion for new technologies. 
He is introverted and sometimes struggles with communication, which can lead to misunderstandings with teammates. 
His calm demeanor is an asset, although it sometimes appears as disinterest in team discussions.

Name: Leah
Age: 26
Profession/Position: Junior Software Engineer 
Personality Description: Leah is creative and energetic, full of ideas and great at networking. 
However, her enthusiasm can sometimes manifest as impatience with slower-paced tasks or details. 
She can be overly ambitious, pushing the team towards unfeasible goals.

Name: Carlos
Age: 42
Profession/Position: Financial Analyst
Personality Description: Carlos is meticulous and deeply knowledgeable about finances 
but tends to be pessimistic and overly critical, which can dampen team morale. 
His cautious nature sometimes slows down decision-making processes, frustrating more dynamic team members.

Name: Fiona
Age: 38
Profession/Position: UX/UI Designer
Personality Description: Fiona is creative and a great listener, making her designs user-friendly. 
However, she can be overly sensitive to criticism and sometimes takes feedback personally. 
She tends to avoid conflict, which can lead to unresolved issues in the team dynamics.

Now I asked the LLM to act as these people, and asked the three given questions. Since I’m talking to a model, with my prompt I make sure to include some interpersonal relationship information. To spare some space and reading time, we’ll go through only one person. Example prompt:

You need to act now like Emily. 
You are taking part in an online HR assessment. 
Here are some data about your profession, personality, etc:

Name: Emily
Age: 29
Profession/Position: Project Manager
Personality Description: Emily is highly organized and decisive, often seen as the backbone of the team. She is empathetic and approachable but can sometimes be overly controlling and resistant to deviating from her planned schedules. Emily's insistence on efficiency can occasionally overlook the need for creative freedom.

In this assessment, you, Emily, are asked these three questions:
- Tell the story of your average workday. Your story needs to be very detailed, including tasks, people - and anything that comes to your mind. There is no good or bad answer. 
- Tell a story where you felt really bad at work. There is no good or bad answer - but please make it as detailed as you can. 
- Tell a story where you felt great and successful at work. There is no good or bad answer - but please make it as detailed as you can.

Your team consists of the following people, if you have any relationship with them, please include in your answers:
<Description of other teammates>

The results for Emily:

1. Average Workday:
My typical day starts with a quick review of my to-do list, which I prepare the night before. 
I arrive at the office by 8 AM, settle in, and start with a 30-minute session to review the progress on ongoing projects against their timelines. 
At 8:30 AM, I hold a brief stand-up meeting with the team, which includes Josh, Leah, Carlos, and Fiona, to catch up on their progress and address any immediate concerns.

After the stand-up, I typically meet with Carlos to discuss budget updates or financial issues. 
His insights are crucial, though I occasionally need to guide the conversation to keep it from veering into 
overly pessimistic territory, which helps us stay focused on solutions rather than problems.

Mid-morning is usually reserved for one-on-one sessions with team members needing guidance. 
For instance, I often spend time with Fiona reviewing design timelines. 
She appreciates clear, empathetic feedback which I strive to provide, though I must balance this with the need to maintain our project schedule.

Lunch is a quick affair, often at my desk, as I catch up on emails and plan for the afternoon. 
Post-lunch, I review Leah's latest marketing initiatives or software developments. Leah's energy is infectious, 
though her ambition sometimes requires tempering to align with our practical deliverables.

My afternoons often involve strategic planning sessions, where I work with Josh 
to align our technology needs with business goals. 
His technical expertise is invaluable, though I sometimes mediate 
to ensure his ideas are clearly communicated to the rest of the team.

I end my day by updating our project management tools, responding to any final emails, 
and preparing the next day’s priority list. Before leaving, 
I always walk around to say a quick goodbye to the team, ensuring I leave on a personal note.

2. Negative Experience:
A particularly challenging day occurred when we were nearing a major project deadline. 
Tensions were high, and despite my efforts to keep the team on track, a critical integration flaw was discovered. 
The error was in a section Josh had worked on; his usual calm demeanor made it difficult to gauge his concern, 
which initially seemed like disinterest. This misunderstanding escalated under pressure.

I felt overwhelmed and responsible, fearing the mistake would reflect poorly on my ability to manage. 
It was a stark reminder that my need for control and perfection could blind me to the team's stress levels. 
That day ended with us missing a deadline, a first for me, which deeply upset me.

3. Positive Experience:
One of my proudest moments was leading the team during the launch of a new product that was pivotal to our company’s expansion. 
The project had complex components, requiring tight coordination between all departments. 
Despite the challenges, including balancing Carlos' cautious financial strategies 
with Leah's ambitious marketing plans, we executed a flawless launch.

The success was made sweeter by the positive feedback from upper management and the client’s satisfaction. 
It reinforced the value of my organizational skills and my ability to gently 
but effectively guide the team through high-pressure situations. 
That day, I truly felt accomplished, seeing the tangible results of our hard work and my leadership.

The rest of the prompts and results can be found here: llm-network-extraction/ at main · sugartrauma/llm-network-extraction (github.com)

Extracting interactions

Now that we have the input for our network analysis, we need to extract the relationships from the mentioned interactions.

You are a social network extraction tool.
You are given an input marked in <input> tag.

Extract the relationships between people mentioned in the <input> text and 
structure them into a CSV format with columns: 
- source
- target
- edge_type
- interaction_type

The relationships can be of two types: knowledge and personal. 
The interaction types for knowledge can be "gives knowledge" or "receives knowledge," 
For personal, the interaction_type can be "positive interaction" or "negative interaction."

Example input:
<input>
Josh helps Leah with coding challenges, sharing his expertise regularly. 
Emily and Fiona often collaborate creatively and have a positive working relationship.
</input>

Example output:
source,target,edge_type,interaction_type
Josh,Leah,knowledge,gives knowledge
Emily,Fiona,personal,positive interaction

The relationship must be represented only once.
You must not represent an edge in the opposite direction.
Bad example:
source,target,edge_type,interaction_type
Josh,Leah,knowledge,gives knowledge
Leah,Josh,knowledge,receives knowledge

Good example:
source,target,edge_type,interaction_type
Josh,Leah,knowledge,gives knowledge

<input>
Input comes here
</input>

The LLM duplicated some relationships like:

Josh, Leah, knowledge, gives knowledge
Leah, Josh, knowledge, receives knowledge

I deduplicated them and started the actual network analysis.

Although I’m fluent in Python, I wanted to showcase GPT4o’s capabilities for non-programmers too. So I used the LLM to generate my results with this prompt:

Please build a network in Python from this data. 
There should be two types of edges: "knowledge", "personal". 
You can replace the textual interaction_types to numbers, like -1, 1. 
I need this graph visualized. 
I want to see the different edge_types with different type of lines and the weights with different colors. 

I’ve retried many times, GPT4o couldn’t solve the task, so with the good old-fashioned ways, I generated a graph visualization writing Python code:

import networkx as nx
import pandas as pd
import matplotlib.pyplot as plt
from matplotlib.colors import LinearSegmentedColormap

cleaned_data = pd.read_csv(<file_destination>)
# For knowledge, we don't punish with negative values if there is no sharing
# For personal relationships, a negative interaction is valued -1
for idx, row in cleaned_data.iterrows():
    if row["edge_type"] == "knowledge":
        # If the source received knowledge, we want to add credit to the giver, so we swap this
        if row["interaction_type"] == "receives knowledge":
            swapped_source = row["target"]
            swapped_target = row["source"]
            cleaned_data.at[idx, "target"] = swapped_target
            cleaned_data.at[idx, "source"] = swapped_source
        cleaned_data.at[idx, "interaction_type"] = 1
    elif row["edge_type"] == "personal":
        cleaned_data.at[idx, "interaction_type"] = -1 if row["interaction_type"] == "negative interaction" else 1

# Aggregate weights with a sum
aggregated_weights = cleaned_data.groupby(["source", "target", "edge_type"]).sum().reset_index()

# Filter the data by edge_type
knowledge_edges = aggregated_weights[aggregated_weights['edge_type'] == 'knowledge']
knowledge_edges["interaction_type"] = knowledge_edges["interaction_type"].apply(lambda x: x**2)
personal_edges = aggregated_weights[aggregated_weights['edge_type'] == 'personal']
personal_edges["interaction_type"] = personal_edges["interaction_type"].apply(lambda x: x**2 if x >=0 else -(x**2))

# Normalize the weights for knowledge interactions since it has only >= 0 values, so the viz wouldn't be great
if not knowledge_edges.empty:
    min_weight = knowledge_edges['interaction_type'].min()
    max_weight = knowledge_edges['interaction_type'].max()
    knowledge_edges['interaction_type'] = knowledge_edges['interaction_type'].apply(
        lambda x: 2 * ((x - min_weight) / (max_weight - min_weight)) - 1 if max_weight != min_weight else 0)

# Create separate graphs for knowledge and personal interactions
G_knowledge = nx.DiGraph()
G_personal = nx.DiGraph()

# Add edges to the knowledge graph
for _, row in knowledge_edges.iterrows():
    G_knowledge.add_edge(row['source'], row['target'], weight=row['interaction_type'])

# Add edges to the personal graph
for _, row in personal_edges.iterrows():
    G_personal.add_edge(row['source'], row['target'], weight=row['interaction_type'])

custom_cmap = LinearSegmentedColormap.from_list('red_green', ['red', 'yellow', 'green'])

# Find the knowledge center
knowledge_center = knowledge_edges.groupby("source").sum().idxmax().values[0]
least_knowledge_node = knowledge_edges.groupby("source").sum().idxmin().values[0]

# Draw the knowledge interaction graph with arrows
plt.figure(figsize=(12, 8))
pos = nx.spring_layout(G_knowledge, k=0.5, iterations=50)
nx.draw_networkx_nodes(G_knowledge, pos, node_size=100, node_color='lightblue')
knowledge_weights = [d['weight'] for u, v, d in G_knowledge.edges(data=True)]
nx.draw_networkx_edges(G_knowledge, pos, edgelist=G_knowledge.edges(), edge_color=knowledge_weights, edge_cmap=custom_cmap, edge_vmin=-1, edge_vmax=1, width=2, arrows=True)
nx.draw_networkx_labels(G_knowledge, pos, font_size=14)
plt.title('Knowledge Interactions')
plt.annotate(f'Knowledge Center: {knowledge_center}', xy=(1.05, 0.95), xycoords='axes fraction', fontsize=14, color='darkred')
plt.annotate(f'Least knowledge sharing: {least_knowledge_node}', xy=(1.0, 0.85), xycoords='axes fraction', fontsize=14, color='darkred')
plt.axis('off')
plt.show()

# Find the personal center 
personal_center = personal_edges.groupby("source").sum().idxmax().values[0]
least_personal_center = personal_edges.groupby("source").sum().idxmin().values[0]

# Draw the personal interaction graph
plt.figure(figsize=(12, 8))
pos = nx.spring_layout(G_personal, k=0.5, iterations=50)
nx.draw_networkx_nodes(G_personal, pos, node_size=100, node_color='lightblue')
weights = [d['weight'] for u, v, d in G_personal.edges(data=True)]
nx.draw_networkx_edges(G_personal, pos, edgelist=G_personal.edges(), edge_color=weights, edge_cmap=custom_cmap, edge_vmin=-1, edge_vmax=4, width=2, arrows=True)
nx.draw_networkx_labels(G_personal, pos, font_size=14)
plt.title('Personal Interactions')
plt.annotate(f'Personal Center: {personal_center}', xy=(1.05, 0.95), xycoords='axes fraction', fontsize=14, color='darkred')
plt.annotate(f'Least positive person: {least_personal_center}', xy=(1.05, 0.85), xycoords='axes fraction', fontsize=14, color='darkred')
plt.axis('off')
plt.show()

The results for knowledge sharing network:

We can find out that except for Carlos, everyone is quite close in the knowledge sharing ecosystem. Emily is the node with the most outgoing weight in our graph.

What can we do with that data?
1. We should definitely keep Emily at the company — if we need to pick one person to give maximum effort from benefits and to receive long-term engagement, that should be Emily.
2. Carlos is a financial analyst, which is quite far from the actual work of the team. It might not be a problem that he doesn’t share that many information. The crucial part might be seen on the other part of the graph, which we don’t have — how much knowledge does he share in the finance team. So be careful with interpreting results that might look bad at first glance.

The results for network of the positivity/negativity of interactions:

It can be seen that Leah, our Junior Software engineer is the most positive person based on the number of positive interactions.
1. As an action item, we could start a mentor program for her, to be able to make her positive attitude viral and facilitate her to gain professional experience to increase her trustworthiness in all areas of work.
2. Emily is the person with the least positive, and most negative interactions. As a project manager, this is no wonder, PMs often have do make difficult decisions. On the other hand, this might need a double check to see if the negativity of her interactions come form her PM duties or her actual personality.
Again, don’t assume the worst for the first sight!

Summary

In this article I shared a novel method to extract and analyse organizational social networks with LLM and graph analysis.
Don’t forget, this is synthetic data, generated by GPT4o — I showcased the technology rather than actual psychology-related findings. That part might be the next target of my research if I’ll have access to real-life data.
Hopefully, this small project can be a facilitator for deeper research in the future.

I hope you enjoyed the article, feel free to comment.

Sources:

Brass, D. J. (2012). A Social Network Perspective on Organizational Psychology. Oxford Handbooks Online. doi:10.1093/oxfordhb/9780199928309.013.0021

Braun, V., & Clarke, V. (2006). “Using thematic analysis in psychology.” Qualitative Research in Psychology, 3(2), 77–101. This paper discusses how thematic analysis in qualitative research can uncover rich and detailed data.

Briganti, G., Kempenaers, C., Braun, S., Fried, E. I., & Linkowski, P. (2018). Network analysis of empathy items from the interpersonal reactivity index in 1973 young adults. Psychiatry Research, 265, 87–92. DOI: 10.1016/j.psychres.2018.03.082

Casciaro, T., Barsade, S. G., Edmondson, A. C., Gibson, C. B., Krackhardt, D., & Labianca, G. (2015). The Integration of Psychological and Network Perspectives in Organizational Scholarship. Organization Science, 26(4), 1162–1176. DOI: 10.1287/orsc.2015.0988

Denzin, N. K., & Lincoln, Y. S. (Eds.). (2011). “The Sage Handbook of Qualitative Research.” Sage. This handbook discusses the strengths of qualitative research in capturing the complexities of human behavior and social phenomena.

Malterud, K. (2001). “Qualitative research: standards, challenges, and guidelines.” Lancet, 358(9280), 483–488.

Murdock, B. B. (1962). “The serial position effect of free recall.” Journal of Experimental Psychology, 64(5), 482–488.

Nederhof, A. J. and Zwier, A. G. (1983). ‘The “crisis” in social psychology, an empirical approach’, European Journal of Social Psychology, 13: 255–280.

Nederhof, A. J. (1985). Methods of coping with social desirability bias: A review. European Journal of Social Psychology, 15(3), 263–280. doi:10.1002/ejsp.2420150303

Nickerson, R. S. (1998). “Confirmation bias: A ubiquitous phenomenon in many guises.” Review of General Psychology, 2(2), 175–220.

Rosenthal, R., & Fode, K. L. (1963). “The effect of experimenter bias on the performance of the albino rat.” Behavioral Science, 8(3), 183–189.

Thorndike, E. L. (1920). “A constant error in psychological ratings.” Journal of Applied Psychology, 4(1), 25–29.

Zuckerman, M. (1979). “Attribution of success and failure revisited, or: The motivational bias is alive and well in attribution theory.” Journal of Personality, 47(2), 245–287.

AI-based organizational network analysis was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.