Intro

Fredrik Brounéus 0:10

Welcome to Drug Safety Matters , a podcast by Uppsala Monitoring Center , where we explore current issues in pharmacovigilance and patient safety . My name is Fredrik Brouneus and I will be your host for today's episode . The narrative fields in individual case safety reports can carry information that is crucial for the discovery of new adverse drug reactions , but they also require significant amounts of manual labor , both on the side of the reporter and of the assessor . To explore how pharmacovigilance professionals interact with these narrative fields in their work , my Uppsala Monitoring Centre colleagues , Alem Zekarias , Senior pharmacovigilance scientist , and Joana Felix , Medical informatician , performed an interview study with assessors from different parts of the world . Today , Joana and Alem are joining me in the studio to tell me and you about the results and give us some tips on how to write the perfect narrative .

Fredrik Brounéus 1:13

Welcome to the Drug Safety Matters studio , Alem .

Alem Zekarias 1:16

Hi, thank you .

Fredrik Brounéus 1:17

. . and Joanna

Joana Félix 1:18

Hi , thank you so much .

Fredrik Brounéus 1:25

I've really been looking forward to talking to you about this study of yours , where you have looked at how pharmacovigilance assessors interact with narrative fields in individual case safety reports . Now I'd like to start off with a very basic question , just so we're all on the same page here . What do we mean when we say narrative fields ?

Joana Félix 1:42

So , yes ,

What are "narrative fields"?

Joana Félix 1:43

so narrative fields are parts of the individual case safety reports and this is where you would find relevant clinical details about a patient experience with a suspected ADR . So here you could see it as a standalone medical story where you bring everything together to understand what happened

Fredrik Brounéus 2:04

Right . . And then how much text are we talking about ?

Joana Félix 2:08

So normally because you want to include patient characteristics , details on their therapy , medical history and all of the clinical course right now , in the current standard , you can go until 100,000 characters , which it would mean around 24 pages A4 of text .

Fredrik Brounéus 2:30

That's a lot of text .

Joana Félix 2:31

It is , yeah .

Fredrik Brounéus 2:32

And what would be a normal ? Because I would suspect that not everyone would write 24 pages

Joana Félix 2:38

Definitely , definitely no . It's I normally , Definitely no . It normally would be maybe less than a page . I would assume around that .

Fredrik Brounéus 2:51

? Do you have to add a narrative to a report to be able to submit it ?

Alem Zekarias 2:56

No , actually . So the minimum information for a valid safety report should be at least one identifiable patient , one ADR , one reporter , and there should be a drug . So on the question , if all reports do have them , unfortunately not . So, free text clinical information does have an important role during the signal assessment , as Joana just provided us , and it differs actually . It's quite hard to quantify what reports, you know the amount of data , but for some reports they are quite lengthy but also repetitive maybe , and for others there are just a couple of sentences more short and sweet on the point . So it differs actually, but also between stakeholders . We can see some stakeholders really like to write more lengthy report , which can be good and complement the structure information , while other reporters do keep it very , very short .

Fredrik Brounéus 3:46

Could you tell me why you decided to carry out this study ? What were the main reasons behind

Why this study?

Fredrik Brounéus 3:53

it ?

Alem Zekarias 3:53

So , having worked both of us with signal detection and assessment for years , we acknowledged , of course , the importance of narratives in signal management and in pharmacovigilance in general . And pharmacovigilance assessors review ICSR narratives to determine causality , severity , but also this expectance of adverse events . And they may use like different tools , like coding systems or algorithms guided by , like regulatory templates or their expertise . But despite this general guidelines on the narrative structure , we know quite little how assessors interact with the narratives during the signal management and what drives this interaction . So we wanted to study and to see like so the study aimed to identify needs and opportunities to support assessors in their work . So we wanted to focus on the challenges and the needs in order to be able to advance the pharmacovigilance science and achieve meaningful impact .

Fredrik Brounéus 4:46

Right . So is this the first study of its kind ?

Alem Zekarias 4:49

To the best of our knowledge, yes .

Fredrik Brounéus 4:51

That's very exciting , then , I think , because it's groundbreaking work , this exploratory study of yours . So how did you perform this

How was the study performed?

Fredrik Brounéus 5:00

study ?

Joana Félix 5:00

So , as Alem was saying , this is exploratory because we believe that nothing has been done on this topic before . So with that sense , we use the qualitative framework and we designed some interview guidelines with some structure around it and we then just selected participants based on our network and based on , also , their hands-on experience in PV . So we wanted experts in the pharmacovigilance area . So basically , through those contacts and networks , we arranged these in-depth interviews . They were mainly online and so we used the interview guide and after that we just transcribed it , did the dirty work and recorded it , transcribed it and then we did an analysis based on the themes we would find .

Fredrik Brounéus 5:56

Right , and how many participants were there in all ?

Joana Félix 6:00

There were five participants from four different countries .

Fredrik Brounéus 6:04

I know that you cannot go into exactly which countries these were , but could you say which regions they covered ?

Alem Zekarias 6:13

Yeah , we can say so . All five participants do represent almost all regions around the world and both low and middle income as well as high income settings and , as Joana said , they have a wide , broad experience in hands-on pharmacovigilance , which was the key here , because the default we really didn't want to have the textbook answer . We really want to have the truth , like how do you actually work with the interaction with the case narratives , and to be able to capture that real or true information, we wanted to have experienced assessors .

Fredrik Brounéus 6:48

So did you see any cultural differences there ?

Alem Zekarias 6:51

No , actually . So , as I said , like they had different backgrounds , you know , different culture , regions , and we had pharmacists as well as medical doctors , and I think the main difference here , you know , they were more related to the experience of working with ICSR rather than related to where they come from , whereas , of course , what resources a center has also plays an important role , rather than what part of the world the assessors came from .

Joana Félix 7:16

I would definitely agree with the resource part . I think this where because we're analyzing or describing interactions is really , really dependent on the resources that you have .

Fredrik Brounéus 7:28

And if we jump forward into the results of your study , your interview questions focused on three areas when , why and how assessors interact with the case narratives , and when analyzing these answers , you identified four main themes . The first is to do with the assessor's view of the narrative's content and value . The second is about the processes where the interaction with narratives takes place . And then the third theme is about the challenges that the assessors experience when interacting with the narratives .

Fredrik Brounéus 8:04

And the fourth and final theme , then , is about the needs , or their needs , how they think that their interaction with narratives could be improved or enhanced in different ways . So if we start with the first theme here content and value could you describe what you found there ?

Joana Félix 8:21

Yeah

Theme 1: Content and Value

Joana Félix 8:22

, for sure . So I think , although they came from different regions and backgrounds , I think overall all of the interviews emphasize really how absolutely fundamental the narratives are for their signal management tasks . So they really see it as the core , the heart of the report . However , like not all reports or in this case , narratives , are complete and some lack key details that are really essential for them to really understand the adverse events . So , at the same time that it's seen as a vital source of clinical information that they can't get from any other field , it's also an obstacle and a challenge to their work .

Fredrik Brounéus 9:06

Considering how important the narrative fields are coming from , a more well , perhaps a more naive question would be wouldn't it make sense for it to be a compulsory part of the reporting , just as the other fields are ?

Alem Zekarias 9:23

That's a good point , but again , you know it is also, we have the structure part of the report as well as the free text information and you do capture quite valuable information with the structure field as well . You know drug and you know what kind of indication , patient characteristics and so on . But it is also time consuming and challenging to report an ADR and to make that part also mandatory for making a report valid can be a challenge , of course , but as we saw in this study that yes , they do see the value narratives , as we can confirm both Joana and I were working with this as well but also examples of information that could contain in these narratives could be a comprehensive , as you said , clinical picture that complements the structured information , but also information regarding risk factors . Can you identify some risk factors for this patient which , of course , may increase the likelihood for developing a specific ADR ?

Alem Zekarias 10:23

Temporal information, do we know what , when , how you know the patient took the drug , for example ? But also motive to report was something one of the interviewees really highlighted , which we also can understand and respect , because someone wants to say something with a report and trying to understand what the reporter wants to convey with the report is important and you can't really always , you know , collect that information in the structured field , and that's why the narrative information is so crucial , since it can , you know , have the possibility to complement the structured information . Then you understand why did this reporter actually take the time to report this ADR .

Fredrik Brounéus 11:03

So then, as I would interpret your answer also , it's sort of a balancing act here too: we want to have as much information as possible , but at the same time we want to make it as easy for the reporters as possible to report , and so if we demand too much information , we risk not getting any information at all .

Joana Félix 11:23

Yeah , and we live in a field that is really marred by underreporting , so definitely that balance is key , like quality , information and quantity and motivating people to provide rich narratives , but also facilitating that process , I think , is key .

Alem Zekarias 11:40

And I also think , like if the reporter understands the importance of including case narratives and also one , you know , one important aspect , because , like we are all humans , of course you've been driving if you have this motivation , if you understand the impact of providing this case narratives , maybe you feel even more like engaged and motivated to report this information . But if you don't really understand or see the value of reporting , of course it makes it even harder to expect the reporters to also share case narratives .

Fredrik Brounéus 12:12

From a communications point of view , I would say that sounds like a very important aim , with your study as well , to actually convey to reporters how important these narrative fields are for the assessors . So if we move on to the second theme here , the processes , this is to do with the when when the assessors interact with the narratives , but also how they do it . Is that right ?

Theme 2: Processes

Alem Zekarias 12:38

Yes , that's correct . So most interviews highlight the importance of narratives in signal detection , but a case analysis , but also during code validation , emphasizing the manner of interaction with the narratives fields without digital support . So the narrative plays a crucial role in coding specific fields . There's meaning like drugs , adverse events and validating pre-processed codes , with some participants using automated coding models , for example . So during prioritization , narratives are also reviewed to identify series on unexpected cases . This is also a manual process and this is because , you know , during the pandemic we saw a steep increase of number of suspected adverse drug reports,

Alem Zekarias 13:20

But that's like in general when you do work with signal detection assessment . We talked about underreporting before , but for some combination , we can say almost that we have an overreporting and to do that manually , also to know , like which of these you know , in this amount of data in front of me , which reports are the most informative ones here and how do I know where to start in this detective work which is signal detection and assessment is a detective work . So that's also something that we could identify in this study , but also in case assessments . You know , narratives provide essential information , with some assessors using tools like ChatGPT for assistance in analyzing free text and additionally assessors often reach out to reporters to clarify or complete missing details in the narratives which we also could identify . That having that possibility to go back to the primary reporter for asking them follow-up or confirmatory questions to what they have shared before .

Fredrik Brounéus 14:22

I think this is very interesting , because a lot of what you describe is it's heavy on the manual labor . This and here we have one participant describing how they use chat GPT . I would expect that that is something we will be seeing a lot more of in the future . What are your reflections on this use here of chat GPT ? Are there quality concerns or are there ?

Joana Félix 14:48

Yeah , yeah , definitely . I think it's a matter of highlight and maybe concern as well , but I think the most important thing in this case specifically highlights how assessors have a task and sometimes they will come up with their own solution to fill those gaps that they know they have . So , for example , in this case specifically , the participant felt that they lacked clinical expertise and they had low resources around them so not a lot of resources that could complement their clinical expertise so they turned to ChatGPT as a tool to support their work , in a way to complement their work . They would always say that they would not be making the decisions for them , but it would be as a friend , almost as a colleague , and I think that's really interesting , because the gaps are there and if we don't actively engage with them and look for them , they will be filled by a solution .

Alem Zekarias 15:49

I agree in this specific case , I think there was a very innovative solution actually to do that . We did the study some time ago now and to be able to know that the strengths but also the limitations which this interviewee knew with it , and to use more Chat GPT for a sounding board was very wise in this that this interview didn't make any decision based on that , but more like a sounding board due to limited resources .

Fredrik Brounéus 16:15

Then that brings us to the third theme , which is also very interesting the challenges that the assessors experience in their interaction with the narratives . Could you tell us a bit more about your findings

Theme 3: Challenges

Fredrik Brounéus 16:29

here ?

Alem Zekarias 16:29

So some narratives are time consuming due to the length and repetitive nature , and extracting relevant data from these narratives , such as timing , risk factors and lab information , for example , is a manual and labour intensive process . So three interviewees noted the challenge of identifying which narratives are informative , as I said before , as the relevance of information varies depending on the clinical situation . Additionally , two interviewees highlighted the difficulty reporters face in writing complete and relevant narratives , which often lead to irrelevant or duplicate information . As I said before, like so narratives , there are certain standards and guidelines how you should write them . If you just repeat what's already been captured in the structure information , they're not really the purpose with the narratives . It's not really . It's pointless , useless , if I can say it , you know , using those words . The narratives do have an important role in reporting adverse drug reactions and I think , like that's , there's like in this amount of data for some combinations, of course , it can be hard to really find the most interesting , helpful case in this specific assessment if they are hidden in this amount of information .

Fredrik Brounéus 17:40

Thinking about this identified challenge here . You say the sort of the lack of knowledge among reporters on how to write a complete narrative , and you touched upon this already . Could you give some more tips on how to write the complete narratives or the perfect narrative, so to say ?

Joana Félix 18:10

So, I have a medical background, and when you have medical studies, narratives, the medical background, is key . I think one thing that is very clear is that things that before maybe we take for granted , that are given to you during education in the field , could be emphasized here , maybe to some education and training about . There are these guidelines , so let's show them and expose them to the actual users . So these guidelines like the CIOMS Working Group 5 , for example , just describes a little bit what Alem has already said . Like , in order to have a complete narrative , you want all the relevant details . You don't want to just repeat , but you want to make sure that everything you need to assess that case is in the narrative . And especially , the most important thing is to have an idea , because you have this piece of paper or this digital form where it's a moment in time . It's not longitudinal .

Joana Félix 19:05

So the idea is that the narrative can provide you more this sequence of time , this logical progression and clinical progression of the adverse event .

Joana Félix 19:15

You can add information there that you can add in other fields , obviously , but they should be connected to the story . So if you have laboratory work , if you have risk factors , if you have other causes , they should be written as the story of the patient and the adverse event and in that way , when you read it , it's seamless , and everything that is relevant is integrated with the chronology and that's basically it . The point is that sometimes , in order for you to write a clinical narrative , you have to understand what is important at each step , and when you have a form , everything is important . Everything has the same fields besides the mandatory information . It's hard to place each field on a specific timeline . So that is the key , I think , is making sure that , especially that you don't provide contradictory information , that you just provide in the structured field and that the information that you have in the narrative follows this logic , this sequence .

Fredrik Brounéus 20:27

Are these tips present in the reporting form in some way ? Do they get a pop-up with with tips or ?

Joana Félix 20:35

Currently , I believe no . No , not at least you know , in our forms . No , I think what has been written in the . . . obviously , in terms terms of medical narratives , a lot has been written before . There's a lot of guidelines and it's teaching material for medical doctors , but I think in terms of pharmacovigilance it's very sparse and sometimes it's integrated in regulatory documents . So you have to really dig . That's literally what I did . You have to read regulatory guidance . That will lead you to some guideline that has been written and there you can find it . But I guess , as a reporter , you might find it difficult , especially because this regulatory material it's not as global as PV is .

Fredrik Brounéus 21:21

Do we know anything about how reporters interact with the narratives ?

Alem Zekarias 21:26

I think all reporters would provide this information if they knew and understood the importance of providing this . As I said before , it's about understanding why it is important to report if you experience an adverse drug reaction . So understanding the importance will hopefully motivate the reporter to report this information as well , even though a report is considered valid without it . But also , as Joana said , there are guidelines how to write a report . Reporters may also think that the information they have may not be relevant for that specific report . And I think like, try to report everything you have , and because that information may be an important piece in the assessment as a whole , together with other types of reports , maybe we see that that specific drug is taken together in five other reports from other parts of the world . So I think it is important to try to stick to the truth , of course , but write down all information you have , since it can be important in the end .

Joana Félix 22:22

Yeah , I think that's really key . I think the fact that even if there are guidelines on how to write a report , there aren't really guidelines on how to write a report for pregnancy , for drug-induced liver injury Each case can be different . So I think the idea of right now providing everything that you have, it could be a good starting point , because there's a lot of nuances to this .

Fredrik Brounéus 22:47

And I think that's a crucial aspect that you mentioned here, that actually the report in itself is not the full story , but the report will be a piece of a much bigger puzzle , and for the reporter you don't know which part of this piece of the puzzle that you are providing that will be interesting . Like you said , it may be aspects there that will tie together with other aspects in other A nd I I would guess as a reporter , it's hard to have that kind of bird's eye view exactly .

Joana Félix 23:23

Yeah . So to your point, it's like when you're reporting , it's really hard to know the global impact that this may have or not , and how will assessors pick this ? So I think that sometimes I think although again , we haven't done a study on this , but I can assume that a reporter might already write something with an idea , with a lens , of what this could be, like or what this or why did this happen . But I agree that most of the times you just don't know . You're hoping to provide some insights but you're not really sure which ones are more important . Yeah , that's a great point .

Alem Zekarias 24:04

And then if you don't really hear anything back , then you start to wonder, what did my report or reports – do they have any impact or value , or what's the point for me to report in the end ? So that's also very important for the national centers to , you know , provide the reporters with feedback , but also , depending on what happens to those reports , to give the providers a feedback which can motivate them to share even more reports in the future .

Fredrik Brounéus 24:31

Talking about that feedback , how often do you get feedback on your individual reports ? Because it would be impossible for a center to provide feedback to everyone .

Joana Félix 24:41

So I only know of one case , at least that has reached my knowledge , that for example the Netherlands center, Lareb, I don't know if they still do it . I think it stopped or at least was a little bit challenging during the pandemic . They would write feedback for each reporter . So in a sense exactly meeting what you just said . I don't know if they still do it . I don't know if other national centers do it , but definitely I think it's something that has been thought about .

Alem Zekarias 25:12

There are actually some national centers that have this automation, you know , thank you, for acknowledging the report and then , in some cases , giving them feedback what happened to the report itself . And that's also something that we try to educate . You know , when we have contact with the national centers to try to keep the reporters engaged , you know , involved and at least acknowledge them for taking the time to share the reports . So there are , but I know some countries that reward reporters , but they have shared reports and they have this yearly event where they reward medical doctors or pharmacists , you know , reporters , depending on how many reports they have shared over the year . And I think that's a very good way to really keep motivating the reporters but also to acknowledge that they are doing a very important role .

Joana Félix 25:57

And that it's seen .

Alem Zekarias 25:58

It's seen – exactly, because these reports are crucial for us to be able to drive pharmacovigilance forward .

Fredrik Brounéus 26:04

So if we then move on to the final theme here , the needs of the assessors as they interact with the narratives , in this theme you have gathered the assessors' own suggestions and ideas for how their interaction with narratives could be made , so to say , easier and also more fruitful , and we've talked a little about this already as well , but it is a lot to do with different kinds of machine assistance . Was this expected ?

Theme 4: Needs

Joana Félix 26:33

Yes , I think the the suggestions align pretty well with what we might expect , given the challenges that these assessors face . They're looking mainly for tools that can streamline their work and improve efficiency . As you said , they're very manual tasks and they do this case by case normally , so they're very repetitive and sometimes in areas they lack expertise because it's a very diverse field and a lot of clinical details . So , for example , there was one mention , I think , of this that goes , also aligned with what we just said, automated guidance . So it was a lot about automation , but like automated guidance for when writing narratives or for when you were reporting a little bit what should be in it and what is relevant or not .

Fredrik Brounéus 27:24

So just looking at how fast things are moving in these areas with automated assistance of different kinds of machine learning , AI assistance and so on , is it possible to say anything about where you think we will be in , say , two years' time , or five years' time ?

Joana Félix 27:45

I would say that I think if you would ask this question two years ago , the answer might be different . I think there's this Gen I wave and the fact that it's available for everyone to use and it's free . So , as you see here , some assessors already use it and it's not even something they have developed in-house . So definitely , I think it's moving faster now . I think it might reach a plateau . I don't know if we'll get better and better models every year , but I think , with a lot that it's out there already and some of them are open source, a lot of this may have been impossible tasks from the past or that required a lot of efforts , especially technical efforts might be facilitated . So I think there's a lot of doors being opened .

Joana Félix 28:38

I don't know if we will revamp the PV system I don't believe so in two years but I think it's part of the picture now . So we have to be aligned with it and I think it's moving . I think it's moving there that way , whether we like it or oppose . So , yes , I think , as hope and a lot of things , this large language model especially , and this idea that we can start trusting them , should be reflected . I think this is , I think , the biggest reflection that we should have , that we believe that maybe they can accomplish the tasks . But should they ? So those are the interesting questions and I think , if we're moving forward in that area , those are the questions that we need from our regulators and our experts . Really , if we are going there and I believe we currently are, we need to know some things before .

Alem Zekarias 29:44

I agree with that , because there is a big need , to automize things in the work we're doing at the moment and to automize the things that you can do, with respect of the risk of data protection , of course , but other things you have to consider , have in mind . We need to find ways to how to have the human in the loop , always involved , of course , to validate and make sure that everything is in good standard . But I think , as Joanna just said , like we are moving there , even if we like it or not . We just need to find ways how to do it and to be aware about the risk with using them without quality , assuring them , but also to do this in collaboration with stakeholders , as the regulators , for example

Joana Félix 30:26

And include everyone , ideally everyone .

Joana Félix 30:30

This should be an opportunity for everyone , so it should be regulated , but also equal . I think that's for us, I think our global vision is exactly that . So , it's not that some should move faster than others .

Joana Félix 30:46

That's at least what we believe .

Alem Zekarias 30:48

Exactly, and some of the manual tasks that we do within signal assessment, signal detection could be automized and that's so the assessor can be focused on the assessment part instead . You know the causality part . Is there a relationship here between the drug and ADR or the vaccine and the ADR ? And I think that could be a good help to be able to release the assessors for this administrative task and instead focus on the assessment itself .

Joana Félix 31:14

So it's kind of two coins .

Joana Félix 31:17

So it's this very procedural , repetitive that it's just like it's a pain sometimes , and then you have the very complex which is normally at the end of this . So you have two dimensions really where I guess if , in regards to your question , maybe the first one like this procedural part , this process , this very bureaucratic and repetitive , is something that having a human in the loop and having these tools is something that maybe we are more comfortable having something like this and then just train , educate , give time for the other very complex , which is at the end of that process .

Fredrik Brounéus 31:58

So more fun work for the assessors .

Joana Félix 32:01

Exactly , please .

Fredrik Brounéus 32:05

Now , if we're looking at the study as a whole , what do you think is the most important takeaway

Takeaways

Fredrik Brounéus 32:12

?

Alem Zekarias 32:12

So we identified several key insights from the study that present , you know , actionable opportunities with the potential to drive meaningful impact . So the primary objective of the study was to deepen our understanding of an area that has not been explored . So , by conducting research in this nature , we gained the ability to gather real-world data that highlights genuine challenges , but also opportunities for improvement . We knew from before , which was also well confirmed in the study , that case narratives are important when trying to assess a possible connection between a medicine and an adverse drug event . To be able to continue doing pharmacovigilance and ultimately save patients and consumers , we must together acknowledge that the narratives are important , but there is a huge need , which is like opportunities in the interactions with it , but also , of course , how to write a narrative is also very important , and there is a big need for education , training , but also other you know tool systems that can support this .

Joana Félix 33:13

Yeah , I think the key thing here is that , as researchers working at a global lens through the Uppsala Monitoring Center , engaging in initiatives like this and talking to where the cycle begins I guess where the report begins . So and I think if we don't do it , normally there's a lot of degrees of separation and in our work , a lot of it is based on assumptions , and I think validating assumptions and doing research about the knowledge that they have and the challenges that they have , I think is key because we can be very separated from the main thing .

Fredrik Brounéus 33:56

So you are both very experienced in the PD field , so to say experienced here at UMC . Were there any results in the study that surprised you ?

Alem Zekarias 34:09

Not really . To be honest , it was more or less what we expected . I think there is a significant need , as I said you know , around optimization to support humans in managing and interacting with increasing volume of pharmacovigilance data , but also the handling of this type of data . But what I take away with me is what Joanna also said , like this is you know , this is true data , this is true needs , challenges that are out there . It's hard to see that maybe from sitting here in Uppsala and working with the global and to know that these are actually the struggles . The PV assessors, our colleagues out there that do have and that know that some uses ChatGPT but also other use this and this, was very , of course , informative , but in general , it was also , as you said , we are , we have been working with this for a while . It is also feels confirming that we are doing this in more or less the same way .

Joana Félix 35:04

Yeah , and I think , a key – I don't know if it was surprising again , because I think we feel a lot of these tensions as well when we look at a report – but this tension of the narrative being fundamental and also being the biggest challenge because just how it can be incomplete or redundant . So this tension , I think , is interesting to explore and to think more in the future , because we are saying that the most important thing it's the one that provides you a lot of challenges and a lot of back and forth and a lot of just questioning .

Fredrik Brounéus 35:45

Speaking about the future . This , as you've said , well , it's an exploratory knowledgey , but you have been able to extract quite a lot of well , if not new knowledge, y ou've been able to confirm sort of knowledge that you already had , but perhaps it also gave you some inspiration as to what to look at next when it comes to narratives . ?

Joana Félix 36:10

I think there's some, we sometimes call them low-hanging fruits , but in this sense I think it's more the spread of some concerns, so how spread these concerns are . Even in the literature you can find , for example , there was a mention of a challenge that the assessors use the narratives to rip code , for example , and that is verified in multiple literature articles that have attempted to build automated models to extract from the narratives and fill in in the structured reports . So in a way , I think , if you align what is in the literature and what the assessors experience , I think this is and with new large language models , it could be something for the future .

Alem Zekarias 37:01

And I also think that this study of course , as you said , this is an exploratory study , but also this inspires us to do more of this kind of study for the future before we start working with the research project , try to understand the problem itself first , talking to , as I said , stakeholders , colleagues out there , and try to really understand their problem in their settings and , based on that , see what we at the Uppsala Monitoring Center can do to help them .

Fredrik Brounéus 37:26

Okay , thank you very much for coming to the show .

Joana Félix 37:28

Thank you .

Joana Félix 37:29

Thank you so much .

Fredrik Brounéus 37:30

I learned a lot today .

Joana Félix 37:31

Yeah good .

Fredrik Brounéus 37:32

I hope to have you here soon again .

Outro

Alem Zekarias 37:39

Thank you so much .

Joana Félix 37:40

Thank you, thank you for your time

Fredrik Brounéus 37:43

And if you'd like to learn more about Alem's and Joanna's study or how to write useful narratives , check out the links in the episode show notes . Apart from these conversations with experts , we offer a series of audio stories from Uppsala Monitoring Center's pharmacovigilance news site called Uppsala Reports Long Reads , so why not check that out too ? That's all for now , but we'll be back soon with more on medicines safety . In the meantime , we'd love to hear from you . Reach out on Facebook , linkedin and X , send comments or suggestions for the show or questions for our guests next time we open up for that and visit our website to learn more about what we do to promote safer use of medicines and vaccines for everyone, everywhere . If you like the podcast , please subscribe to make sure you won't miss an episode; and spread the word so other listeners can find us too . For Drug Safety Matters, I'm Fredrik Brouneus . Thanks for listening .