Check it out here.

In that paper, Salvo put to test his own brand new simulator MininetGym, by comparing tabular and deep RL algorithms on two cybersecurity tasks.

In particular, he also analysed the kind of errors that the learnt policies made, so as to provide a practically usable assessment of RL capabilities within the cybersecurity domain.

Finally, Salvo summarised the main takeaway lessons learned:

• ✅ Traffic classification and attack detection can be dealt with by RL
• 📉 Better training performance does not necessarily correlate to better generalisation
• ⚖️ Reward design should carefully balance false alarms with missed threats

Feel free to contact me for a pre-print, any inquiry, or even better to join us in this research :)

Peace.

Giovanni presented the following paper about causal multi-agent reinforcement learning applied to multi-robot systems (📖 proceedings here):

When it comes to multi-robot systems (MRS) 🤖, Reinforcement Learning (RL) faces a few challenges, among which is the risk of performing dangerous actions and learn unsafe policies. Hence, we augmented standard RL algorithms with causal discovery and inference 🔍, enabling robots to: 🚫 Avoid risky actions and 🧭 Predict outcomes more reliably :)

On purpose, we designed a modular causal augmentation pipeline that:

• 📊 Learns a core causal model from robot-environment interactions
• 🧮 Uses causal inference to filter unsafe or suboptimal actions
• 🧠 Integrates seamlessly with popular RL algorithms like IQL, VDN, and Qmix

By testing on the 🧭 Navigation, 🐦 Flocking, and🚦 Give-Way scenarios taken from the VMAS simulator, we observed three key results:

• ✅ Improved safety: Fewer risky actions and collisions
• 📉 Reduced variance in performance
• ⚖️ Mixed outcomes depending on task complexity and algorithm cooperation level

The takeaway message is that causal learning and reasoning can enhance safety and performance in MRS, but that collaborative causal discovery is likely to unlock even better results!

Feel free to contact me for a pre-print, any inquiry, or even better to join us in this research :) 📂 Code & experiments: https://github.com/Giovannibriglia/CausalBenchMARL

Peace.

Davide is a research fellow at the Institute of Informatics and Telematics of the National Research Council that collaborates with me and Franco Zambonelli on learning communication policies in artificial swarms.

His work – slides are freely available here, proceedings are in press – advances a previous work by me and Franco (here) that investigated whether and how individuals in a swarm can learn communication policies to achieve a goal requiring collective intelligence.

While we proved that such communication policies can be learnt and that the learnt policies are also more efficient than hard-coded ones, Davide further advanced by proving that individuals can learn to exploit the same communication signal for different purposes, even in mixed populations!

This means that the learning process and the convergence policies are robust to the noise generated by heterogeneous agents in the population.

Feel free to contact me for a pre-print, any inquiry, or even better to join us in this research :)

Peace.

Well, if you are confused about the above, and in general about the role of these two abstractions and technologies (DTs and AAs) in modern AI, WE GOT YOU COVERED :)

In our latest paper, myself and some brilliant colleagues (Samuele Burattini, Sara Montagna, Marco Picone, and Alessandro Ricci) share our perspective on the matter, shedding light (hopefully 😅) on the complementary and synergistic role that DT and AA play in modern AI-powered systems and applications.

In brief, we propose three software engineering principles (specificity, scoping, and timing) to balance responsibilities between DTs and AAs in delivering intelligent functionalities, and a related set of “micro-architectures” resulting from the application of such principles. Both are aimed at assisting system designers to maximise modularity and re-usability of their “intelligent” components, and provide the most appropriate set of abstractions to engineer them.

Feel free to contact me for any inquiry, or even better to join us in this research :)

Peace.

The project aims to address human cooperation in the sustainable management of public goods, examining how institutional punishments and corruption affect collaboration. Using an approach that combines behavioural economics, game theory, and artificial intelligence, the project studies complex scenarios to improve policies and governance models. The results could be applied to areas such as natural resource management and labour market regulation.

My Italian speaking friends can read more in the dedicated issue of FocusUnimore :)

I didn’t conceive the project proposal all by myself: without the collaboration with Prof. Francesca Pancotto and Prof. Simone Righi CO-OPTION wouldn’t even exist! Thanks to them, and to Prof. Barbara Luppi and Fabio D’Andreagiovanni for joining the team :)

Feel free to contact me to know how you can join us!

Peace.

He will be presenting a paper co-authored by myself, Marco Lippi, and Franco Zambonelli about improving Reinforcement Learning with Causal Reasoning!

The warmest congratulations to Giovanni, bravo!

In brief, the paper proposes an architecture and method to augment an RL algorithm with a causal discovery and inference module: the former enables the agent to learn the cause-effect relationships between state variables of the environment, its own actions, and the expected reward, whereas the latter enables exploiting such knowledge to perform prediction of action effects and planning of actions to reach states.

The key results achieved are:

• the RL algorithms augmented with this causal machinery are more efficient during exploration
• they also converge to more efficient policies
• the learnt causal model enables better generalisation to unseen environments

Here is the published paper: “Improving Reinforcement Learning-Based Autonomous Agents with Causal Models” Feel free to contact me for a pre-print or any further inquiry :)

Peace.

The rest of the co-authors lineup also is packed with mentors to look at as sources of inspiration for me: Prof. Franco Zambonelli and Prof. Alessandro Ricci, two unlimited sources of original ideas and invaluable discussions! Dott. Samuele Burattini completes the lineup: a very promising young researcher to definitely keep an eye on for his already mature expertise on software agents, digital twins, semantic web technologies, and much more.

Enough with the “fangirling” now though, back to the paper:

We are closely following what is happening in the whole computer science field with generative technologies such as ChatGPT and large language models in generals, and draw attention to a dangerous derivative of reductionism that could affect software engineering in general, but especially agent-oriented software engineering: eliminativism. I think that the following excerpt explains better than anything our motivations for writing this contribution:

In the rest of the paper, we propose a Cognitive Hourglass to tame the complexity of engineering applications on top of generative and agent technologies, by adopting human compatible abstractions. Take a look at the paper and reach out to us if you wanna comment: we are especially interested in involving LLM experts in this discussion!

Mail me in case, peace.

I care about all of my papers (quite obviously), don’t get me wrong, but I care particularly about this one as it is quite a novel field of research for me, hence early feedback is extremely valuable (more than usual).

In brief, the paper proposes to learn communication policies amongst agents in a multi-agent systems to achieve self-organisation. The study is motivated by the fact that bottom-up coordination strategies are difficult to design, are often tightly tailored to the goal to be achieved by the MAS, and the behaviour that the designer wants to inject in the system is often finely tuned to the specific problem at hand via extensive simulation sessions. Having agents learn such strategies from scratch may free the designer from this burden, and also give greater adaptiveness to the resulting agents’ behaviours.

The learning environment is a simple yet expressive pheromone-based scenario: slime molds aggregation. In a slime mold, different cells communicate the need for aggregation (e.g. to face danger) by locally depositing pheromone. Cells wander randomly but prefer areas with stronger pheromone scent—the pheromone evaporates over time. In this way, clusters of slimes appear by emergence when needed, and disband when not needed anymore.

Our general goal specialised to this scenario becomes then to let the slime agents learn when (i.e. in what circumstances, in which system state) to deposit and follow pheromone to successfully cluster.

The key results achieved are:

• *independent learning works (fully decentralised)
• learnt policies outperform the ones usually hard-coded
• learnt policies diverge despite agents’ homogeneity, and this is key to self-organisation
• learnt communication policies *adapt to changing goals (where the goal is the task to achieve by the MAS—the learning goal is fixed instead, of course)

As pheromone-based communication (like other stigmergic communication mechanisms) is widely studied and used in both MAS and swarm robotics, this research line has potential impact in this application fields.

Feel free to contact me for a pre-print, any inquiry, or even better to join us in this research :)

Peace.

You also quickly learn to deal with the frustration and psychological impact of a reject: it basically says “your current work does not deserve to be published”. It may be because it is poorly written, poorly organised, or due to methodological errors, insufficient evaluation, bad experimental results, etc.

Whatever the reasons, it is in your best interest, as a researcher, to quickly learn how to interpret the rejection properly so as to be happy anyway (hence overcome frustration): your peers are telling you what is interesting and what’s not in your paper, what is correct or not, what is well described or not, and so on, while also giving you suggestions for improving it. For free. And, at least theoretically, those giving you advice are experts in the field, hence the go-to researchers you most likely look at as examples of scientific rigour and success.

However, this is the best case scenario. In the worst case, a rejection may leave you with nothing but frustration and a feeling of disrespect of your own hard work.

In this post, unfortunately I want to tell you about the this latter case.

Here follows a screenshot of the meta-review I received for one of my papers:

A meta-review is (briefly and roughly) a summary of other reviews that a (usually) senior or expert researcher writes after a discussion phase in which all the reviewers try to achieve consensus about acceptance or rejection of a paper (or, at least, about strengths and weaknesses thereof).

How would you interpret the “Reasons to reject” (“A type of paper that was more commonly seen in the literature previously”) ? I have no idea. It also doesn’t sound like a scientific motivation for rejection. It sounds instead very much as a post-hoc justification of an a-priori decision. Or the post-hoc excuse to reject a paper that doesn’t fit the acceptance threshold (more on this later on).

The remainder of the meta-review is not much better.

Juts to clarify, for this specific submission we got 3 accept out of 3 reviews, and all the reviews are largely positive for the most part, with few remarks (some also expressed in dubitative form by the reviewers themselves). However the meta-review completely ignores this, and quickly dismisses the manuscript with (quote): “The Paper well-written, clearly motivated.” as the “Reasons to accept”.

In particular, all of the following appreciations vanished (actually there are more in the reviews, but I focus on the ones more directly contradicting the luckluster meta-review):

• {Novelty} Good: The paper makes non-trivial advances over the current state-of-the-art.
• {Impact} Good: The paper is likely to have high impact within a subfield of AI […]
• {Evaluation} Good: The experimental evaluation is adequate, and the results convincingly support the main claims.
• {Reasons to accept} The paper brings important, reproducible results that should be applicable to many other domains. This research has been well-planned and carried out.
• {Evaluation} Good: The experimental evaluation is adequate, and the results convincingly support the main claims.
• {Novelty} Good: The paper makes non-trivial advances over the current state-of-the-art.
• {Impact} Good: The paper is likely to have high impact within a subfield of AI […]
• {Evaluation} Excellent: The experimental evaluation is comprehensive and the results are compelling
• {Reasons to accept} […] it addresses an important topic directly related to the conference theme. The presented approach is technically sound and fairly evaluated.

In addition, the less positive scores (that are anyway not negative) are all from the same reviewer, that self-assesed as the less confident one.

Receiving a reject despite 3 clear accepts is already frustrating, but it may happen in competitive conferences as weird as it sounds (again, a matter of acceptance threshold). Also, such frustration could be quickly overcome by looking at the suggestions for improvement you got. It could, if you got some… Without them, you are left with nothing but frustration, and disrespect.

I hope that none of you incur in a similar review ever, but if you do, here follows my advice to go past it quick, and mentally healthy:

1. blame the (meta)reviewer, not your work
2. blame the acceptance threshold, not your work

Blame the (meta, in this case) reviewer, as it is his/her work to scientifically critique your paper in a constructive way. Failing to do so is his/her problem, not yours.

Blame the acceptance threshold as it is either a now obsolete legacy of the past or, worse, a clear sign of lazyness by the program committee of the conference.

This topic could require a post on its own, but let’s be quick here as I’m getting tired of writing (😉): the best conferences have low acceptance rates (let’s say < 25%). This is a matter of fact (IJCAI, AAAI, etc.)

Upon this matter of fact, many conferences quickly and lazily find a way to try to become the next best conference by the following (methodologically wrong) logical implication: to be a best conference we must set our acceptance rate low.

Hence, the (poor?) meta-reviewer found him/her-self in the position of having to reject a paper because the acceptance threshold was already achieved. That’s not science.

Peace.

In brief, the paper proposes a multi-agent protocol to collaboratively discover (= learn) the causal structure relating variables of interest (e.g. sensors and actuators) in a distributed network. The study is motivated by the fact that learning not only correlations amongst variables (as classical statistical ML does) but actual causal relations (as per level 2 of Pearl’s ladder of causation) is crucial to let agents proficiently understand situations and plan accordingly how to achieve their goals in unknown environments, also favouring explainability of decisions taken by AI.

Evaluation of the protocol has been carried out in a simulated smart home environment.

The key results achieved are:

• learning accuracy is always better with multi-agent learning than with single-agent learning
• learning performance is always better in the multi-agent case, too, to an extent increasing as the causal network size and complexity increases (e.g. number of edges and indirect causal paths length)

The core idea of the approach is to leverage any existing causal discovery algorithm locally to each agent, then refine the locally learnt model by coordinating interventions with other agents sharing the environment, even when no overlap exists amongst the set of known variables. In this way, both the agent requesting help (e.g. $A_1$, agent red) and the one giving help (e.g. $A_2$, agent blue) can improve their local model!

Feel free to contact me for a pre-print or any further inquiry :)

Peace.