A Deep Dive into FIL-RetroPGF-2 Results

The Filecoin network recognizes that public goods are essential to a healthy ecosystem. To support these often underfunded efforts, Filecoin runs a RetroPGF program, initially modeled after the Optimism RetroPGF, in which public goods are funded based on a retroactive impact assessment by badgeholders.

FIL-RetroPGF-2 is the second instance of the RetroPGF effort in Filecoin. In it, projects were asked to apply under one of the following categories: Infrastructure & Dependencies, Tooling & Utilities, Education & Outreach, Protocol R&D, Governance, and Product & UX. Badgeholders then voted to distribute to these projects 270k FIL donated by Holon Global, Filecoin Foundation and Protocol Labs, based on their assessment of retroactive impact that projects provided to the Filecoin ecosystem between April — September 2024.

This document deep-dives into the funding allocated to projects in FIL-RetroPGF-2. We begin by summarizing the results of Round 2. We then compare the results to Round 1, and conclude with some exciting developments anticipated for Round 3.

Project Funding Breakdown

The top 5 projects to receive funding were:

1. Curio — 14645.92 FIL
2. Lighthouse — 9140.97 FIL
3. GLIF Nodes & RPC API Service — 8740.62 FIL
4. Filecoin Spark — 8317.24 FIL
5. Interplanetary Network Indexers — 7755.15 FIL

These projects captured 18% of the total funding available in Round 2.

The mean and median projects were allocated 2786.60 FIL and 1986.49 FIL, respectively, and the top-scoring project was allocated 14645.92 FIL.

See here for a full results breakdown.

Fig 2 shows how votes and funding were allocated across the different categories. Projects in the Infrastructure and Dependencies and Protocol R&D categories were funded above average. Projects in all other categories received less than average funding per project.

Fig 2: Aggregate view of funding allocation

Ballots Cast

Fig 3 shows information about badgeholder voting patterns. The most common number of ballots a project received was 4, and the minimum was 1. 63.2% of projects that submitted applications were eligible for funding because they met quorum requirements (at least 6 ballots cast). No projects were eliminated due to the minimum funding cutoff of 150 FIL (targeting 1000 USD).

Fig 3: Histogram and CDF of the number of ballots cast per project

Fig 4 shows the ballots cast by the badgeholders for each project. It is sorted by the number of ballots each project received. The top 5 projects to receive the most ballots were:

1. Curio — next-gen platform in the Filecoin ecosystem, streamlining storage provider operations.
2. GLIF Nodes & RPC API Service for publicly accessible hosted endpoints for Lotus, the most popular client for the Filecoin network.
3. Lighthouse — Programmable data layer powering AI, DePin, NFTs with decentralized storage and encryption.
4. Filecoin Spark — Spark is a trustless protocol for sampling retrievals from Filecoin Storage Providers.
5. Randamu — Real-world consensus via threshold cryptography and web3 innovation.

Note that four out of the top five funded projects were also in the top five projects that received the most ballots. This alignment indicates strong consensus across all badgeholders, regardless of the amounts allocated, of the impact of these projects on the Filecoin ecosystem.

Fig 4: Project voting patterns by Badgeholders

Fig 5A shows how the funding was distributed across the different projects. Fig 5b shows the rank ordering of projects and the total percentage of funding they received. The Top 2.08% of projects received 10% of the total allocation, the top 8.33% of projects received 25% of the total allocation, the top 23.96% of projects received 50% of the total funding, the top 45.83% of projects received 75% of the total funding. Both of these represent a power law distribution with a heavy tail of funds to projects, indicating that badgeholders were very discerning in their funding allocation decisions.

Fig 5: A) The distribution of FIL allocated across all projects that received funding, and B) the percentage of total allocation parametrized by the project's rank.

Badgeholder Voting Patterns

In FIL-RetroPGF-2, badgeholders were instructed to distribute up to 270k FIL across all eligible projects. The maximum vote allowed was 40k FIL. Fig 6 shows a histogram of the allocation of funds across all projects. The most common amount of funding allocated by badgeholders to projects was 10000 FIL, and the second most common was 1000 FIL.

Fig 6: Allocation Histogram

To examine badgeholder voting patterns in more detail, let us define the temperature of a badgeholder to be related to the number of ballots cast. A cold badgeholder distributes their votes across many ballots. A hot badgeholder concentrates their votes to a few projects. Using this framework, we can rank badgeholders by their "temperature" and observe how they voted across the various projects. This is shown in Fig 7 — the top of the chart shows the coldest badgeholder, and the bottom of the chart shows the hottest badgeholder.

Fig 7 — Badgeholder voting patterns, ordered by coldest to hottest badgeholder (top to bottom)

How reliable is the size of the badgeholder set?

How confident are we in the distribution of funds indicated in Fig 5? We performed a bootstrap analysis by selecting 1000 possible subsets of ballots cast by badgeholders, with a minimum of 28 badgeholders and a maximum of 33. We then compute the confidence intervals of these distributions and overlay them with the actual funds distribution. This is shown below in Fig 8. The IQR indicates that having 33 badgeholders is reasonable at estimating the true signal, since the confidence intervals shown from different partitions of the badgeholders are not very dispersed.

Fig 8: Bootstrapped allocation distribution

Counterfactuals

Considering the presented results, we now perform counterfactual analysis to understand how the allocation distribution would have changed had different scoring rules been applied.

Quorum Size

If quorum was a different size, what would the distribution of funds have been and how many projects would have been funded? Fig 9 shows that there is a sustained but roughly linearly decreasing relationship in the number of projects that are funded as quorum increases, starting at 4. As quorum decreases, the distribution of funds is flattened, but note that regardless of the quorum value, the funds remain relatively exponentially distributed — this is a positive signal that badgeholders in general have been discerning in funding allocations.

Fig 9: Allocation distribution as a function of Quorum

Scoring Mechanism

In FIL-RetroPGF-2, we used a sum scoring rule — in contrast to FIL-RetroPGF-1 where we used mean scoring. The rationale for this change was twofold: i) to make the voting mechanism easier for badgeholders to understand, and ii) to make the left-hand cusp sharper so several projects were funded in significant amounts, in addition to having a long righthand tail.

How would the distribution of funds change had we used alternate scoring? Fig 10 shows how the funding distribution would have changed. We observe that the sum scoring rule provides a more power-law scoring distribution.

Fig 10: Allocation distribution as a function of scoring function

Hot/Cold Badgeholders

We previously introduced the concept of the temperature of a badgeholder. What if we remove the top 10% of cold badgeholders (voting diffusely) and the top 10% of hot badgeholders (voting concentratedly). How would the distribution of funds change? Fig 11 shows that had we removed both the most concentrated and most diffuse badgeholders, the funding distribution would be even more exponential than it already is. Removing only the hottest or coldest badgeholders independently would not have altered the distribution as much.

Fig 11: Distribution of funds when removing hot and cold badgeholders

Effect of UX on Votes + Distribution

We updated the Gitcoin easy-retropgf software to randomize the order in which projects were shown to badgeholders, to combat any potential bias. Although badgeholders had the option of using a static CSV to upload the results (which necessarily was not randomized), we posit that at least some of the badgeholders used the web interface.

Aggregating across all of these votes, we used MCMC to infer the distribution of the correlation between the letter that a project starts with, and the number of votes as well as the total funding allocated to each project. This is shown in Fig 12A and 12B. We observe a very small negative correlation between the project starting letter and the number of votes it received, and no significant correlation between the project starting letter and the funding that the project received. This means that the small alphabetical ordering effect on allocation observed in round 1 has now effectively been removed.

Fig 12: Correlation between the project starting letter and A) the number of votes it received and B) the total funding allocated to that project

Comparison to Round 1

There were several significant updates to FIL-RetroPGF-2:

1. The scoring function was updated to use sum, rather than median.
2. Badgeholders were asked to allocate actual votes rather than percentages.
3. Quorum was increased from 5 to 6, since the number of badgeholders who participated in the process increased.
4. Updates to the badgeholder guidance documentation on high-impact areas for the network to develop.

What effect did these changes have on the funding allocations? The summary differences between the funding allocation in Round 1 and 2 is shown below in Table 1, normalized by the total funding amount which is expected to be on an increasing trajectory as rounds progress.

The most notable difference is that top projects received more funding in Round 2 than in Round 1. This is also shown in the allocation distribution in Fig 5, which is much more exponentially distributed than what was experienced in FIL-RetroPGF-1. We can attribute this directly to the change in the scoring function and quorum, as observed in Fig 9 & 10. It is also possible that asking badgeholders to vote with actual allocations rather than percentages influenced this change, since it may feel more intuitive to assign a monetary value to a project rather than a percentage.

Table 1: Differences in summary statistics of funding allocations between Round 1 and Round 2

Conclusion

FIL-RetroPGF-2 allocated 270,000 FIL to 97 projects. Badgeholders showed careful judgment and due diligence in their allocations. This is evidenced by a power-law distribution of funds in Fig 5A. Furthermore, the strong correlation between the top-scoring and top-voted projects indicates a clear alignment amongst badgeholders in identifying projects most impactful to the Filecoin ecosystem.

Looking longitudinally between FIL-RetroPGF-1 and FIL-RetroPGF-2, we observe the following. 77 projects applied in both Round 1 and Round 2; of that set, 60 received funding in Round 2. Those 60 projects accounted for approximately 248k FIL or ~92% of the total available funding. This is positive since sustained impact to the ecosystem is valuable and necessary. However, it is possible to tailor the FIL-RetroPGF efforts to spark more innovation in the ecosystem. This could manifest as more diversity and turnover in projects funded across rounds.

There are several promising developments in the Filecoin ecosystem that will enable better tracking and assessment of project funding, which can lead to a more dynamic ecosystem. The introduction of stablecoins will facilitate on-chain payments and Drips will improve funding flow transparency. These tools will enable badgeholders to more accurately evaluate the impact-profit gap — a metric that was difficult to assess due to a lack of accurate information in Round 1 and 2. As we look forward to Round 3, these improvements will enable a more effective allocation of resources by the FIL-RetroPGF program across the ecosystem.

Thank you to the Filecoin ecosystem, nominators, applicants, software providers, donors and badgeholders for making FIL-RetroPGF-2 happen. Be on the lookout for FIL-RetroPGF-3 in early 2025!!

If you're interested in working with us on RetroPGF, tokenomics, mechanism design, quantitative modeling, or other related subjects, we'd love to hear from you. Please reach out to us at advisory@cel.build.

References

1. Anonymized Raw Results
2. Notebook to produce plots

---

This research was conducted by CryptoEconLab as part of our ongoing work in governance mechanism design and RetroPGF analysis. For more insights on crypto economics and protocol design, explore our other publications and case studies.