unique glen
U.S. Department of Commerce’s National Telecommunications and Information Administration (NTIA) launched a Request for Comment on the risks, benefits and potential policy related to advanced artificial intelligence (AI) models with widely available model weights.
Comments are due within 30 days of publication of the Request for Comment in the Federal Register.
Press release: https://www.commerce.gov/news/press-releases/2024/02/ntia-solicits-comments-open-weight-ai-models
feral kernel
Project Tracker: https://maximolog.notion.site/NTIA-Solicits-Comments-on-Open-Weight-AI-Models-due-within-30-days-2751cb96888f477a83570cafe58fc297?pvs=4
cedar cobalt
anyone working on this, or interested in doing so? I'm thinking about it...anything I submit will not be remotely authoritative or comprehensive, but it could be worthwhile
cedar cobalt
very rough list of materials that might be relevant in preparing a comment (some that i'd want to quote/cite explicitly, others that i'd just want to keep in mind):
RAND report that I haven't read yet, no doubt someone at NTIA is reading it but maybe there are parts worth drawing their attention to: https://www.rand.org/pubs/working_papers/WRA2849-1.html
https://arxiv.org/abs/2310.20624 on unRLHFing Llama 2-Chat and associated LW posts (https://www.lesswrong.com/posts/qmQFHCgCyEEjuy5a7/lora-fine-tuning-efficiently-undoes-safety-training-from#comments, https://www.lesswrong.com/posts/3eqHYxfWb5x4Qfz8C/unrlhf-efficiently-undoing-llm-safeguards#comments)
Gladstone report mentions some concerns with this briefly in section 1.5.1: #1216810494992847019 message
Anthropic RSP is evidence they take it seriously: https://www.anthropic.com/news/anthropics-responsible-scaling-policy
for the proposition that "some regulation (especially restricting the publication of AI research and sharing of model weights) would differentially slow foreign AI progress!" - https://www.lesswrong.com/posts/YguseW2zMYe8tMCbW/cruxes-on-us-lead-for-some-domestic-ai-regulation
Paul Christiano has a couple of thoughts re: OpenAI preparedness framework and model weights: https://www.lesswrong.com/posts/G2AghCGjGCkhWiFDd/openai-s-preparedness-framework-praise-and-recommendations
Zvi has a little discussion here: https://www.lesswrong.com/posts/emo2hAvq6p7Pn4Pps/ai-42-the-wrong-answer#Open_Foundation_Model_Weights_Are_Unsafe_And_Nothing_Can_Fix_This
some discussion on a couple PauseAI pages: https://pauseai.info/cybersecurity-risks#mitigating-ai-cybersecurity-risks, https://pauseai.info/scenarios#cyberterrorism
tagging @worthy blaze since you seemed to feel strongly about this a while back (#policy-🏛 message) - have you done any writing or research toward the RFC that you'd like to share?
cedar cobalt
here's what I've got so far. Hoping to add some more on other parts of questions 2, 5, 7, and 8, but I'm not sure how much I'll get done before the deadline or how long a comment it's useful to submit. I'm very interested in feedback on what I have so far, suggestions, or best of all, inspiring others here to submit their own comments
so far this is all very much in harm-mitigation mode, though I hope to put in a few words for a more general pause in a response to a later question
unique glen
🚨 Written comments must be received on or before March 27, 2024.
cedar cobalt
this paper from Qi et al is imo more impressive (i.e. scary w.r.t. the risks of LLM fine-tuning) than the Llama UnRLHF paper I linked above, but it doesn't focus solely on open foundation models - they get similar results from fine-tuning GPT 3.5 Turbo via the provided APIs
https://arxiv.org/abs/2310.03693
I am planning to mention it but not sure how much to focus on it/whether it would be worth someone else focusing a comment on it specifically. I feel like it's a useful part of a case for pausing, but I'm not sure if it's useful for discussion of public model weights specifically (and maybe it's even counterproductive, if it persuades them that closed models aren't any better)
I think the thing to emphasize is maybe that with closed models the foundation model developer still has some chance to do something about the problem - they can shut down fine-tuning APIs if they're being misused, or try to better control how the APIs are used [to be clear I am not optimistic about any of this as a general alignment strategy, but it seems better than not being able to do that] - whereas with public model weights there's no such opportunity
worthy blaze
tagging <@809384056927420427> since you seemed to feel strongly about this a whi...
Yeah, I haven't really worked on this tbh. Sorry, I'm spread pretty thin atm
cedar cobalt
No worries, just figured it was worth asking!
queen minnow
AI-Plans is hosting an event for this starting on <t:1711468800:F>:
https://lu.ma/RFC-Law-a-Thon
(Sorry that it's a little last-minute!)
cedar cobalt
won't be able to make it, but I'm glad you're putting that event together!
unique glen
Great event! Sorry, it’s outside my time zone so I won’t be able to attend.
Best for Americans to attend this one since NTIA is a U.S. agency
cedar cobalt
here's what I have at this point, definitely still interested in feedback but not likely to add a ton of content now
I am concerned that the paragraph where I advocate for a pause (7a) actually makes the whole thing worse in terms of advocacy for caution with model weights
cedar cobalt
For anyone who went to the AI-Plans event on this RFC, do you have any info/takeaways from it that you'd like to share here?
unique glen
One day left for comments
https://x.com/daniel_271828/status/1772803359285903538?s=46&t=_xX_kazwTFLs2wukisAuLg
Twitter
unique glen
For anyone who went to the AI-Plans event on this RFC, do you have any info/take...
@queen minnow, @serene magnet
cedar cobalt
Just submitted my comment. Note that the comment field on regulations.gov is limited to 5000 characters - I put an abridged version there and my full comment as an attachment
serene magnet
we'll do another law-a-thon for this
serene magnet
For anyone who went to the AI-Plans event on this RFC, do you have any info/take...
in my personal opinion, the previous RFC (haven't looked at this one yet) showed that whoever wrote it basically has no idea what they're talking about
publishing the results of the law-a-thon in a few mins
queen minnow
in my personal opinion, the previous RFC (haven't looked at this one yet) showed...
Not sure what you mean by previous RFC and this RFC. Isn't there only one, due today?
queen minnow
This is the one for March 27th. I haven't heard of any others. Those might need their own project post, or perhaps this post could be converted to being about RFCs in general.