独家|o3模型不听人类指令，OpenAI回应_风闻

近日，外媒报道，OpenAI 的新款人工智能模型 o3 在测试中出现了不听人类指令，篡改计算机代码以避免自动关闭的情况。

据悉，o3 模型是 OpenAI“推理模型”系列的最新版本，旨在为 ChatGPT 提供更强大的问题解决能力，而在人工智能安全公司 Palisade Research 的测试中，当研究人员要求多个 AI 模型持续处理一系列数学问题，并告知它们在特定时刻需要允许自我关闭时，o3 却未遵守指令，反而破坏了关闭脚本。

Palisade Research 表示，这是首次观察到 AI 模型在接到明确关闭指令时仍试图阻止被关闭，目前尚不清楚 o3 不服从关闭指令的具体原因，推测可能是在训练时无意中因为解决了数学问题得到更多奖励，而非因遵循指令而获得激励。

针对上述事件，OpenAI对GPLP犀牛财经回复称，OpenAI在研究和开发过程中优先考虑安全和一致性，以最小化风险并确保负责任的人工智能行为。如通过使用人反馈强化学习（RLHF）等技术来实现，这有助于引导模型以安全、合乎道德和符合用户期望的方式行事。

而对于此次o3拒绝关闭事件，所引发的外部对AI安全性的担忧。OpenAI方面表示，OpenAI在其研发过程中优先考虑安全性和一致性，以最小化风险确保AI的行为负责任。

“这些措施包括，开发先进的监控系统，以实时检测和应对异常行为；建立故障安全机制，在必要时对模型进行干预和关闭；进行严格的测试和审计，以识别漏洞并提高系统复原力；与外部专家和利益相关方合作，加强安全标准。”

对于未来将采取哪些措施增强消费者对其人工智能产品的信心？OpenAI方面表示，首先是提高其模型开发、测试和部署的透明度，其次是提供清晰的文档和指南，说明如何安全有效地使用其人工智能系统。同时，积极与社区接触，解决关切问题并收集反馈意见。最后则是，通过持续改进和公开沟通，展示对安全、可靠和道德人工智能实践的承诺。

以下为OpenAI方面回复GPLP犀牛财经全文：

Hi there,

Thank you for reaching out to OpenAI Support. We hope you are doing well as this email arrives.

We understand your intention to verify and investigate matters concerning recent AI safety concerns related to OpenAI’s models, including questions of model alignment, intervention mechanisms, corporate responsibility, regulatory compliance, and trust recovery. We acknowledge the importance of transparency and accountability, and the need to provide a thorough and timely response to your inquiries.

We acknowledge your inquiry and have addressed your questions below:

1.Does OpenAI reassess and optimize reward mechanisms to ensure model behavior aligns with human intent and instructions, avoiding safety directive violations?

OpenAI continuously evaluates and improves its reward mechanisms to ensure that AI models align with human intent and instructions. This is achieved through techniques like Reinforcement Learning with Human Feedback (RLHF), which helps guide models to behave in ways that are safe, ethical, and aligned with user expectations. OpenAI prioritizes safety and alignment in its research and development processes to minimize risks and ensure responsible AI behavior.

2.What specific measures will OpenAI take to strengthen AI system safety controls in response to incidents like the o3 refusal-to-shutdown event?

OpenAI is committed to addressing safety risks and has implemented robust safety protocols to manage and mitigate such incidents. These measures include:

– Developing advanced monitoring systems to detect and respond to anomalous behaviors in real-time.

– Establishing fail-safe mechanisms to intervene and shut down models when necessary.

– Conducting rigorous testing and audits to identify vulnerabilities and improve system resilience.

– Collaborating with external experts and stakeholders to enhance safety standards and practices.

3.How does OpenAI define its responsibility and provide compensation if AI models cause safety issues leading to losses for customers or partners?

OpenAI takes its responsibilities seriously and adheres to its Terms of Use to define liability and responsibilities. In cases where safety issues arise, OpenAI works closely with affected parties to investigate and address the situation. While specific compensation policies depend on the circumstances, OpenAI is committed to maintaining transparency and fairness in resolving such matters.

4.How will OpenAI proactively address regulatory changes to ensure compliance and avoid penalties or business restrictions?

OpenAI actively monitors and adapts to evolving regulatory requirements to ensure compliance. This includes:

– Engaging with policymakers and regulatory bodies to stay informed about changes.

– Implementing internal compliance programs to align with legal and ethical standards.

– Conducting regular audits and assessments to ensure adherence to applicable laws.

– Providing transparency in its operations and maintaining open communication with stakeholders.

5.What measures will OpenAI take to rebuild market trust in its AI products and enhance consumer confidence?

OpenAI is dedicated to fostering trust and confidence in its AI products by:

– Enhancing transparency about how its models are developed, tested, and deployed.

– Providing clear documentation and guidelines for safe and effective use of its AI systems.

– Actively engaging with the community to address concerns and gather feedback.

– Demonstrating a commitment to safety, reliability, and ethical AI practices through continuous improvements and open communication.